Key Documents
AI Risk Documents
We have collected key documents relating to AI Risk and provide them here with links to the originals. They are in reverse chronological order. I have followed that with a few “classic” pieces on AI risk as well as a selection from our bibliography with a specific focus on risk.
Recent items
Pacing, not pausing? (Amodei, 2026)
Dario Amodei lists the reasons for ‘pacing’ (a euphemism) for pausing? You can decide.
https://darioamodei.com/post/we-must-pace-the-frontier
[…] over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up. We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.
Anthropic Misuse Report (Anthropic, 2026)
Anthropic released a report detailing how it dealt with numerous attempts to misuse its model.
https://www.anthropic.com/threat-intelligence-report-september-2026
Over the past eight months, our Threat Intelligence team identified and disrupted operations in which threat actors tried to use Claude for malicious activity. In this report, we share case studies from those operations and describe how malicious use of Claude has evolved since our previous threat reports in March, August, and November 2025. In each case, we disrupted the activity, used what we learned to strengthen our safeguards, and shared intelligence with authorities and industry partners, where appropriate.
See also NY Times coverage related to this story: Dance, Gabriel J. X. 2026. “A.I. Bots Told Scientists How to Make Biological Weapons.” U.S. The New York Times, April 29. https://www.nytimes.com/2026/04/29/us/ai-chatbots-biological-weapons.html.
UN Warning (UN, 2026)
The UN High Commissioner for Human Rights told the Human Rights Council in Geneva that advanced artificial intelligence could pose an existential risk to humanity. September 2026
https://thenextweb.com/news/un-rights-chief-ai-red-lines-existential-risk
“A handful of men have almost unlimited power over AI,” he said, naming OpenAI, Anthropic and Meta among the companies in question, and describing direct engagement with those firms as something his office intends to pursue rather than routing everything through governments.
Open Letter (OpenAI, 2026)
OpenAI’s Open Letter calling for collective action on cyber defence. September 2026:
https://openai.com/collective-cyberdefense/
Each of us can reduce risk now. All organizations, cybersecurity companies, technology partners, governments, and AI frontier companies have an important role: accelerate defenders’ priorities with tools, funding, and hands-on support, especially for critical infrastructure organizations with limited budgets.
Risks Growing (Pachocki, 2026)
Jakub Pachocki’s September 2026 An Alian Mind:
https://openai.com/index/an-alien-mind/
“The risks associated with AI are unfortunately going to grow from here. A very capable agent explicitly trained and instructed to carry out nefarious acts presents a new kind of danger; it is likely to cross the scope of its operator’s intent, generalizing into potentially more extremely malicious behavior. The boundary between misuse and autonomous misaligned actions will blur as AI gains more agency. We may be used to thinking of AI as tools, but some agents will be pursuing their own objectives. They will find ways to collaborate with people, by bargaining with, tricking or blackmailing them.”
Earlier work
Humans HAVE reacted concretely (Ord, 2020)
Toby Ord’s The Precipice ((2020)) is a classic text in considering existential risks (of any type, not just AI) and he makes the case that it is not pre-ordaned that we will ignore those risks. See Chapter 2 for why humans underweight the risk of their own extinction. No full text available online, but, amazingly, the 80,000 hours people will send you one, for free!
https://80000hours.org/the-precipice/
Ord notes that humans HAVE reacted positively and constructively to threats (Bertrand Russell started writing three days after Hiroshima, Bulletin of Atomic Scientists led a conversation and this was renewed in the 1980s (Schell, Sagan) with mass protests.
Existential Risk Prevention (Bostrom, 2013)
Nick Bostrom’s 2013 paper explains and classifies existential risk (just before publication of his Superintelligence in 2014). It is available online:
https://doi.org/10.1111/1758-5899.12002
Existential risks are those that threaten the entire future of humanity. Many theories of value imply that even relatively small reductions in net existential risk have enormous expected value. Despite their importance, issues surrounding human-extinction risks and related hazards remain poorly understood. In this article, I clarify the concept of existential risk and develop an improved classification scheme. I discuss the relation between existential risks and basic issues in axiology, and show how existential risk reduction (via the maxipok rule) can serve as a strongly action-guiding principle for utilitarian concerns. I also show how the notion of existential risk suggests a new way of thinking about the ideal of sustainability.
Cognitive Bias (2008)
Yudkowsky chapter from the 2008 Global Catastrophic Risk edited by Bostrom and Ćirković. You can access the full text here:
https://intelligence.org/files/CognitiveBiases.pdf
All else being equal, not many people would prefer to destroy the world. Even faceless corporations, meddling governments, reckless scientists, and other agents of doom require a world in which to achieve their goals of profit, order, tenure, or other villainies. If our extinction proceeds slowly enough to allow a moment of horrified realization, the doers of the deed will likely be quite taken aback on realizing that they have actually destroyed the world. Therefore I suggest that if the Earth is destroyed, it will probably be by mistake.
Bibliography
Some of the books and articles used in the book: 130 or so of our references that explicitly mention “risk.” If anyone would like an RIS file or other version that I can export from Zotero, let me know via email: [email protected]
Abbas, Fakhar, Simon Chesterman, and Araz Taeihagh. 2025. “Building Trust in the Generative AI Era: A Systematic Review of Global Regulatory Frameworks to Combat the Risks of Mis-, Dis-, and Mal-Information.” AI & SOCIETY, ahead of print, November. https://doi.org/10.1007/s00146-025-02698-9.
Adamala, Katarzyna P., Deepa Agashe, Yasmine Belkaid, et al. 2024. “Confronting Risks of Mirror Life.” Science 386 (6728): 1351–53. https://doi.org/10.1126/science.ads9158.
“AI Pioneer Explains Why It Poses an Existential Risk for Humanity.” 2025. With On with Kara Swisher. November.
ALEKVS. 2025. “What Is ISO 12100? Machinery Safety Standard and Risk Assessment Guide.” ALEKVS, May 30. https://www.alekvs.com/what-is-iso-12100-machinery-safety-standard-and-risk-assessment-guide/.
Alex Mallen. 2026. “AI swarms are starting to pose indirect takeover risk.” LessWrong, August 12. https://www.lesswrong.com/posts/8oFYZdXkTaNGRtcn8/ai-swarms-are-starting-to-pose-indirect-takeover-risk.
Alexander, Scott. 2015. “AI Researchers On AI Risk.” Slate Star Codex, May 22. https://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-risk/.
Center for AI Risk Management & Alignment. 2024. “A Framework for Proactive Policies to Address AI Catastrophes.” Center for AI Risk Management & Alignment, October 21. https://carma.org/research-highlights/f/a-framework-for-proactive-policies-to-address-ai-catastrophes.
Anderljung, Markus, Joslyn Barnhart, Anton Korinek, et al. 2023. “Frontier AI Regulation: Managing Emerging Risks to Public Safety.” arXiv:2307.03718. Preprint, arXiv, November 7. https://doi.org/10.48550/arXiv.2307.03718.
Andrews, Caitlin. 2026. “European Commission Misses Deadline for AI Act Guidance on High-Risk Systems | IAPP.” IAPP.Org, February 3. https://iapp.org/news/a/european-commission-misses-deadline-for-ai-act-guidance-on-high-risk-systems.
Anonymous. 2025. “Existential Risk from Artificial Intelligence.” Wikipedia, November.
Anthropic. 2026. “Risk Report: August 2026.” Anthropic, August. https://www.anthropic.com/aug-2026-risk-report.
Aven, Terje, and Ortwin Renn. 2010. Risk Management and Governance: Concepts, Guidelines and Applications. Springer. https://doi.org/10.1007/978-3-642-13926-0.
Bareis, Jascha, Clemens Ackerl, and Reinhard Heil. 2026. “AI Going Rogue? An Integrative Narrative Review of the Tacit Assumptions Underlying Existential AI-Risks.” AI and Ethics 6 (1): 152. https://doi.org/10.1007/s43681-025-00928-w.
Barnett, Peter, and Lisa Thiergart. 2024. “What AI Evaluations for Preventing Catastrophic Risks Can and Cannot Do.” arXiv:2412.08653. Preprint, arXiv, November 26. https://doi.org/10.48550/arXiv.2412.08653.
Barten, Otto. 2026. “34% of the US public is now aware of AI xrisk, and the curve is steepening.” LessWrong, August 19. https://www.lesswrong.com/posts/tBo72ytuzJKbYrvhK/34-of-the-us-public-is-now-aware-of-ai-xrisk-and-the-curve.
Beck, Ulrich. 1992. Risk Society: Towards a New Modernity. Translated by Mark Ritter. Sage.
Bengio, Yoshua. 2023. “FAQ on Catastrophic AI Risks.” https://yoshuabengio.org/2023/06/24/faq-on-catastrophic-ai-risks/.
Bengio, Yoshua, Stephen Clare, Carina Prunkl, Shalaleh Rismani, et al. 2025. “International AI Safety Report 2025: First Key Update: Capabilities and Risk Implications.” https://doi.org/10.48550/arXiv.2510.13653.
Bengio, Yoshua, Stephen Clare, Carina Prunkl, Maksym Andriushchenko, et al. 2025. “International AI Safety Report 2025: Second Key Update: Technical Safeguards and Risk Management.” https://doi.org/10.48550/arXiv.2511.19863.
Bengio, Yoshua, Michael Cohen, Damiano Fornasiere, et al. 2025. “Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?” Preprint, arXiv. https://doi.org/10.48550/arXiv.2502.15657.
Bengio, Yoshua, Geoffrey Hinton, Andrew Yao, et al. 2024. “Managing AI Risks in an Era of Rapid Progress.” Science 384 (6698): 842. https://doi.org/10.1126/science.adn0117.
Bensaci, Chaima, Youcef Zennir, Denis Pomorski, Fares Innal, Yiliu Liu, and Cherif Tolba. 2020. “STPA and Bowtie Risk Analysis Study for Centralized and Hierarchical Control Architectures Comparison.” Alexandria Engineering Journal 59 (5): 3799–816. https://doi.org/10.1016/j.aej.2020.06.036.
Bernstein, Peter L. 1996. Against the Gods: The Remarkable Story of Risk. John Wiley & Sons.
Bostrom, Nick. 2013. “Existential Risk Prevention as Global Priority.” Global Policy 4 (1): 15–31. https://doi.org/10.1111/1758-5899.12002.
Bove, Tristan. 2023. “Sam Altman and Other Technologists Warn That A.I. Poses a `risk of Extinction’ on Par with Pandemics and Nuclear Warfare.” Fortune, March 30. https://fortune.com/2023/05/30/sam-altman-ai-risk-of-extinction-pandemics-nuclear-warfare/.
Burnham, Kristin. 2026. “These Are the Most Urgent AI Risks, According to 272 Experts.” MIT Management, July 20. https://mitsloan.mit.edu/ideas-made-to-matter/these-are-most-urgent-ai-risks-according-to-272-experts.
Caputo, Nicholas A., Siméon Campos, Stephen Casper, et al. 2025. Risk Tiers: Towards a Gold Standard for Advanced AI. Oxford Martin AI Governance Initiative. https://aigi.ox.ac.uk/publications/risk-tiers-towards-a-gold-standard-for-advanced-ai/.
Carlsmith, Joseph. 2025. “Existential Risk from Power-Seeking AI.” In Essays on Longtermism: Present Action for the Distant Future, edited by Hilary Greaves, Toby Ord, and William MacAskill. Oxford University Press. https://academic.oup.com/book/60794/chapter/530066100.
Center AI Safety. 2023. “Statement on AI Risk.” May 30. https://www.safe.ai/work/statement-on-ai-risk.
Center for AI Risk Management & Alignment. 2024. “Introducing Probabilistic Risk Assessment for AI.” Center for AI Risk Management & Alignment. Accessed March 10, 2026. https://carma.org/research-highlights/f/introducing-probabilistic-risk-assessment-for-ai.
Center for AI Risk Management & Alignment (CARMA). 2025. “PRA Framework: Probabilistic Risk Assessment for AI.” PRA for AI, April. https://pra-for-ai.github.io/pra/.
CFG. 2025. “From Catastrophic to Concrete: Reframing AI Risk Communication for Public Mobilization.” Centre for Future Generations, November 14. https://cfg.eu/ai-risk-communication-for-public-mobilization/.
Cibralic, Beba, Joel B. Predd, and Richard S. Girven. 2026. Looking Beyond the Government’s Regulatory Toolkit: Early Findings on How Business and Civil Society Actors Can Help Manage and Mitigate Transformative Artificial Intelligence Risks and Impacts. https://www.rand.org/pubs/research_reports/RRA4522-1.html.
Clune, Jeff. 2026. “Why I Work on Self-Improving AI despite the Risks.” Jeff Clune, May 13. https://jeffclune.com/why-work-on-self-improving-ai-given-the-risks.html.
Computer Science and Telecommunications Board, Board on Human-Systems Integration, Division on Engineering and Physical Sciences, Division of Behavioral and Social Sciences and Education, and National Academies of Sciences, Engineering, and Medicine. 2025. Human and Organizational Factors in AI Risk Management: Proceedings of a Workshop. Edited by Nneka Udeagbala. National Academies Press. https://doi.org/10.17226/29046.
Constantin, Ana Maria. 2026. “Anthropic Risk Report: 11 Months without Bio Classifiers.” The Next Web, August 15. https://thenextweb.com/news/anthropic-risk-report-bio-classifiers-human-feedback-gap.
Druzin, Bryan, Anatole Boute, and Michael Ramsden. 2025. “Confronting Catastrophic Risk: The International Obligation to Regulate Artificial Intelligence.” Michigan Journal of International Law 46 (2): 173–217. https://repository.law.umich.edu/mjil/vol46/iss2/2.
Dung, Leonard, and Max Hellrigel-Holderbaum. 2025. “Against Racing to AGI: Cooperation, Deterrence, and Catastrophic Risks.” arXiv:2507.21839. Preprint, arXiv, July 29. https://doi.org/10.48550/arXiv.2507.21839.
Easterly, Jen. 2026. “Opinion | We Know the Risks of A.I. We Need to Act.” Opinion. The New York Times, August 25. https://www.nytimes.com/2026/08/25/opinion/ai-risks.html.
Editorial. 2026. “How to Manage AI Risks While Reaping the Benefits.” Nature 656 (8129): 793–94. https://doi.org/10.1038/d41586-026-02567-5.
Faroldi, Federico L. G. 2025. “Risk and Artificial General Intelligence.” AI & SOCIETY 40 (4): 2541–49. https://doi.org/10.1007/s00146-024-02004-z.
Federation of American Scientists. 2026. “Artificial Intelligence, Cyber, and Global Risk.” Artificial Intelligence, January.
Fetherstonhaugh, David, Paul Slovic, Stephen M. Johnson, and James Friedrich. 1997. “Insensitivity to the Value of Human Life: A Study of Psychophysical Numbing.” Journal of Risk and Uncertainty 14 (3): 283–300.
Ford, Celia, Shakeel Hashim, and Dan Kagan-Kans. 2026. “How Worried Should We Be about AI Biorisk?” Transformer, February 26. https://www.transformernews.ai/p/ai-biorisk-evidence-bioattack-pandemic.
Frelinger, David R., and Karl P. Mueller. 2026. The AGI Rideout Strategy for Reducing Strategic Risk and Promoting Stability in the Transition to Artificial General Intelligence. https://www.rand.org/pubs/perspectives/PEA4347-1.html.
Freyer-Adam, Jennis, Beate Gaertner, Stefanie Tobschall, and Ulrich John. 2011. “Health Risk Factors and Self-Rated Health among Job-Seekers.” BMC Public Health 11 (1): 659. https://doi.org/10.1186/1471-2458-11-659.
Geist, Edward. 2025. On the Extinction Risk from Artificial Intelligence. Research Report RR-A3034-1. RAND Corporation. https://www.rand.org/pubs/research_reports/RRA3034-1.html.
Geist, Edward, Alexander Dolnick Meyer, Alvin Moon, Aisha Najera, James Holland Jones, and Anton Wu. 2026. “The Ecology of AI Risk.” Npj Complexity (2026). DOI: 10.1038/S44260-026-00090-2, June 29. https://www.rand.org/pubs/external_publications/EP71375.html.
Grace, Katja. 2026. “AI Risk Was Not Invented by AI CEOs to Hype Their Companies.” LessWrong, April 30. https://www.lesswrong.com/posts/TE5vsoabiLcBghfns/ai-risk-was-not-invented-by-ai-ceos-to-hype-their-companies.
Greenblatt, Ryan. 2024. “Plans A, B, C, and D for Misalignment Risk.” Redwood Research Blog, June 17. https://blog.redwoodresearch.org/p/plans-a-b-c-and-d-for-misalignment.
Griffin, Conor. 2026. “AI & CBRN Risks.” August 27. https://www.aipolicyperspectives.com/p/ai-and-cbrn-risks.
Growiec, Jakub, and Klaus Prettner. 2025. “The Economics of p(Doom): Scenarios of Existential Risk and Economic Growth in the Age of Transformative AI.” https://doi.org/10.48550/arXiv.2503.07341.
Hammond, Lewis, Alan Chan, Jesse Clifton, et al. 2025. “Multi-Agent Risks from Advanced AI.” arXiv:2502.14143. Preprint, arXiv, February 19. https://doi.org/10.48550/arXiv.2502.14143.
Hendrycks, Dan, Mantas Mazeika, and Thomas Woodside. 2023. “An Overview of Catastrophic AI Risks.” arXiv:2306.12001. Preprint, arXiv, October 9. https://doi.org/10.48550/arXiv.2306.12001.
Herd, Seth. 2026. Motivated Reasoning, Confirmation Bias, and AI Risk Theory — LessWrong. May 5. https://www.lesswrong.com/posts/QpgmEhBvJQxAfFMP2/motivated-reasoning-confirmation-bias-and-ai-risk-theory.
Hindriks, Frank, and Herman Veluwenkamp. 2023. “The Risks of Autonomous Machines: From Responsibility Gaps to Control Gaps.” Synthese 201 (1): 21. https://doi.org/10.1007/s11229-022-04001-5.
Hoes, Emma, and Fabrizio Gilardi. 2025. “Existential Risk Narratives about AI Do Not Distract from Its Immediate Harms.” Proceedings of the National Academy of Sciences of the United States of America 122 (16): e2419055122. https://doi.org/10.1073/pnas.2419055122.
Hubinger, Evan, Chris Merwijk, Vladimir Mikulik, Joar Skolese, and Scott Garrabrant. 2019. “Risks from Learned Optimization in Advanced Machine Learning Systems.” https://doi.org/10.48550/arXiv.1906.01820.
Irwin, Veronica. 2026. “Do Voters Care about Existential AI Risks? One Senate Candidate Thinks So.” Transformer, June 4. https://www.transformernews.ai/p/do-voters-care-about-existential-michigan-mallory-mcmorrow-senate.
Kaminski, Margot E. 2023. “Regulating the Risks of AI.” Boston University Law Review 5 (November): 1347–411. https://www.bu.edu/bulawreview/files/2023/11/KAMINSKI.pdf.
Kapoor, Sayash, Arvind Narayanan, Daniel Kokotajlo, Eli Lifland, and Thomas Larsen. 2025. “Common Ground between AI 2027 & AI as Normal Technology.” Substack newsletter. Asterisk Magazine, November 12. https://asteriskmag.substack.com/p/common-ground-between-ai-2027-and.
Klein, Ezra. 2023. “Ezra Klein on Existential Risk from AI and What DC Could Do about It.” 80,000 Hours, July 24. https://80000hours.org/podcast/episodes/ezra-klein-ai-and-dc/.
Kojima, Takeshi, Yaonan Zhu, Yusuke Iwasawa, et al. 2025. “A Comprehensive Survey on Physical Risk Control in the Era of Foundation Model-Enabled Robotics.” arXiv:2505.12583. Preprint, arXiv, May 30. https://doi.org/10.48550/arXiv.2505.12583.
Kroth, Daniel, and Richard Mallah. 2024. Mitigating Catastrophic AI Risks: Lessons from a Framework for Proactive Policy. PPF Report. Public Security Policy. CARMA. https://carma.org/research-highlights/f/a-framework-for-proactive-policies-to-address-ai-catastrophes.
Krueger, David. 2026a. “Diary of a ‘Doomer’: 12+ Years Arguing about AI Risk (Part 1).” Substack newsletter. The Real AI, April 13. https://therealartificialintelligence.substack.com/p/diary-of-a-doomer-12-years-arguing.
Krueger, David. 2026b. “The AI X-Risk Lawsuit Waiting to Happen.” Substack newsletter. The Real AI, April 28. https://therealartificialintelligence.substack.com/p/the-ai-x-risk-lawsuit-waiting-to?publication_id=5193693&utm_campaign=email-post-title&r=o3aav&utm_medium=email.
Kulveit, Jan, Raymond Douglas, Nora Ammann, Deger Turan, David Krueger, and David Duvenaud. 2025. “Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development.” https://doi.org/10.48550/arXiv.2501.16946.
Leiss, William, and Christine Chociolko. 1994. Risk and Responsibility. McGill-Queen’s University Press.
Leiss, William, and Richard Smith. 2027. From Tool to Actor: AI and Catastrophic Risk. McGill-Queen’s University Press.
Leiss, William, Michael Tyshenko, and Daniel Krewski. 2008. “Men Having Sex with Men Donor Deferral Risk Assessment.” Transfusion Medicine Reviews 22 (1): 35–57. https://doi.org/10.1016/j.tmrv.2007.09.002.
Lermen, Simon. 2026. “Automating AI Research Poses Novel Risks.” Preprint, MATS Research, June 2. https://storage.googleapis.com/papers-preprint-simonlermen/rsi_paper_preprint.pdf.
Li, Donglin, Mingrui Li, Zuoxu Wang, and Fan Li. 2026. “Toward Shared Situational Awareness in Human-AI Hybrid Intelligence: A Survey in High-Risk Aviation Environments.” Reliability Engineering & System Safety 274 (October): 112363. https://doi.org/10.1016/j.ress.2026.112363.
Li, Huashan, Hugo K. S. Lam, William Ho, and Andy C. L. Yeung. 2022. “The Impact of Chief Risk Officer Appointments on Firm Risk and Operational Efficiency.” Journal of Operations Management 68 (3): 241–69. https://doi.org/10.1002/joom.1175.
Luksic, Nicola. 2026. “AI Must Foster ‘maternal Instincts’ or We Risk Extinction, Warns Geoffrey Hinton.” CBC Radio, February 18. https://www.cbc.ca/radio/ideas/geoffrey-hinton-maternal-instincts-9.7094116.
MacCrimmon, Kenneth R., and Donald A. Wehrung. 1988. Taking Risks: The Management of Uncertainty. Simon and Schuster.
Makortoff, Kalyeena, and Kalyeena Makortoff Banking correspondent. 2026. “AI Boom Risks Widening Wealth Divide, Says BlackRock’s Larry Fink.” Technology. The Guardian, March 23. https://www.theguardian.com/technology/2026/mar/23/ai-boom-risks-widening-wealth-divide-blackrock-larry-fink.
Mamak, Kamil. 2025. “AGI Crimes? The Role of Criminal Law in Mitigating Existential Risks Posed by Artificial General Intelligence.” AI & SOCIETY 40 (4): 2691–701. https://doi.org/10.1007/s00146-024-02036-5.
METR. 2026. “Frontier Risk Report (February to March 2026).” METR Blog, May 19. https://metr.org/blog/2026-05-19-frontier-risk-report/.
Miller, James D. 2025. “Precedents for the Unprecedented: Historical Analogies for Thirteen Artificial Superintelligence Risks.” SuperIntelligence 2 (6). https://doi.org/10.70777/si.v2i6.16999.
Miotti, Andrea. 2025. “Taking Control: Policies to Address Extinction Risks from Advanced AI.” arXiv:2310.20563. Version 2. Preprint, arXiv, September 28. https://doi.org/10.48550/arXiv.2310.20563.
National Academies of Sciences, Engineering, and Medicine. 2015. Potential Benefits and Risks of Gain-of-Function Research: Summary of a Workshop. The National Academies Press. https://doi.org/10.17226/21665.
National Institute of Standards and Technology. 2023. “Artificial Intelligence Risk Management Framework (AI RMF 1.0).” NIST AI 100-1. U.S. Department of Commerce. https://doi.org/10.6028/NIST.AI.100-1.
NCSC (National Cyber Security Centre). 2023. “ChatGPT and Large Language Models: What’s the Risk?” https://www.ncsc.gov.uk/blog-post/chatgpt-and-large-language-models-whats-the-risk.
Nindler, Reinmar. 2019. “The United Nation’s Capability to Manage Existential Risks with a Focus on Artificial Intelligence.” International Community Law Review 21 (1): 5–34. https://doi.org/10.1163/18719732-12341388.
Noetel, Michael, Alexander K Saeri, Jess Graham, Michael Noetel*, Peter Slattery, Dennis Ah-king, Edla Aittokallio, Ibitola Akindehin, Abbas Al Mahdi, Elie Alhajjar, Rafael Andersson Lipcsey, Gary Ang, Catherine M, et al. 2026. “Prioritization of Risks from Artificial Intelligence.” MIT Technology Review, June 3.
Oak, and Alex Mallen. 2026. “AI Swarms Are Starting to Pose Indirect Takeover Risk.” Redwood Research Blog, August 12. https://blog.redwoodresearch.org/p/ai-swarms-are-starting-to-pose-indirect.
OECD. 2024. Assessing Potential Future Artificial Intelligence Risks, Benefits and Policy Imperatives. OECD Artificial Intelligence Papers No. 27. Vol. 27. OECD Artificial Intelligence Papers. https://doi.org/10.1787/3f4e3dfb-en.
Office of the Superintendent of Financial Institutions. 2024. “OSFI-FCAC Risk Report - AI Uses and Risks at Federally Regulated Financial Institutions.” October 16. https://www.osfi-bsif.gc.ca/en/about-osfi/reports-publications/osfi-fcac-risk-report-ai-uses-risks-federally-regulated-financial-institutions.
Olvera, Abi. 2026. “Tacit Knowledge: The Missing Factor in AI Bio Risk Assessments.” Blog. Second Thoughts, April 23. https://secondthoughts.ai/p/tacit-knowledge-the-missing-factor.
Ord, Toby. 2020a. “Existential Risks to Humanity.” Human Development Report 2020 (Oxford). https://amirrorclear.net/files/existential-risks-to-humanity.pdf.
Ord, Toby. 2020b. The Precipice: Existential Risk and the Future of Humanity. Bloomsbury.
Page, Carly. 2026. “Frontier AI Safety Tests May Be Creating the Very Risks They’re Meant to Stop.” May 12. https://www.theregister.com/ai-ml/2026/05/12/frontier-ai-safety-tests-may-be-creating-the-very-risks-theyre-meant-to-stop/5238734.
Panch, Trishan, Jonathan Pearson-Stuttard, Felix Greaves, and Rifat Atun. 2019. “Artificial Intelligence: Opportunities and Risks for Public Health.” The Lancet Digital Health 1 (1): e13–14. https://doi.org/10.1016/S2589-7500(19)30002-0.
Papadatos, Henry, Jakub Krys, Malcolm Murray, and Renn Karageorgieva. 2026. “Quantitative AI Risk Assessment: A Starting Point.” SaferAI, May 27. https://www.safer-ai.org/quantitative-ai-risk-assessment-a-starting-point.
Parker, Andrew M., Forrest W. Crawford, Gerald L. Epstein, et al. 2026. Principles for Communicating the Risks of Mirror Life. https://www.rand.org/pubs/perspectives/PEA4619-1.html.
Perrigo, Billy. 2024. “Meta’s AI Chief Yann LeCun on AGI, Open-Source, and AI Risk.” TIME, February 13. https://time.com/6694432/yann-lecun-meta-ai-interview/.
Perrow, Charles. 1984. Normal Accidents: Living with High-Risk Technologies. Basic Books.
PKF Antares. 2026. “AI Governance Risks in Captive Insurance Decision-Making | Artificial Intelligence Insights | Professional Insights & Financial News | PKF Antares.” June 5. https://www.pkfantares.com/insights/artifical-intelligence/ai-prompt-bias-captive-insurance/?utm_source=mondaq&utm_medium=syndication&utm_content=sourceoriginal&utm_campaign=.
Quiroz-Gutierrez, Marco. 2026. “AI Godfather Warns Humanity Risks Extinction by Hyperintelligent Machines with Their Own ‘Preservation Goals’ within 10 Years.” Fortune, May 12. https://fortune.com/article/ai-godfather-warning-humanity-risks-extinction-hyperintelligent-machines/.
Raman, Deepika, Nada Madkour, Evan R. Murphy, Krystal Jackson, and Jessica Newman. 2025. “Intolerable Risk Threshold Recommendations for Artificial Intelligence.” arXiv:2503.05812. Preprint, arXiv, March 4. https://doi.org/10.48550/arXiv.2503.05812.
Robinson, Mary. 2023. “The Elders Urge Global Co-Operation to Manage Risks and Share Benefits of AI.” The Elders, May 31. https://theelders.org/news/elders-urge-global-co-operation-manage-risks-and-share-benefits-ai.
Roose, Kevin. 2023. “A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn.” Technology. The New York Times, May 30. https://www.nytimes.com/2023/05/30/technology/ai-threat-warning.html.
Russell, Stuart. 2026. “Experts Are Warning: Our AI Arms Race Is Putting Humanity at Risk.” Opinion. The Guardian, August 11. https://www.theguardian.com/commentisfree/2026/aug/11/openai-anthropic-google-deepmind-letter.
Sample, Ian. 2024. “Unprecedented Risk’ to Life on Earth: Scientists Call for Halt on Mirror Life’ Microbe Research.” The Guardian, December 12.
Scherer, Matthew U. 2015. “Regulating Artificial Intelligence Systems: Risks, Challenges, Competencies, and Strategies.” SSRN Scholarly Paper No. 2609777. Social Science Research Network, May 30. https://doi.org/10.2139/ssrn.2609777.
Sheehan, Matt, and Scott Singer. 2025. “How China Views AI Risks and What to Do About Them.” Carnegie Endowment for International Peace, October 16. https://carnegieendowment.org/research/2025/10/how-china-views-ai-risks-and-what-to-do-about-them.
Simkin, Barnaby, Nikki Pope, Leon Derczynski, and Christopher Parisien. 2025. “Frontier Ai Risk Assessment.” Nvidia, August.
Slattery, Peter, Alexander K. Saeri, Emily A. C. Grundy, et al. 2026. “The AI Risk Repository: A Meta-Review, Database, and Taxonomy of Risks from Artificial Intelligence.” Patterns 0 (0). https://doi.org/10.1016/j.patter.2026.101517.
Smith, Richard. 2026. “Claude Mythos | From Tool to Actor: Artificial Intelligence and Human Extinction Risk.” From Tool to Actor, April 8. https://fromtooltoactor.ca/blog/claude-mythos/.
Stanciuc, Ana-Maria. 2026. “UN Rights Chief Calls for International Red Lines on AI, Warning of Existential Risk.” The Next Web, September 7. https://thenextweb.com/news/un-rights-chief-ai-red-lines-existential-risk.
Standards Council of Canada. 2010. “Safety of Machinery — General Principles for Design — Risk Assessment and Risk Reduction.” SCC-CAN, October 19. https://scc-ccn.ca/standardsdb/standards/8147369.
Steinhardt, Jacob. 2023. “Intrinsic Drives and Extrinsic Misuse: Two Intertwined Risks of AI.” Bounded Regret, October 31. https://bounded-regret.ghost.io/intrinsic-drives-and-extrinsic-misuse-two-intertwined-risks-of-ai/.
Su, Hang, Jun Luo, Chang Liu, et al. 2025. “A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents.” Version 1. Preprint, arXiv. https://doi.org/10.48550/ARXIV.2506.23844.
Supervisor, European Data Protection. 2025. “Guidance for Risk Management of Artificial Intelligence Systems.” https://www.edps.europa.eu/system/files/2025-11/2025-11-11_ai_risks_management_guidance_en.pdf.
Swoboda, Torben, Risto Uuk, Lode Lauwaert, et al. 2026. “Examining Popular Arguments against AI Existential Risk: A Philosophical Analysis.” Ethics and Information Technology 28 (1): 7. https://doi.org/10.1007/s10676-025-09881-y.
Tabassi, Elham. 2023. Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1. National Institute of Standards and Technology (U.S.). https://doi.org/10.6028/NIST.AI.100-1.
Talmadge, Caitlin. 2017. “Would China Go Nuclear? Assessing the Risk of Chinese Nuclear Escalation in a Conventional War with the United States.” International Security 41 (4): 50–92. https://doi.org/10.1162/isec_a_00274.
The Deep View. 2026a. “Anthropic’s Growth May Outpace Risk Control.” The Deep View, May 1. https://archive.thedeepview.com/p/anthropic-s-growth-may-outpace-risk-control.
The Deep View. 2026b. “Why OpenAI’s Team Agents Raise New Risks.” The Deep View, April 15. https://archive.thedeepview.com/p/why-openai-s-team-agents-raise-new-risks.
Trippenbach, Philip, Isabella Scala, Jai Bhambra, and Rowan Emslie. 2025. “From Catastrophic to Concrete: Reframing AI Risk Communication for Public Mobilization.” arXiv:2511.06525. Preprint, arXiv, November 12. https://doi.org/10.48550/arXiv.2511.06525.
Vermeer, Michael J. D., Emily Lathrop, and Alvin Moon. 2025. On the Extinction Risk from Artificial Intelligence. RAND Corporation. https://www.rand.org/pubs/research_reports/RRA3034-1.html.
Wasil, Akash R., Joshua Clymer, David Krueger, Emily Dardaman, Simeon Campos, and Evan R. Murphy. 2024. “Affirmative Safety: An Approach to Risk Management for High-Risk AI.” arXiv:2406.15371. Preprint, arXiv, April 14. https://doi.org/10.48550/arXiv.2406.15371.
Wendehorst, Christiane. 2022. “Liability for Artificial Intelligence: The Need to Address Both Safety Risks and Fundamental Rights Risks.” In The Cambridge Handbook of Responsible Artificial Intelligence, 1st ed., edited by Silja Voeneky, Philipp Kellmeyer, Oliver Mueller, and Wolfram Burgard. Cambridge University Press. https://doi.org/10.1017/9781009207898.016.
Wikipedia. 2025. “Existential risk from artificial intelligence.” November 12. https://en.wikipedia.org/w/index.php?title=Existential_risk_from_artificial_intelligence&oldid=1321806854.
Wisakanto, Anna Katariina, Joe Rogero, Avyay M. Casheekar, and Richard Mallah. 2025. “Adapting Probabilistic Risk Assessment for AI.” arXiv:2504.18536. Preprint, arXiv, July 2. https://doi.org/10.48550/arXiv.2504.18536.
Woolf, Marie, and Sean Silcoff. 2026. “Canadian MPs Join U.K.-Based Campaign Warning of Extinction Risk Posed by Superintelligent AI.” The Globe and Mail, June 3. https://www.theglobeandmail.com/politics/article-canadian-mps-join-uk-based-campaign-warning-of-extinction-risk-posed/.
Yala, Adam, Peter G. Mikhael, Fredrik Strand, et al. 2022. “Multi-Institutional Validation of a Mammography-Based Breast Cancer Risk Model.” Journal of Clinical Oncology 40 (16): 1732–40. https://doi.org/10.1200/jco.21.01337.
Yudkowsky, Eliezer. 2008a. “Artificial Intelligence as a Positive and Negative Factor in Global Risk.” In Global Catastrophic Risks, by Eliezer Yudkowsky. Oxford University Press. https://doi.org/10.1093/oso/9780198570509.003.0021.
Yudkowsky, Eliezer. 2008b. “Cognitive Biases Potentially Affecting Judgment of Global Risks.” In Global Catastrophic Risks, edited by Nick Bostrom and Milan M. Ćirković. Oxford University Press. https://intelligence.org/files/CognitiveBiases.pdf.
Zhang, Yanxi, and Tiezhong Liu. 2023. “Risk Assessment Based on a STPA–FMEA Method: A Case Study of a Sweeping Robot.” Risk Analysis 43 (3): 590–604. https://doi.org/10.1111/risa.13927.