Thinking Rationally About Artificial Intelligence–Caused Industrial Accidents, "9/11s," Extinction Events, and Other Incidents
ResearchPublished Oct 6, 2025
Concern about the potential negative effects of artificial intelligence (AI) systems — and artificial general intelligence or super intelligence systems, in particular — is a central element of policy debate around the technology. A part of that risk concern is uncertainty about how AI systems could cause damage directly, including causing harm so vast that it could pose a global, extinction-level risk. Planning to manage AI risks is challenging because of the great uncertainty surrounding those potential risks.
In this report, the authors explore a cost-benefit logic for considering what must be true for the benefits of different approaches to prevention, mitigation, and response to AI-caused risks to justify the costs of such approaches. Informed by the U.S. response to terrorism in the wake of the September 11, 2001, terrorist attacks — which were also marked by significant uncertainty about the scope of the risk posed to the United States — this approach can help frame reasonable and defensible ways forward while factoring in the benefits promised by AI technologies. The discussion should be of interest to policymakers and stakeholders who are interested in responding to potential AI risks in a balanced and efficient manner, whether those risks are seen as immediate and potentially catastrophic or as a manageable part of the introduction and adoption of a new and powerful technology.
This work was independently initiated and conducted within the Technology and Security Policy Center of RAND Global and Emerging Risks using income from operations and gifts from philanthropic supporters. A complete list of donors and funders is available at www.rand.org/TASP.
This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.
