Arguments about catastrophic risk from advanced AI
Which published texts, from 2000 to 2026, state, contest or bear on the arguments that advanced AI could cause catastrophe or human extinction, and on the arguments that the risk is lower, later, placed elsewhere or better handled otherwise? Which texts answer one another? For each proposition, which text is the earliest located?
Each row is one text’s stance on one proposition, as a stored record states it: articulates, contests or bears on. A stance belongs to the text, not to its author. Rows are in date order, which records no influence between texts. Authors appear as the texts’ own bylines print them, pseudonyms included. “Earliest located” marks only a text the collection’s records name as the earliest the searches found.
R1 · Arguments that advanced AI could pose catastrophic risk, and their premises
R01 Orthogonality of intelligence and goals
- Articulates · Earliest locatedNick BostromEthical Issues in Advanced Artificial Intelligence2003 · published · wording attested 2003-08-06Inspect the record ↗
- Articulates · Earliest located use of the name “orthogonality thesis”Nick BostromThe Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents2012-05 · publishedInspect the record ↗
- Contests · Earliest locatedBen GoertzelSuperintelligence: Fears, Promises and Potentials2015-11 · publishedInspect the record ↗
- Bears onRodney BrooksThe Seven Deadly Sins of Predicting the Future of AI2017-09-07 · publishedInspect the record ↗
- ArticulatesStuart RussellMany Experts Say We Shouldn’t Worry About Superintelligent AI. They’re Wrong2019-10-08 · publishedInspect the record ↗
- ContestsJames D. Miller; Roman Yampolskiy; Olle HäggströmAn AGI Modifying Its Utility Function in Violation of the Orthogonality Thesis2020-01-28 · publishedInspect the record ↗
- ContestsVincent C. Müller; Michael CannonExistential risk from AI and orthogonality: Can we have it both ways?2021 · publishedInspect the record ↗
- ArticulatesOlle HäggströmAI, orthogonality and the Müller-Cannon instrumental vs general intelligence distinction2021-09-14 · publishedInspect the record ↗
R02 Instrumental convergence
- Bears onNick BostromEthical Issues in Advanced Artificial Intelligence2003 · published · wording known only from a later, undated copyInspect the record ↗
- Articulates · Earliest locatedStephen M. OmohundroThe Nature of Self-Improving Artificial Intelligence2007-09-05 · published · wording attested 2008-01-21 · claimed 2007-09-05Inspect the record ↗
- ArticulatesSelf-Aware Systems (author-hosted copy)The Basic AI Drives2008 · publishedInspect the record ↗
- Articulates · Earliest located use of the name “convergent instrumental”Nick BostromThe Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents2012-05 · publishedInspect the record ↗
- ContestsKevin KellyWhy I don’t fear super intelligence (response to "The Myth of AI", Edge Reality Club)2014 · publishedInspect the record ↗
- Contests · Earliest locatedSteven PinkerResponse to Jaron Lanier, "The Myth of AI" (Edge Reality Club)2014 · published · wording attested 2014-11-16Inspect the record ↗
- ArticulatesStuart RussellOf Myths And Moonshine2014 · published · wording attested 2014-12-06Inspect the record ↗
- ArticulatesTsvi Benson-Tilsen; Nate SoaresFormalizing Convergent Instrumental Goals2015-11 · publishedInspect the record ↗
- ContestsAnthony Zador; Yann LeCunDon’t Fear the Terminator2019-09-26 · publishedInspect the record ↗
- ContestsByline not recordedYann LeCun's comments in the Facebook thread on instrumental convergence (2019)2019-10-04 · published · wording attested 2019-10-04Inspect the record ↗
- ArticulatesByline not recordedStuart Russell's comments in the Facebook thread on instrumental convergence (2019)2019-10-04 · published · wording attested 2019-10-04Inspect the record ↗
- ContestsByline not recordedAnthony Zador's comments in the Facebook thread on instrumental convergence (2019)2019-10-04 · published · wording attested 2019-10-04Inspect the record ↗
- ArticulatesStuart RussellMany Experts Say We Shouldn’t Worry About Superintelligent AI. They’re Wrong2019-10-08 · publishedInspect the record ↗
- ArticulatesAlexander Matt TurnerOptimal Farsighted Agents Tend to Seek Power2019-12-03 · publishedInspect the record ↗
- ArticulatesAlexander Matt Turner; Prasad TadepalliParametrically Retargetable Decision-Makers Tend To Seek Power2022-06-27 · publishedInspect the record ↗
- ArticulatesVictoria Krakovna; Janos KramarPower-seeking can be probable and predictive for trained agents2023-04-13 · publishedInspect the record ↗
- ContestsMarc AndreessenWhy AI Will Save the World2023-06-06 · publishedInspect the record ↗
- ArticulatesYoshua BengioReasoning through arguments against taking AI safety seriously2024-07-09 · publishedInspect the record ↗
- ContestsChristian TarsneyWill artificial agents pursue power by default?2025-06-02 · publishedInspect the record ↗
- ContestsDavid ThorstadInstrumental convergence and power-seeking2026-06-07 · publishedInspect the record ↗
R03 Inner or deceptive misalignment
- Articulates · Earliest locatedNate SoaresEnsuring smarter-than-human intelligence has a positive outcome2017-04-12 · publishedInspect the record ↗
- ArticulatesJack Koch; Lauro Langosco; Jacob Pfau; James Le; Lee SharkeyObjective Robustness in Deep Reinforcement Learning (later: Goal Misgeneralization in Deep Reinforcement Learning)2021-05-28 · publishedInspect the record ↗
- ArticulatesRichard NgoThe Alignment Problem from a Deep Learning Perspective2022-08-30 · publishedInspect the record ↗
- Articulates · Earliest located use of the name “goal misgeneralization”Rohin Shah; Vikrant Varma; Ramana Kumar; Mary Phuong; Victoria Krakovna; Jonathan Uesato; Zac KentonGoal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals2022-10-04 · publishedInspect the record ↗
- ArticulatesJoseph CarlsmithScheming AIs: Will AIs fake alignment during training in order to get power?2023-11-14 · publishedInspect the record ↗
- Contests · Earliest locatedNora Belrose; Quintin PopeAI is easy to control2023-11-28 · publishedInspect the record ↗
- ArticulatesEvan Hubinger; Carson Denison; Jesse Mu; Mike Lambert; Meg Tong; Monte MacDiarmid; Tamera Lanham; Daniel M. Ziegler; Tim Maxwell; Newton Cheng; Adam Jermyn; Amanda Askell; Ansh Radhakrishnan; Cem Anil; David Duvenaud; Deep Ganguli; Fazl Barez; Jack Clark; Kamal Ndousse; Kshitij Sachan; Michael Sellitto; Mrinank Sharma; Nova DasSarma; Roger Grosse; Shauna Kravec; Yuntao Bai; Zachary Witten; Marina Favaro; Jan Brauner; Holden Karnofsky; Paul Christiano; Samuel R. Bowman; Logan Graham; Jared Kaplan; Sören Mindermann; Ryan Greenblatt; Buck Shlegeris; Nicholas Schiefer; Ethan PerezSleeper Agents: Training Deceptive LLMs that Persist Through Safety Training2024-01-10 · publishedInspect the record ↗
- ContestsNora Belrose; Quintin PopeCounting arguments provide no evidence for AI doom2024-02-27 · publishedInspect the record ↗
- ContestsViolet HourTwo Tales of AI Takeover: My Doubts2024-03-05 · published · wording attested 2024-03-22Inspect the record ↗
- ContestsAlexander Matt TurnerMany Arguments for AI X-Risk Are Wrong2024-03-05 · published · wording attested 2026-08-19Inspect the record ↗
- ArticulatesRyan Greenblatt; Carson Denison; Benjamin Wright; Fabien Roger; Monte MacDiarmid; Sam Marks; Johannes Treutlein; Tim Belonax; Jack Chen; David Duvenaud; Akbir Khan; Julian Michael; Sören Mindermann; Ethan Perez; Linda Petrini; Jonathan Uesato; Jared Kaplan; Buck Shlegeris; Samuel R. Bowman; Evan HubingerAlignment faking in large language models2024-12-18 · publishedInspect the record ↗
B1 Treacherous turn
- Bears onVernor VingeThe Coming Technological Singularity: How to Survive in the Post-Human Era1993-12 · published · claimed 1993-03-30Inspect the record ↗
- Bears onLee Daniel CrockerRe: Goal-based AI1996-12-31 · publishedInspect the record ↗
- Bears onEliezer YudkowskyRe: Goal-based AI1996-12-31 · publishedInspect the record ↗
- Bears onEliezer YudkowskyRe: Goal-based AI1996-12-31 · publishedInspect the record ↗
- Articulates · Earliest locatedEliezer YudkowskyCreating Friendly AI 1.0: The Analysis and Design of Benevolent Goal Architectures2001-06-15 · published · wording attested 2001-06-17Inspect the record ↗
- Bears on · Earliest located use of the name “treacherous turn”Katja GraceSuperintelligence reading group2014-08-31 · publishedInspect the record ↗
- Articulates · Relay of another authorKatja GraceSuperintelligence 11: The treacherous turn2014-11-25 · publishedInspect the record ↗
- ArticulatesNate Soares; Benja Fallenstein; Eliezer Yudkowsky; Stuart ArmstrongCorrigibility2015-01-25 · publishedInspect the record ↗
- ArticulatesEliezer YudkowskyContext disaster2015-06-08 · published · wording known only from a later, undated copyInspect the record ↗
- ContestsBen GoertzelSuperintelligence: Fears, Promises and Potentials2015-11 · publishedInspect the record ↗
- ArticulatesEvan Hubinger; Chris van Merwijk; Vladimir Mikulik; Joar Skalse; Scott GarrabrantDeceptive Alignment (Risks from Learned Optimization, part 4)2019-06-05 · publishedInspect the record ↗
- ArticulatesAjeya CotraWhy AI alignment could be hard with modern deep learning2021-09-21 · publishedInspect the record ↗
- ContestsPaul ChristianoWhere I agree and disagree with Eliezer2022-06-19 · publishedInspect the record ↗
- ArticulatesAjeya CotraWithout specific countermeasures, the easiest path to transformative AI likely leads to AI takeover2022-07-18 · publishedInspect the record ↗
- ArticulatesRichard NgoThe Alignment Problem from a Deep Learning Perspective2022-08-30 · publishedInspect the record ↗
- ArticulatesRohin Shah; Vikrant Varma; Ramana Kumar; Mary Phuong; Victoria Krakovna; Jonathan Uesato; Zac KentonGoal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals2022-10-04 · publishedInspect the record ↗
- ArticulatesJoseph CarlsmithScheming AIs: Will AIs fake alignment during training in order to get power?2023-11-14 · publishedInspect the record ↗
- ContestsNora Belrose; Quintin PopeAI is easy to control2023-11-28 · publishedInspect the record ↗
- Bears onRyan Greenblatt; Carson Denison; Benjamin Wright; Fabien Roger; Monte MacDiarmid; Sam Marks; Johannes Treutlein; Tim Belonax; Jack Chen; David Duvenaud; Akbir Khan; Julian Michael; Sören Mindermann; Ethan Perez; Linda Petrini; Jonathan Uesato; Jared Kaplan; Buck Shlegeris; Samuel R. Bowman; Evan HubingerAlignment faking in large language models2024-12-18 · publishedInspect the record ↗
A4 Fast or hard takeoff
- Bears on · SatiricalCellarius (Samuel Butler)Darwin among the Machines1863-06-13 · publishedInspect the record ↗
- Bears on · FictionSamuel Butler (published anonymously)Erewhon, or Over the Range1872 · publishedInspect the record ↗
- Bears onAlan M. TuringIntelligent Machinery, a Heretical Theory1951Inspect the record ↗
- Bears onAlan M. TuringCan Digital Computers Think?1951-05-15Inspect the record ↗
- Bears onNorbert WienerSome Moral and Technical Consequences of Automation1960-05-06 · published · claimed 1959-12-27Inspect the record ↗
- Bears onVernor VingeFirst Word1983-01 · publishedInspect the record ↗
- Bears onRay J. SolomonoffThe Time Scale of Artificial Intelligence: Reflections on Social Effects1985 · published · wording attested 2006-07-16Inspect the record ↗
- Bears onK. Eric DrexlerEngines of Creation: The Coming Era of Nanotechnology1986 · published · wording attested 1997-07-09Inspect the record ↗
- Bears onHans MoravecHuman Culture—A Genetic Takeover Underway1989 · published · wording attested 2001-03-06 · claimed 1988-07Inspect the record ↗
- Bears onM.P.-Infinity (byline)The Opening of the Transhuman Mind1990 · published · wording known only from a later, undated copyInspect the record ↗
- Bears onFutique Neologisms compiler (various contributors)Futique Neologisms1991 · publishedInspect the record ↗
- Bears onFutique Neologisms 2 compiler; entry credited to Mark PlusFutique Neologisms 21991 · publishedInspect the record ↗
- Articulates · Earliest locatedVernor VingeThe Coming Technological Singularity: How to Survive in the Post-Human Era1993-12 · published · claimed 1993-03-30Inspect the record ↗
- Bears onRobin HansonIf Uploads Come First: The Crack of a Future Dawn1994 · publishedInspect the record ↗
- Bears onDani EderRe: The Singularity (Eder)1994-05-23 · published · wording attested 1997-07-12Inspect the record ↗
- Bears onByline not recordedFuture Forecasts: Mark Miller’s comments1995 · publishedInspect the record ↗
- Bears onDan ClemmensenRe: Thinking about the future...1996-08-30 · publishedInspect the record ↗
- Bears onDan ClemmensenRe: Thinking about the future...1996-09-03 · publishedInspect the record ↗
- ContestsRobin HansonRe: Thinking about the future...1996-09-03 · publishedInspect the record ↗
- Bears onNick BostromRe: Thinking about the future...1996-09-04 · publishedInspect the record ↗
- ArticulatesDan ClemmensenRe: Thinking about the future...1996-09-04 · publishedInspect the record ↗
- ArticulatesDan ClemmensenRe: SI ultimatum speculation1996-09-07 · publishedInspect the record ↗
- Bears onEliezer S. YudkowskyStaring into the Singularity1996-11-18 · published · wording attested 2000-12-04Inspect the record ↗
- Bears onEliezer S. YudkowskySingularity: Generation gap1997-09-24 · publishedInspect the record ↗
- ContestsMax MoreSingularity Meets Economy (Comments on Vinge’s Singularity)1998 · published · wording attested 2000-08-17Inspect the record ↗
- Bears onEliezer S. YudkowskyHuman AI to superhuman (Re: Max More)1998-09-07 · publishedInspect the record ↗
- ContestsRobin HansonSingularity: Human AI to superhuman1998-09-10 · publishedInspect the record ↗
- Articulates · Earliest located use of the name “hard takeoff” · Relay of another authorVernor VingeSingularity: Vinge responds1998-09-16 · publishedInspect the record ↗
- ArticulatesEliezer YudkowskyCreating Friendly AI 1.0: The Analysis and Design of Benevolent Goal Architectures2001-06-15 · published · wording attested 2001-06-17Inspect the record ↗
- ArticulatesNick BostromEthical Issues in Advanced Artificial Intelligence2003 · published · wording attested 2003-08-06Inspect the record ↗
- ArticulatesEliezer YudkowskyArtificial Intelligence as a Positive and Negative Factor in Global Risk2006-02-05Inspect the record ↗
- ContestsRobin HansonWhat Core Argument?2008-12 · published · wording known only from a later, undated copyInspect the record ↗
- ArticulatesEliezer YudkowskyRecursive Self-Improvement2008-12-01 · publishedInspect the record ↗
- ArticulatesEliezer YudkowskyHard Takeoff2008-12-02 · publishedInspect the record ↗
- Bears onDavid J. ChalmersThe Singularity: A Philosophical Analysis2010 · publishedInspect the record ↗
- Bears onLuke Muehlhauser; Anna SalamonIntelligence Explosion: Evidence and Import2012 · publishedInspect the record ↗
- ArticulatesByline not recordedIntelligence Explosion Microeconomics2013-09-13 · publishedInspect the record ↗
- ContestsRobin HansonI Still Don't Get Foom2014-07-24 · publishedInspect the record ↗
- Bears onSam AltmanMachine intelligence, part 12015-02-25 · publishedInspect the record ↗
- ContestsToby WalshThe Singularity May Never Be Near2016-02-20 · publishedInspect the record ↗
- Bears onEliezer YudkowskyOptimization daemons2016-03-22 · published · wording known only from a later, undated copyInspect the record ↗
- ContestsSebastian BenthallDon't Fear the Reaper: Refuting Bostrom's Superintelligence Argument2017-02-27 · publishedInspect the record ↗
- ContestsKevin KellyThe Myth of a Superhuman AI2017-04-27 · publishedInspect the record ↗
- ContestsRodney BrooksThe Seven Deadly Sins of Predicting the Future of AI2017-09-07 · publishedInspect the record ↗
- ContestsAI ImpactsLikelihood of discontinuous progress around the development of AGI2018-02-23 · published · wording attested 2020-05-26Inspect the record ↗
- ContestsPaul ChristianoTakeoff speeds2018-02-24 · publishedInspect the record ↗
- Bears onPaul ChristianoWhat failure looks like2019-03-17 · publishedInspect the record ↗
- ContestsRobin HansonFoom Update2022-05-06 · publishedInspect the record ↗
- Bears onEliezer YudkowskyAGI Ruin: A List of Lethalities2022-06-05 · publishedInspect the record ↗
- Bears onPaul ChristianoWhere I agree and disagree with Eliezer2022-06-19 · publishedInspect the record ↗
- ArticulatesAjeya CotraWithout specific countermeasures, the easiest path to transformative AI likely leads to AI takeover2022-07-18 · publishedInspect the record ↗
- ContestsKatja GraceCounterarguments to the basic AI x-risk case2022-08-31 · publishedInspect the record ↗
- ContestsDavid ThorstadAgainst the singularity hypothesis2022-11 · publishedInspect the record ↗
- ArticulatesTom DavidsonWhat a compute-centric framework says about AI takeoff speeds2023-01-23 · published · wording attested 2025-04-01Inspect the record ↗
- Bears onLeopold AschenbrennerSituational Awareness: The Decade Ahead2024 · publishedInspect the record ↗
- ContestsArvind Narayanan; Sayash KapoorAI as Normal Technology2025-04-15 · publishedInspect the record ↗
- Bears onTom Davidson; Tom HouldenHow quick and big would a software intelligence explosion be?2025-08-04 · publishedInspect the record ↗
R06 Difficulty of alignment
- Articulates · Earliest locatedBen GoertzelThoughts on AI Morality2002-05 · publishedInspect the record ↗
- ArticulatesStuart RussellOf Myths And Moonshine2014 · published · wording attested 2014-12-06Inspect the record ↗
- Contests · Earliest locatedErnest DavisEthical Guidelines for A Superintelligence2014-10-23 · publishedInspect the record ↗
- Bears onStuart Russell; Daniel Dewey; Max TegmarkResearch Priorities for Robust and Beneficial Artificial Intelligence2015 · published · claimed 2015-01Inspect the record ↗
- Bears onFuture of Life InstituteResearch Priorities for Robust and Beneficial Artificial Intelligence: An Open Letter2015-10-28 · published · claimed 2015-01Inspect the record ↗
- Bears onDario Amodei; Chris Olah; Jacob Steinhardt; Paul Christiano; John Schulman; Dan ManéConcrete Problems in AI Safety2016-06-21 · publishedInspect the record ↗
- ArticulatesEliezer YudkowskyAI Alignment: Why It's Hard, and Where to Start2016-12-28 · published · claimed 2016-05Inspect the record ↗
- ArticulatesNate SoaresEnsuring smarter-than-human intelligence has a positive outcome2017-04-12 · publishedInspect the record ↗
- ContestsSteven PinkerWe’re told to fear robots. But why do we think they’ll turn on us?2018-02-14 · publishedInspect the record ↗
- Articulates · Earliest located use of the name “alignment problem”Byline not recordedStuart Russell's comments in the Facebook thread on instrumental convergence (2019)2019-10-04 · published · wording attested 2019-10-04Inspect the record ↗
- ArticulatesRichard NgoThe Alignment Problem from a Deep Learning Perspective2022-08-30 · publishedInspect the record ↗
- Bears onJan LeikeWhy I'm optimistic about our alignment approach2022-12-05 · publishedInspect the record ↗
- ContestsNora Belrose; Quintin PopeAI is easy to control2023-11-28 · publishedInspect the record ↗
- ContestsBeren MillidgeAlignment likely generalizes further than capabilities.2024-05-15 · publishedInspect the record ↗
R07 Risk of extinction or comparable catastrophe
- Articulates · Earliest locatedBill JoyWhy the Future Doesn't Need Us2000-04 · publishedInspect the record ↗
- ArticulatesNick BostromExistential Risks: Analyzing Human Extinction Scenarios and Related Hazards2002-03 · published · claimed 2001-04Inspect the record ↗
- Contests · Earliest locatedSteven PinkerResponse to Jaron Lanier, "The Myth of AI" (Edge Reality Club)2014 · publishedInspect the record ↗
- ArticulatesStephen Hawking; Stuart Russell; Max Tegmark; Frank WilczekTranscendence looks at the implications of artificial intelligence - but are we taking AI seriously enough?2014-05-01 · publishedInspect the record ↗
- ArticulatesSam AltmanMachine intelligence, part 12015-02-25 · publishedInspect the record ↗
- ContestsJeff HawkinsThe Terminator Is Not Coming. The Future Will Thank Us.2015-03-02 · publishedInspect the record ↗
- Articulates · Earliest located use of the name “risk of extinction”Center for AI SafetyStatement on AI Risk2023-05-30 · publishedInspect the record ↗
- ContestsAndrew NgExistential Risk? I Don't Get It!2023-06-07 · publishedInspect the record ↗
R08 Existential risk from misalignment or loss of control
- Articulates · Earliest locatedNick BostromExistential Risks: Analyzing Human Extinction Scenarios and Related Hazards2002-03 · published · claimed 2001-04Inspect the record ↗
- Contests · Earliest locatedSteven PinkerResponse to Jaron Lanier, "The Myth of AI" (Edge Reality Club)2014 · publishedInspect the record ↗
- ArticulatesSam AltmanMachine intelligence, part 12015-02-25 · publishedInspect the record ↗
- ContestsJeff HawkinsThe Terminator Is Not Coming. The Future Will Thank Us.2015-03-02 · publishedInspect the record ↗
- ArticulatesAllan Dafoe; Stuart RussellYes, We Are Worried About the Existential Risk of Artificial Intelligence2016-11-02 · publishedInspect the record ↗
- ContestsByline not recordedYann LeCun's comments in the Facebook thread on instrumental convergence (2019)2019-10-04 · published · wording attested 2019-10-04Inspect the record ↗
- ContestsJohn G. SotosOn the Unimportance of Superintelligence2021-08-30 · publishedInspect the record ↗
- Articulates · Earliest located use of the name “disempowerment”Joseph CarlsmithIs Power-Seeking AI an Existential Risk?2022-06-16 · published · claimed 2021-04Inspect the record ↗
- Bears onFuture of Life InstitutePause Giant AI Experiments: An Open Letter2023-03-22 · publishedInspect the record ↗
- ArticulatesYoshua BengioHow Rogue AIs may Arise2023-05-22 · published · wording attested 2023-05-23Inspect the record ↗
- ContestsMarc AndreessenWhy AI Will Save the World2023-06-06 · publishedInspect the record ↗
- ArticulatesDan Hendrycks; Mantas Mazeika; Thomas WoodsideAn Overview of Catastrophic AI Risks2023-06-21 · publishedInspect the record ↗
- ContestsJoshua GansWhy I am not worried about superintelligent machines killing us all2023-06-23 · publishedInspect the record ↗
- ContestsBlake Richards; Blaise Agüera y Arcas; Guillaume Lajoie; Dhanya SridharThe Illusion Of AI's Existential Risk2023-07-18 · publishedInspect the record ↗
- ArticulatesYoshua Bengio; Geoffrey Hinton; Andrew Yao; Dawn Song; Pieter Abbeel; Yuval Noah Harari; Ya-Qin Zhang; Lan Xue; Shai Shalev-Shwartz; Gillian Hadfield; Jeff Clune; Tegan Maharaj; Frank Hutter; Atılım Güneş Baydin; Sheila McIlraith; Qiqi Gao; Ashwin Acharya; David Krueger; Anca Dragan; Philip Torr; Stuart Russell; Daniel Kahneman; Jan Brauner; Sören MindermannManaging AI Risks in an Era of Rapid Progress2023-10-26 · publishedInspect the record ↗
- ArticulatesGeoffrey HintonBanquet speech, Nobel Banquet, 10 December 20242024-12-10Inspect the record ↗
- Bears onYoshua Bengio; and othersInternational AI Safety Report2025-01-29 · publishedInspect the record ↗
- ContestsArvind Narayanan; Sayash KapoorAI as Normal Technology2025-04-15 · publishedInspect the record ↗
R09 Catastrophic risk from misuse
- Articulates · Earliest locatedNick BostromExistential Risks: Analyzing Human Extinction Scenarios and Related Hazards2002-03 · published · claimed 2001-04Inspect the record ↗
- Contests · Earliest locatedSteven PinkerResponse to Jaron Lanier, "The Myth of AI" (Edge Reality Club)2014 · publishedInspect the record ↗
- ArticulatesByline not recordedYoshua Bengio's comments in the Facebook thread on instrumental convergence (2019)2019-10-04 · published · wording attested 2019-10-04Inspect the record ↗
- ArticulatesJohn G. SotosOn the Unimportance of Superintelligence2021-08-30 · publishedInspect the record ↗
- ArticulatesGary MarcusAI risk ≠ AGI risk2023-03-28 · publishedInspect the record ↗
- ArticulatesYoshua BengioHow Rogue AIs may Arise2023-05-22 · published · wording attested 2026-10-29Inspect the record ↗
- Articulates · Earliest located use of the name “malicious use”Dan Hendrycks; Mantas Mazeika; Thomas WoodsideAn Overview of Catastrophic AI Risks2023-06-21 · publishedInspect the record ↗
- ArticulatesUnited States Senate, Committee on the JudiciaryStatement of Dario Amodei, Senate Judiciary Subcommittee hearing "Oversight of A.I.: Principles for Regulation"2023-07-25Inspect the record ↗
- Bears onRAND CorporationThe Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study2024 · publishedInspect the record ↗
R2 · Arguments that the risk is lower, later, placed elsewhere or better handled otherwise
R10a Intelligence brings benevolence or no drive to dominate
- Articulates · Earliest locatedBen GoertzelEncouraging a Positive Transcension2004-02-17 · publishedInspect the record ↗
- Contests · Earliest locatedSelf-Aware Systems (author-hosted copy)The Basic AI Drives2008 · publishedInspect the record ↗
- ContestsNick BostromThe Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents2012-05 · publishedInspect the record ↗
- ArticulatesSteven PinkerResponse to Jaron Lanier, "The Myth of AI" (Edge Reality Club)2014 · publishedInspect the record ↗
- Bears onRodney BrooksArtificial intelligence is a tool, not a threat2014-11 · published · wording attested 2014Inspect the record ↗
- ArticulatesSteven PinkerThinking Does Not Imply Subjugating2015 · publishedInspect the record ↗
- ContestsNate Soares; Benja Fallenstein; Eliezer Yudkowsky; Stuart ArmstrongCorrigibility2015-01-25 · publishedInspect the record ↗
- ArticulatesJeff HawkinsThe Terminator Is Not Coming. The Future Will Thank Us.2015-03-02 · publishedInspect the record ↗
- ArticulatesSteven PinkerWe’re told to fear robots. But why do we think they’ll turn on us?2018-02-14 · publishedInspect the record ↗
- ArticulatesAnthony Zador; Yann LeCunDon’t Fear the Terminator2019-09-26 · publishedInspect the record ↗
- ArticulatesMarc AndreessenWhy AI Will Save the World2023-06-06 · publishedInspect the record ↗
- ContestsYoshua BengioReasoning through arguments against taking AI safety seriously2024-07-09 · publishedInspect the record ↗
R10b Advanced AI can be kept under control by design
- Articulates · Earliest locatedKevin KellyWhy I don’t fear super intelligence (response to "The Myth of AI", Edge Reality Club)2014 · published · wording attested 2014-11-18Inspect the record ↗
- ArticulatesSteven PinkerThinking Does Not Imply Subjugating2015 · publishedInspect the record ↗
- Contests · Earliest locatedNate Soares; Benja Fallenstein; Eliezer Yudkowsky; Stuart ArmstrongCorrigibility2015-01-25 · publishedInspect the record ↗
- ArticulatesAnthony Zador; Yann LeCunDon’t Fear the Terminator2019-09-26 · publishedInspect the record ↗
- ContestsByline not recordedYoshua Bengio's comments in the Facebook thread on instrumental convergence (2019)2019-10-04 · published · wording attested 2019-10-04Inspect the record ↗
- ArticulatesByline not recordedYann LeCun's comments in the Facebook thread on instrumental convergence (2019)2019-10-04 · published · wording attested 2019-10-04Inspect the record ↗
- ContestsStuart RussellMany Experts Say We Shouldn’t Worry About Superintelligent AI. They’re Wrong2019-10-08 · publishedInspect the record ↗
- Bears onMarc AndreessenWhy AI Will Save the World2023-06-06 · publishedInspect the record ↗
- ArticulatesNora Belrose; Quintin PopeAI is easy to control2023-11-28 · publishedInspect the record ↗
R11 Risk claims are speculative or unfalsifiable
- Contests · Earliest locatedStuart RussellOf Myths And Moonshine2014 · published · wording attested 2014-12-06Inspect the record ↗
- Articulates · Earliest locatedRodney BrooksArtificial intelligence is a tool, not a threat2014-11 · published · wording attested 2014-11-11Inspect the record ↗
- ArticulatesOren EtzioniNo, the Experts Don't Think Superintelligent AI is a Threat to Humanity2016-09-20 · publishedInspect the record ↗
- ContestsAllan Dafoe; Stuart RussellYes, We Are Worried About the Existential Risk of Artificial Intelligence2016-11-02 · publishedInspect the record ↗
- ArticulatesTimnit Gebru; Emily M. Bender; Angelina McMillan-MajorStatement from the listed authors of Stochastic Parrots on the "AI pause" letter2023-03-31 · publishedInspect the record ↗
- ArticulatesMarc AndreessenWhy AI Will Save the World2023-06-06 · publishedInspect the record ↗
- ArticulatesAndrew NgThe Batch, issue 220: letter on fears of AI-caused extinction2023-10 · publishedInspect the record ↗
- ContestsRose HadsharA Review of the Evidence for Existential Risk from AI via Misaligned Power-Seeking2023-10-27 · publishedInspect the record ↗
- ContestsYoshua BengioReasoning through arguments against taking AI safety seriously2024-07-09 · publishedInspect the record ↗
- ArticulatesArvind Narayanan; Sayash KapoorAI existential risk probabilities are too unreliable to inform policy2024-07-26 · publishedInspect the record ↗
R12 Present harms deserve priority
- Articulates · Earliest locatedKate CrawfordArtificial Intelligence's White Guy Problem2016-06-25 · publishedInspect the record ↗
- Bears on · Relay of another authorKate Crawford; Ryan CaloThere is a blind spot in AI research2016-10-13 · publishedInspect the record ↗
- Bears onByline not recordedAnthony Zador's comments in the Facebook thread on instrumental convergence (2019)2019-10-04 · published · wording attested 2019-10-04Inspect the record ↗
- Contests · Earliest locatedGary MarcusAI risk ≠ AGI risk2023-03-28 · publishedInspect the record ↗
- ArticulatesTimnit Gebru; Emily M. Bender; Angelina McMillan-MajorStatement from the listed authors of Stochastic Parrots on the "AI pause" letter2023-03-31 · publishedInspect the record ↗
- Contests · Earliest located use of the name “current harms”Yoshua BengioFAQ on Catastrophic AI Risks2023-06-24 · published · wording attested 2026-10-29Inspect the record ↗
- ArticulatesNatureStop talking about tomorrow's AI doomsday when AI poses risks today2023-06-27 · publishedInspect the record ↗
- ArticulatesBlake Richards; Blaise Agüera y Arcas; Guillaume Lajoie; Dhanya SridharThe Illusion Of AI's Existential Risk2023-07-18 · publishedInspect the record ↗
- ArticulatesEmily M. Bender; Alex HannaAI Causes Real Harm. Let's Focus on That over the End-of-Humanity Hype2023-08-12 · published · wording attested 2026-10-29Inspect the record ↗
- ContestsEncode Justice; Future of Life InstituteAI Licensing for a Better Future: On Addressing Both Present Harms and Emerging Threats2023-10-25 · publishedInspect the record ↗
- ArticulatesTimnit Gebru; Émile P. TorresThe TESCREAL bundle: Eugenics and the promise of utopia through artificial general intelligence2024-04-01 · publishedInspect the record ↗
- ContestsEmma Hoes; Fabrizio GilardiExistential risk narratives about AI do not distract from its immediate harms2025-04-17 · publishedInspect the record ↗
R13 Capabilities will plateau
- Articulates · Earliest locatedGary MarcusDeep Learning: A Critical Appraisal2018-01-02 · publishedInspect the record ↗
- Bears onRich SuttonThe Bitter Lesson2019-03-13 · publishedInspect the record ↗
- Contests · Earliest locatedGwern BranwenThe Scaling Hypothesis2020-05-28 · published · wording attested 2022-01-02Inspect the record ↗
- Bears onMelanie MitchellWhy AI is Harder Than We Think2021-04-26 · publishedInspect the record ↗
- Articulates · Earliest located use of the name “diminishing returns”Gary MarcusDeep Learning Is Hitting a Wall2022-03-10 · publishedInspect the record ↗
- ContestsYoshua Bengio; Geoffrey Hinton; Andrew Yao; Dawn Song; Pieter Abbeel; Yuval Noah Harari; Ya-Qin Zhang; Lan Xue; Shai Shalev-Shwartz; Gillian Hadfield; Jeff Clune; Tegan Maharaj; Frank Hutter; Atılım Güneş Baydin; Sheila McIlraith; Qiqi Gao; Ashwin Acharya; David Krueger; Anca Dragan; Philip Torr; Stuart Russell; Daniel Kahneman; Jan Brauner; Sören MindermannManaging AI Risks in an Era of Rapid Progress2023-10-26 · publishedInspect the record ↗
- ContestsLeopold AschenbrennerSituational Awareness: The Decade Ahead2024 · publishedInspect the record ↗
- ArticulatesGary MarcusEvidence that LLMs are reaching a point of diminishing returns - and what that might mean2024-04-13 · publishedInspect the record ↗
- ArticulatesArvind Narayanan; Sayash KapoorAI scaling myths2024-06-27 · publishedInspect the record ↗
- ContestsYoshua BengioReasoning through arguments against taking AI safety seriously2024-07-09 · publishedInspect the record ↗
- ContestsSam AltmanThe Intelligence Age2024-09-23 · publishedInspect the record ↗
R14 Language models lack agency or understanding
- Articulates · Earliest locatedGary Marcus; Ernest DavisIf Computers Are So Smart, How Come They Can't Read?2019-09-10 · publishedInspect the record ↗
- ArticulatesGary MarcusGPT-2 and the Nature of Intelligence2020-01-25 · publishedInspect the record ↗
- ArticulatesEmily M. Bender; Alexander KollerClimbing towards NLU: On Meaning, Form, and Understanding in the Age of Data2020-07 · publishedInspect the record ↗
- Articulates · Earliest located use of the name “autocomplete”Gary MarcusNonsense on Stilts2022-06-12 · publishedInspect the record ↗
- Contests · Earliest locatedSteven T. Piantadosi; Felix HillMeaning without reference in large language models2022-08-05 · publishedInspect the record ↗
- Bears onJacob Browning; Yann LeCunAI And The Limits Of Language2022-08-23 · publishedInspect the record ↗
- Bears onjanusSimulators2022-09-02 · publishedInspect the record ↗
- Bears onMelanie Mitchell; David C. KrakauerThe Debate Over Understanding in AI's Large Language Models2022-10-14 · publishedInspect the record ↗
- ContestsJacob AndreasLanguage Models as Agent Models2022-12-03 · publishedInspect the record ↗
- ArticulatesMurray ShanahanTalking About Large Language Models2022-12-07 · publishedInspect the record ↗
- Bears onKyle Mahowald; Anna A. Ivanova; Idan A. Blank; Nancy Kanwisher; Joshua B. Tenenbaum; Evelina FedorenkoDissociating language and thought in large language models: a cognitive perspective2023-01-16 · publishedInspect the record ↗
- ContestsSébastien Bubeck; Varun Chandrasekaran; Ronen Eldan; Johannes Gehrke; Eric Horvitz; Ece Kamar; Peter Lee; Yin Tat Lee; Yuanzhi Li; Scott Lundberg; Harsha Nori; Hamid Palangi; Marco Tulio Ribeiro; Yi ZhangSparks of Artificial General Intelligence: Early experiments with GPT-42023-03-22 · publishedInspect the record ↗
- Bears onSubbarao Kambhampati; Karthik Valmeekam; Lin Guan; Kaya Stechly; Mudit Verma; Siddhant Bhambri; Lucas Saldyt; Anil MurthyLLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks2024-02-02 · publishedInspect the record ↗
- ContestsAlexander Meinke; Bronson Schoen; Jérémy Scheurer; Mikita Balesni; Rusheb Shah; Marius HobbhahnFrontier Models are Capable of In-context Scheming2024-12-06 · publishedInspect the record ↗
R15 Open models reduce risk
- Articulates · Earliest located · Earliest located use of the name “open source”Ben Goertzel; Joel PittNine Ways to Bias Open-Source AGI Toward Friendliness2012-02 · publishedInspect the record ↗
- Contests · Earliest locatedScott AlexanderShould AI Be Open?2015-12-17 · publishedInspect the record ↗
- ContestsNick BostromStrategic Implications of Openness in AI Development2017 · publishedInspect the record ↗
- ContestsMiles Brundage; Shahar Avin; Jack Clark; Helen Toner; and othersThe Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation2018-02 · publishedInspect the record ↗
- Bears onIrene Solaiman; Miles Brundage; Jack Clark; Amanda Askell; and othersRelease Strategies and the Social Impacts of Language Models2019-08 · publishedInspect the record ↗
- ArticulatesConnor LeahyWhy Release a Large Language Model?2021-06-02 · publishedInspect the record ↗
- ContestsToby ShevlaneStructured access to AI capabilities: an emerging paradigm for safe AI deployment2022-01 · publishedInspect the record ↗
- ArticulatesArvind Narayanan; Sayash KapoorThe LLaMA is out of the bag. Should we expect a tidal wave of disinformation?2023-03-06 · publishedInspect the record ↗
- Bears onMarc AndreessenWhy AI Will Save the World2023-06-06 · publishedInspect the record ↗
- ContestsDan Hendrycks; Mantas Mazeika; Thomas WoodsideAn Overview of Catastrophic AI Risks2023-06-21 · publishedInspect the record ↗
- ArticulatesJeremy HowardAI Safety and the Age of Dislightenment2023-07-10 · publishedInspect the record ↗
- ArticulatesMeta (statement text); signatories listed below itStatement of Support for Meta's Open Approach to Today's AI2023-07-18 · publishedInspect the record ↗
- ArticulatesHugo Touvron; Louis Martin; Kevin Stone; Peter Albert; Amjad Almahairi; Yasmine Babaei; Nikolay Bashlykov; Soumya Batra; Prajjwal Bhargava; Shruti Bhosale; Dan Bikel; Lukas Blecher; Moya Chen; Guillem Cucurull; David Esiobu; Jude Fernandes; Jeremy Fu; Wenyin Fu; Brian Fuller; Cynthia Gao; Vedanuj Goswami; Naman Goyal; Anthony Hartshorn; Saghar Hosseini; Rui Hou; Hakan Inan; Marcin Kardas; Viktor Kerkez; Madian Khabsa; Isabel Kloumann; Artem Korenev; Punit Singh Koura; Thibaut Lavril; Jenya Lee; Diana Liskovich; Yinghai Lu; Yuning Mao; Xavier Martinet; Todor Mihaylov; Pushkar Mishra; Igor Molybog; Yixin Nie; Andrew Poulton; Jeremy Reizenstein; Rashi Rungta; Kalyan Saladi; Alan Schelten; Ruan Silva; Eric Michael Smith; Ranjan Subramanian; Binh Tang; Ross Taylor; Adina Williams; Puxin Xu; Zheng Yan; Iliyan Zarov; Yuchen Zhang; Angela Fan; Melanie Kambadur; Sharan Narang; Aurelien Rodriguez; Robert Stojnic; Sergey Edunov; Thomas ScialomLlama 2: Open Foundation and Fine-Tuned Chat Models2023-07-18 · publishedInspect the record ↗
- ContestsarXiv (preprint, Elizabeth Seger and 21 other authors; Centre for the Governance of AI and others)Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives2023-09-29 · publishedInspect the record ↗
- ContestsAnjali Gopal; Nathan Helm-Burger; Lenni Justen; Emily H. Soice; Tiffany Tzeng; Geetha Jeyapragasan; Simon Grimm; Benjamin Mueller; Kevin M. EsveltWill releasing the weights of large language models grant widespread access to pandemic agents?2023-10-25 · publishedInspect the record ↗
- ArticulatesMozilla (host); signatories listed below the statementJoint Statement on AI Safety and Openness2023-10-31 · publishedInspect the record ↗
- ArticulatesarXiv (preprint, Sayash Kapoor, Rishi Bommasani and others)On the Societal Impact of Open Foundation Models2024-02-27 · publishedInspect the record ↗
- ArticulatesArvind Narayanan; Sayash KapoorAI safety is not a model property2024-03-12 · publishedInspect the record ↗
- ArticulatesFrancisco Eiras; Aleksandar Petrov; Bertie Vidgen; and othersRisks and Opportunities of Open-Source Generative AI2024-05-14 · publishedInspect the record ↗
- Bears onNational Telecommunications and Information AdministrationDual-Use Foundation Models with Widely Available Model Weights2024-07 · publishedInspect the record ↗
- ContestsYoshua BengioReasoning through arguments against taking AI safety seriously2024-07-09 · publishedInspect the record ↗
- ArticulatesMark ZuckerbergOpen Source AI Is the Path Forward (2024)2024-07-23 · publishedInspect the record ↗
- ContestsDavid Gray Widder; Meredith Whittaker; Sarah Myers WestWhy 'open' AI systems are actually closed, and why this matters2024-11-27 · publishedInspect the record ↗
R16 Takeoff will be gradual
- Articulates · Earliest locatedRodney BrooksThe Seven Deadly Sins of Predicting the Future of AI2017-09-07 · publishedInspect the record ↗
- ContestsEliezer YudkowskyAlphaGo Zero and the Foom Debate2017-10-20 · publishedInspect the record ↗
- ContestsEliezer YudkowskyA reply to Francois Chollet on intelligence explosion2017-12-06 · publishedInspect the record ↗
- Articulates · Earliest located use of the name “slow takeoff”Paul ChristianoTakeoff speeds2018-02-24 · publishedInspect the record ↗
- ArticulatesArvind Narayanan; Sayash KapoorAI as Normal Technology2025-04-15 · publishedInspect the record ↗
R17 Economic incentives will produce safety
- Contests · Earliest locatedStuart Armstrong; Nick Bostrom; Carl ShulmanRacing to the Precipice: a Model of Artificial Intelligence Development2013-10 · publishedInspect the record ↗
- ContestsNick BostromStrategic Implications of Openness in AI Development2017 · publishedInspect the record ↗
- Contests · Earliest located use of the name “incentives”Amanda Askell; Miles Brundage; Gillian HadfieldThe Role of Cooperation in Responsible AI Development2019-07-10 · publishedInspect the record ↗
- ContestsDan HendrycksNatural Selection Favors AIs over Humans2023-03-28 · publishedInspect the record ↗
- Articulates · Earliest locatedRobin HansonWhat Are Reasonable AI Fears?2023-04-14 · publishedInspect the record ↗
- ContestsDan Hendrycks; Mantas Mazeika; Thomas WoodsideAn Overview of Catastrophic AI Risks2023-06-21 · publishedInspect the record ↗
- ContestsYoshua BengioReasoning through arguments against taking AI safety seriously2024-07-09 · publishedInspect the record ↗
- ContestsGeoffrey HintonBanquet speech, Nobel Banquet, 10 December 20242024-12-10Inspect the record ↗
- ArticulatesArvind Narayanan; Sayash KapoorAI as Normal Technology2025-04-15 · publishedInspect the record ↗
- ArticulatesArctotheriumAI Race Dynamics Are Not Always Bad2026-10-01 · publishedInspect the record ↗