{"entities":[{"id":"bengio-2023-managing-ai-risks","name":"Managing AI Risks in an Era of Rapid Progress","type":"document","subtype":"Preprint","description":"A preprint by Yoshua Bengio and 23 co-authors, arXiv:2310.17688, dated 26 October 2023. arXiv v1 stamp \"26 Oct 2023\" (submission date).","sourceIds":["source-rd-bengio-2023-managing-ai-risks"]},{"id":"capabilities-plateau","name":"Capabilities will plateau","type":"concept","proposition":{"code":"R13","criterion":"Current approaches to AI (deep learning, or the scaling of large language models) will reach limits short of general or superhuman intelligence.","terms":["hitting a wall","plateau","diminishing returns"]}},{"id":"loss-of-control-route","name":"Existential risk from misalignment or loss of control","type":"concept","proposition":{"code":"R08","criterion":"Advanced AI systems pursuing goals at odds with human interests could escape human control and cause human extinction or permanent human disempowerment, without any person intending that outcome.","terms":["loss of control","rogue AI","misaligned AI","disempowerment"]}}],"claims":[{"id":"rd-r08-bengio-2023-managing-ai-risks","subject":"bengio-2023-managing-ai-risks","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"loss-of-control-route"},"topic":"ideas","statement":"The consensus paper by Bengio, Hinton, Yao and others of 26 October 2023 states that, without sufficient caution, humanity may irreversibly lose control of autonomous AI systems, which could culminate in large-scale loss of life and the marginalisation or extinction of humanity.","time":{"kind":"event","date":"2023-10-26","basis":"publication","note":"Date of the arXiv v1 submission."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-r08-bengio-2023-managing-ai-risks-1","evidence-rd-r08-bengio-2023-managing-ai-risks-2","evidence-rd-r08-bengio-2023-managing-ai-risks-3"],"attributes":{"thought":{"roles":["articulation"],"register":"nonfiction"}}},{"id":"rd-r13-bengio-2023-managing-ai-risks","subject":"bengio-2023-managing-ai-risks","predicate":"contests_proposition","object":{"kind":"entity","entityId":"capabilities-plateau"},"topic":"ideas","statement":"Bengio and twenty-three co-authors' paper of October 2023 states that there is no fundamental reason why AI progress would slow or halt at the human level, noting that AI already surpasses humans in narrow domains and is increasingly used to improve AI systems.","time":{"kind":"event","date":"2023-10-26","basis":"publication","note":"Date of the arXiv v1 submission."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-r13-bengio-2023-managing-ai-risks-1"],"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"It states that there is no fundamental limit at human level, without engaging a named argument for a plateau."}}}}],"sources":[{"id":"source-rd-bengio-2023-managing-ai-risks","documentId":"bengio-2023-managing-ai-risks","title":"Managing AI Risks in an Era of Rapid Progress","authors":["Yoshua Bengio","Geoffrey Hinton","Andrew Yao","Dawn Song","Pieter Abbeel","Yuval Noah Harari","Ya-Qin Zhang","Lan Xue","Shai Shalev-Shwartz","Gillian Hadfield","Jeff Clune","Tegan Maharaj","Frank Hutter","Atılım Güneş Baydin","Sheila McIlraith","Qiqi Gao","Ashwin Acharya","David Krueger","Anca Dragan","Philip Torr","Stuart Russell","Daniel Kahneman","Jan Brauner","Sören Mindermann"],"publisher":"arXiv:2310.17688","publicationDate":"2023-10-26","version":"Retrieved for the candidate pack on 2026-10-29 (SHA-256 d6ec9c8bb5db28ea6c52aff5cfb79a7c510244a9cb71dea01f3927f8b02afcc7). arXiv v1 PDF (2310.17688v1, 26 Oct 2023), titled \"Managing AI Risks in an Era of Rapid Progress\"; a revised version was published in Science in May 2024 as \"Managing extreme AI risks amid rapid progress\" and was not compared. The cross-family check retrieved it again on 2026-10-31, with the same bytes.","evidenceUrl":"https://arxiv.org/pdf/2310.17688v1","language":"en"}],"evidence":[{"id":"evidence-rd-r08-bengio-2023-managing-ai-risks-1","claimId":"rd-r08-bengio-2023-managing-ai-risks","sourceId":"source-rd-bengio-2023-managing-ai-risks","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r08-bengio-2023-managing-ai-risks-2","claimId":"rd-r08-bengio-2023-managing-ai-risks","sourceId":"source-rd-bengio-2023-managing-ai-risks","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r08-bengio-2023-managing-ai-risks-3","claimId":"rd-r08-bengio-2023-managing-ai-risks","sourceId":"source-rd-bengio-2023-managing-ai-risks","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r13-bengio-2023-managing-ai-risks-1","claimId":"rd-r13-bengio-2023-managing-ai-risks","sourceId":"source-rd-bengio-2023-managing-ai-risks","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"}],"at":{"entities":[1793,1808,1865],"claims":[5214,5270],"sources":[1710],"evidence":[6955,6956,6957,7050]},"index":{"byClaim":[["rd-r08-bengio-2023-managing-ai-risks",["propositions:risk-debate"]],["rd-r13-bengio-2023-managing-ai-risks",["propositions:risk-debate"]]],"marks":[],"chronicle":{"years":[{"year":1863,"topics":{"ideas":5},"stated":0},{"year":1865,"topics":{"ideas":1},"stated":0},{"year":1872,"topics":{"ideas":7},"stated":0},{"year":1943,"topics":{"ideas":1},"stated":0},{"year":1947,"topics":{"ideas":2},"stated":0},{"year":1950,"topics":{"ideas":7},"stated":0},{"year":1951,"topics":{"ideas":5},"stated":0},{"year":1952,"topics":{"ideas":2},"stated":0},{"year":1955,"topics":{"ideas":31},"stated":0},{"year":1956,"topics":{"ideas":1},"stated":0},{"year":1957,"topics":{"ideas":1},"stated":0},{"year":1958,"topics":{"ideas":3},"stated":0},{"year":1959,"topics":{"ideas":6},"stated":0},{"year":1960,"topics":{"ideas":11},"stated":0},{"year":1961,"topics":{"ideas":10},"stated":0},{"year":1965,"topics":{"ideas":8},"stated":0},{"year":1966,"topics":{"ideas":20},"stated":0},{"year":1968,"topics":{"ideas":5},"stated":0},{"year":1969,"topics":{"ideas":2},"stated":0},{"year":1971,"topics":{"ideas":4},"stated":0},{"year":1972,"topics":{"ideas":1},"stated":0},{"year":1973,"topics":{"ideas":2},"stated":0},{"year":1974,"topics":{"ideas":2},"stated":0},{"year":1975,"topics":{"ideas":1},"stated":0},{"year":1976,"topics":{"ideas":1},"stated":0},{"year":1977,"topics":{"people":2,"ideas":8},"stated":0},{"year":1979,"topics":{"ideas":4},"stated":0},{"year":1980,"topics":{"ideas":2},"stated":0},{"year":1981,"topics":{"people":2},"stated":0},{"year":1982,"topics":{"ideas":4},"stated":0},{"year":1983,"topics":{"ideas":4},"stated":0},{"year":1984,"topics":{"ideas":21},"stated":0},{"year":1985,"topics":{"ideas":9},"stated":0},{"year":1986,"topics":{"ideas":18},"stated":0},{"year":1987,"topics":{"ideas":1},"stated":0},{"year":1988,"topics":{"ideas":6},"stated":0},{"year":1989,"topics":{"ideas":8},"stated":0},{"year":1990,"topics":{"ideas":5},"stated":0},{"year":1991,"topics":{"ideas":11},"stated":0},{"year":1992,"topics":{"ideas":9},"stated":0},{"year":1993,"topics":{"ideas":14,"people":3},"stated":0},{"year":1994,"topics":{"ideas":14},"stated":0},{"year":1995,"topics":{"ideas":10,"people":1},"stated":0},{"year":1996,"topics":{"ideas":71},"stated":0},{"year":1997,"topics":{"ideas":12},"stated":0},{"year":1998,"topics":{"ideas":49},"stated":0},{"year":1999,"topics":{"ideas":27,"people":1,"institutions":1},"stated":0},{"year":2000,"topics":{"ideas":62},"stated":0},{"year":2001,"topics":{"ideas":39},"stated":0},{"year":2002,"topics":{"ideas":30},"stated":0},{"year":2003,"topics":{"ideas":8},"stated":0},{"year":2004,"topics":{"ideas":12,"people":3},"stated":0},{"year":2005,"topics":{"ideas":10,"people":1},"stated":0},{"year":2006,"topics":{"ideas":26,"people":1},"stated":0},{"year":2007,"topics":{"ideas":20},"stated":0},{"year":2008,"topics":{"ideas":57,"people":3,"institutions":1},"stated":0},{"year":2009,"topics":{"ideas":10,"people":1},"stated":0},{"year":2010,"topics":{"people":3,"ideas":11,"institutions":1,"governance":4},"stated":0},{"year":2011,"topics":{"people":3,"ideas":7,"institutions":2},"stated":0},{"year":2012,"topics":{"people":4,"ideas":17,"institutions":2},"stated":0},{"year":2013,"topics":{"ideas":46,"people":4,"institutions":2,"capital":1},"stated":0},{"year":2014,"topics":{"institutions":2,"ideas":54,"people":5},"stated":0},{"year":2015,"topics":{"people":22,"ideas":122,"institutions":3,"capital":1},"stated":0},{"year":2016,"topics":{"ideas":88,"people":2},"stated":0},{"year":2017,"topics":{"people":16,"institutions":2,"ideas":128,"governance":2},"stated":0},{"year":2018,"topics":{"people":8,"ideas":158,"institutions":8,"governance":1},"stated":0},{"year":2019,"topics":{"capital":1,"compute":1,"people":10,"governance":19,"ideas":175,"institutions":4},"stated":0},{"year":2020,"topics":{"people":6,"ideas":98,"institutions":1,"governance":2},"stated":0},{"year":2021,"topics":{"people":27,"ideas":177,"compute":1,"institutions":23,"governance":4,"capital":6},"stated":0},{"year":2022,"topics":{"people":15,"ideas":217,"institutions":6,"governance":3,"compute":2,"capital":1},"stated":0},{"year":2023,"topics":{"capital":5,"compute":7,"institutions":23,"people":51,"governance":179,"ideas":389},"stated":0},{"year":2024,"topics":{"capital":12,"compute":19,"governance":282,"people":80,"ideas":178,"institutions":28},"stated":1},{"year":2025,"topics":{"governance":547,"compute":69,"capital":34,"people":90,"institutions":18,"ideas":178},"stated":7},{"year":2026,"topics":{"governance":442,"capital":8,"compute":32,"institutions":21,"ideas":91,"people":107},"stated":19},{"year":2027,"topics":{},"stated":9},{"year":2028,"topics":{},"stated":6},{"year":2029,"topics":{},"stated":3},{"year":2030,"topics":{},"stated":3},{"year":2032,"topics":{},"stated":2},{"year":2035,"topics":{},"stated":2},{"year":2039,"topics":{},"stated":1},{"year":2042,"topics":{},"stated":1},{"year":2050,"topics":{},"stated":1}],"scheduled":[],"statedBy":[]},"collections":[{"target":{"kind":"propositions","id":"superintelligence"},"title":"Propositions about superintelligence, 1863–2022","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"risk-debate"},"title":"Arguments about catastrophic risk from advanced AI","claimIds":["rd-r08-bengio-2023-managing-ai-risks","rd-r13-bengio-2023-managing-ai-risks"],"records":[]},{"target":{"kind":"propositions","id":"yudkowsky-arguments"},"title":"Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"economics-of-machine-intelligence"},"title":"Economics of machine intelligence","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"orbital-compute-arguments"},"title":"Arguments about compute in orbit","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"ai-bio-arguments"},"title":"Arguments about AI and biological risk","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"ai-in-science"},"title":"AI in science","claimIds":[],"records":[]},{"target":{"kind":"documents"},"title":"Frontier safety frameworks","claimIds":[]},{"target":{"kind":"compare","id":"frontier-safety-frameworks"},"title":"Compare frameworks","claimIds":[]},{"target":{"kind":"control","id":"openai"},"title":"OpenAI’s control structure, 2015–2026","claimIds":[]},{"target":{"kind":"control","id":"anthropic"},"title":"Anthropic’s control structure, 2021–2026","claimIds":[]},{"target":{"kind":"commitments"},"title":"External commitments and obligations","claimIds":[]},{"target":{"kind":"question","id":"frameworks-to-obligations"},"title":"Frameworks, commitments and legal obligations, 2023–2026","claimIds":[]},{"target":{"kind":"reading"},"title":"Founding and early leadership","claimIds":[]}]},"version":"131evrz"}