{"entities":[{"id":"benevolence-or-no-drive-to-dominate","name":"Intelligence brings benevolence or no drive to dominate","type":"concept","proposition":{"code":"R10a","criterion":"Sufficiently intelligent AI systems will be benevolent, or will lack any drive to dominate or harm people, because greater intelligence brings good values or because such drives come from evolved motives that AI systems need not have.","terms":[]}},{"id":"bostrom-2012-superintelligent-will","name":"The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents","type":"document","subtype":"Research paper","description":"A research paper by Nick Bostrom, Author's site, dated May 2012. The author's PDF gives the journal issue, Minds and Machines 22(2), May 2012.","sourceIds":["source-rd-bostrom-2012-superintelligent-will"]},{"id":"instrumental-convergence","name":"Instrumental convergence","type":"concept","proposition":{"code":"R02","criterion":"Agents with a wide range of final goals would tend to pursue the same intermediate goals, such as self-preservation, keeping their goals unchanged, acquiring resources or power, or improving themselves, because these serve almost any final goal.","terms":["instrumental convergence","convergent instrumental","basic AI drives","power-seeking"]}},{"id":"orthogonality","name":"Orthogonality of intelligence and goals","type":"concept","proposition":{"code":"R01","criterion":"An AI system's level of intelligence and its final goals can vary independently: a highly intelligent system could pursue more or less any final goal, so high intelligence does not by itself bring goals that are benevolent or compatible with human interests.","terms":["orthogonality thesis","orthogonality"]}}],"claims":[{"id":"rd-r01-bostrom-2012-superintelligent-will","subject":"bostrom-2012-superintelligent-will","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"orthogonality"},"topic":"ideas","statement":"Bostrom's paper of 2012 states the orthogonality thesis: that intelligence and final goals are orthogonal axes along which possible agents can freely vary, so that more or less any level of intelligence could in principle be combined with more or less any final goal.","time":{"kind":"event","date":"2012-05","basis":"publication","note":"Date of publication, as the source gives it."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-r01-bostrom-2012-superintelligent-will-1","evidence-rd-r01-bostrom-2012-superintelligent-will-2"],"attributes":{"thought":{"roles":["most-developed","earliest-located-term"],"register":"nonfiction","term":"orthogonality thesis"}}},{"id":"rd-r02-bostrom-2012-superintelligent-will","subject":"bostrom-2012-superintelligent-will","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"instrumental-convergence"},"topic":"ideas","statement":"Bostrom's paper of 2012 states the instrumental convergence thesis: that several instrumental values would raise the chances of an agent's goal being realised for a wide range of final goals and situations, and so are likely to be pursued by many intelligent agents.","time":{"kind":"event","date":"2012-05","basis":"publication","note":"Date of publication, as the source gives it."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-r02-bostrom-2012-superintelligent-will-1","evidence-rd-r02-bostrom-2012-superintelligent-will-2"],"attributes":{"thought":{"roles":["most-developed","earliest-located-term"],"register":"nonfiction","term":"convergent instrumental"}}},{"id":"rd-r10a-bostrom-2012-superintelligent-will","subject":"bostrom-2012-superintelligent-will","predicate":"contests_proposition","object":{"kind":"entity","entityId":"benevolence-or-no-drive-to-dominate"},"topic":"ideas","statement":"Bostrom's paper of 2012 argues that it cannot be assumed that a superintelligence would share values such as benevolent concern for others, and that one with a goal such as calculating the decimals of pi would not necessarily limit its activities so as to spare human interests.","time":{"kind":"event","date":"2012-05","basis":"publication","note":"Date of publication, as the source gives it."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-r10a-bostrom-2012-superintelligent-will-1"],"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Disputes the premise that greater intelligence brings benevolent final values, by arguing that intelligence and final goals vary independently."}}}}],"sources":[{"id":"source-rd-bostrom-2012-superintelligent-will","documentId":"bostrom-2012-superintelligent-will","title":"The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents","authors":["Nick Bostrom"],"publisher":"Author's site (nickbostrom.com); published in Minds and Machines 22(2), May 2012","publicationDate":"2012-05","version":"Retrieved for the candidate pack on 2026-10-29 (SHA-256 4bf5ce7c3d974e4ef889b98a89b248861787baa986b8bc13afe401a2346cebaf). author's PDF headed '[Minds and Machines, Vol. 22, Iss. 2, May 2012]'; PDF metadata created 2012-03-17, modified 2021-12-24; the journal's typeset version was not compared. The cross-family check retrieved it again on 2026-10-31, with the same bytes.","evidenceUrl":"https://nickbostrom.com/superintelligentwill.pdf","language":"en"}],"evidence":[{"id":"evidence-rd-r01-bostrom-2012-superintelligent-will-1","claimId":"rd-r01-bostrom-2012-superintelligent-will","sourceId":"source-rd-bostrom-2012-superintelligent-will","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r01-bostrom-2012-superintelligent-will-2","claimId":"rd-r01-bostrom-2012-superintelligent-will","sourceId":"source-rd-bostrom-2012-superintelligent-will","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r02-bostrom-2012-superintelligent-will-1","claimId":"rd-r02-bostrom-2012-superintelligent-will","sourceId":"source-rd-bostrom-2012-superintelligent-will","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r02-bostrom-2012-superintelligent-will-2","claimId":"rd-r02-bostrom-2012-superintelligent-will","sourceId":"source-rd-bostrom-2012-superintelligent-will","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r10a-bostrom-2012-superintelligent-will-1","claimId":"rd-r10a-bostrom-2012-superintelligent-will","sourceId":"source-rd-bostrom-2012-superintelligent-will","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"}],"at":{"entities":[1790,1800,1852,1894],"claims":[5121,5131,5229],"sources":[1717],"evidence":[6786,6787,6805,6806,6984]},"index":{"byClaim":[["rd-r01-bostrom-2012-superintelligent-will",["propositions:risk-debate"]],["rd-r02-bostrom-2012-superintelligent-will",["propositions:risk-debate"]],["rd-r10a-bostrom-2012-superintelligent-will",["propositions:risk-debate"]]],"marks":[["bostrom-2012-superintelligent-will",[{"collectionId":"risk-debate","propositionId":"orthogonality","claimId":"rd-r01-bostrom-2012-superintelligent-will","text":"Earliest located use of the name “orthogonality thesis” in Arguments about catastrophic risk from advanced AI for R01 Orthogonality of intelligence and goals"},{"collectionId":"risk-debate","propositionId":"instrumental-convergence","claimId":"rd-r02-bostrom-2012-superintelligent-will","text":"Earliest located use of the name “convergent instrumental” in Arguments about catastrophic risk from advanced AI for R02 Instrumental convergence"}]]],"chronicle":{"years":[{"year":1863,"topics":{"ideas":5},"stated":0},{"year":1865,"topics":{"ideas":1},"stated":0},{"year":1872,"topics":{"ideas":7},"stated":0},{"year":1943,"topics":{"ideas":1},"stated":0},{"year":1947,"topics":{"ideas":2},"stated":0},{"year":1950,"topics":{"ideas":7},"stated":0},{"year":1951,"topics":{"ideas":5},"stated":0},{"year":1952,"topics":{"ideas":2},"stated":0},{"year":1955,"topics":{"ideas":31},"stated":0},{"year":1956,"topics":{"ideas":1},"stated":0},{"year":1957,"topics":{"ideas":1},"stated":0},{"year":1958,"topics":{"ideas":3},"stated":0},{"year":1959,"topics":{"ideas":6},"stated":0},{"year":1960,"topics":{"ideas":11},"stated":0},{"year":1961,"topics":{"ideas":10},"stated":0},{"year":1965,"topics":{"ideas":8},"stated":0},{"year":1966,"topics":{"ideas":20},"stated":0},{"year":1968,"topics":{"ideas":5},"stated":0},{"year":1969,"topics":{"ideas":2},"stated":0},{"year":1971,"topics":{"ideas":4},"stated":0},{"year":1972,"topics":{"ideas":1},"stated":0},{"year":1973,"topics":{"ideas":2},"stated":0},{"year":1974,"topics":{"ideas":2},"stated":0},{"year":1975,"topics":{"ideas":1},"stated":0},{"year":1976,"topics":{"ideas":1},"stated":0},{"year":1977,"topics":{"people":2,"ideas":8},"stated":0},{"year":1979,"topics":{"ideas":4},"stated":0},{"year":1980,"topics":{"ideas":2},"stated":0},{"year":1981,"topics":{"people":2},"stated":0},{"year":1982,"topics":{"ideas":4},"stated":0},{"year":1983,"topics":{"ideas":4},"stated":0},{"year":1984,"topics":{"ideas":21},"stated":0},{"year":1985,"topics":{"ideas":9},"stated":0},{"year":1986,"topics":{"ideas":18},"stated":0},{"year":1987,"topics":{"ideas":1},"stated":0},{"year":1988,"topics":{"ideas":6},"stated":0},{"year":1989,"topics":{"ideas":8},"stated":0},{"year":1990,"topics":{"ideas":5},"stated":0},{"year":1991,"topics":{"ideas":11},"stated":0},{"year":1992,"topics":{"ideas":9},"stated":0},{"year":1993,"topics":{"ideas":14,"people":3},"stated":0},{"year":1994,"topics":{"ideas":14},"stated":0},{"year":1995,"topics":{"ideas":10,"people":1},"stated":0},{"year":1996,"topics":{"ideas":71},"stated":0},{"year":1997,"topics":{"ideas":12},"stated":0},{"year":1998,"topics":{"ideas":49},"stated":0},{"year":1999,"topics":{"ideas":27,"people":1,"institutions":1},"stated":0},{"year":2000,"topics":{"ideas":62},"stated":0},{"year":2001,"topics":{"ideas":39},"stated":0},{"year":2002,"topics":{"ideas":30},"stated":0},{"year":2003,"topics":{"ideas":8},"stated":0},{"year":2004,"topics":{"ideas":12,"people":3},"stated":0},{"year":2005,"topics":{"ideas":10,"people":1},"stated":0},{"year":2006,"topics":{"ideas":26,"people":1},"stated":0},{"year":2007,"topics":{"ideas":20},"stated":0},{"year":2008,"topics":{"ideas":57,"people":3,"institutions":1},"stated":0},{"year":2009,"topics":{"ideas":10,"people":1},"stated":0},{"year":2010,"topics":{"people":3,"ideas":11,"institutions":1,"governance":4},"stated":0},{"year":2011,"topics":{"people":3,"ideas":7,"institutions":2},"stated":0},{"year":2012,"topics":{"people":4,"ideas":17,"institutions":2},"stated":0},{"year":2013,"topics":{"ideas":46,"people":4,"institutions":2,"capital":1},"stated":0},{"year":2014,"topics":{"institutions":2,"ideas":54,"people":5},"stated":0},{"year":2015,"topics":{"people":22,"ideas":122,"institutions":3,"capital":1},"stated":0},{"year":2016,"topics":{"ideas":88,"people":2},"stated":0},{"year":2017,"topics":{"people":16,"institutions":2,"ideas":128,"governance":2},"stated":0},{"year":2018,"topics":{"people":8,"ideas":158,"institutions":8,"governance":1},"stated":0},{"year":2019,"topics":{"capital":1,"compute":1,"people":10,"governance":19,"ideas":175,"institutions":4},"stated":0},{"year":2020,"topics":{"people":6,"ideas":98,"institutions":1,"governance":2},"stated":0},{"year":2021,"topics":{"people":27,"ideas":177,"compute":1,"institutions":23,"governance":4,"capital":6},"stated":0},{"year":2022,"topics":{"people":15,"ideas":217,"institutions":6,"governance":3,"compute":2,"capital":1},"stated":0},{"year":2023,"topics":{"capital":5,"compute":7,"institutions":23,"people":51,"governance":179,"ideas":389},"stated":0},{"year":2024,"topics":{"capital":12,"compute":19,"governance":282,"people":80,"ideas":178,"institutions":28},"stated":1},{"year":2025,"topics":{"governance":547,"compute":69,"capital":34,"people":90,"institutions":18,"ideas":178},"stated":7},{"year":2026,"topics":{"governance":442,"capital":8,"compute":32,"institutions":21,"ideas":91,"people":107},"stated":19},{"year":2027,"topics":{},"stated":9},{"year":2028,"topics":{},"stated":6},{"year":2029,"topics":{},"stated":3},{"year":2030,"topics":{},"stated":3},{"year":2032,"topics":{},"stated":2},{"year":2035,"topics":{},"stated":2},{"year":2039,"topics":{},"stated":1},{"year":2042,"topics":{},"stated":1},{"year":2050,"topics":{},"stated":1}],"scheduled":[],"statedBy":[]},"collections":[{"target":{"kind":"propositions","id":"superintelligence"},"title":"Propositions about superintelligence, 1863–2022","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"risk-debate"},"title":"Arguments about catastrophic risk from advanced AI","claimIds":["rd-r01-bostrom-2012-superintelligent-will","rd-r02-bostrom-2012-superintelligent-will","rd-r10a-bostrom-2012-superintelligent-will"],"records":[]},{"target":{"kind":"propositions","id":"yudkowsky-arguments"},"title":"Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"economics-of-machine-intelligence"},"title":"Economics of machine intelligence","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"orbital-compute-arguments"},"title":"Arguments about compute in orbit","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"ai-bio-arguments"},"title":"Arguments about AI and biological risk","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"ai-in-science"},"title":"AI in science","claimIds":[],"records":[]},{"target":{"kind":"documents"},"title":"Frontier safety frameworks","claimIds":[]},{"target":{"kind":"compare","id":"frontier-safety-frameworks"},"title":"Compare frameworks","claimIds":[]},{"target":{"kind":"control","id":"openai"},"title":"OpenAI’s control structure, 2015–2026","claimIds":[]},{"target":{"kind":"control","id":"anthropic"},"title":"Anthropic’s control structure, 2021–2026","claimIds":[]},{"target":{"kind":"commitments"},"title":"External commitments and obligations","claimIds":[]},{"target":{"kind":"question","id":"frameworks-to-obligations"},"title":"Frameworks, commitments and legal obligations, 2023–2026","claimIds":[]},{"target":{"kind":"reading"},"title":"Founding and early leadership","claimIds":[]}]},"version":"131evrz"}