{"entities":[{"id":"fast-or-hard-takeoff","name":"Fast or hard takeoff","type":"concept","proposition":{"code":"A4","criterion":"The transition from roughly human-level to far superhuman capability happens over a short period (hours to months), or is named as a “hard” or “fast” takeoff, as opposed to a gradual transition over years or decades. A transition described only as rapid, or rapid relative to biological or historical change, without such a period, does not meet the criterion.","terms":["hard takeoff","fast takeoff"]}},{"id":"bostrom-2003-ethical-issues","name":"Ethical Issues in Advanced Artificial Intelligence","type":"document","subtype":"Research paper","description":"A research paper by Nick Bostrom, Author's site, dated 2003. Year of the 2003 volume named on the author's page.","sourceIds":["source-rd-bostrom-2003-ethical-issues"]},{"id":"instrumental-convergence","name":"Instrumental convergence","type":"concept","proposition":{"code":"R02","criterion":"Agents with a wide range of final goals would tend to pursue the same intermediate goals, such as self-preservation, keeping their goals unchanged, acquiring resources or power, or improving themselves, because these serve almost any final goal.","terms":["instrumental convergence","convergent instrumental","basic AI drives","power-seeking"]}},{"id":"orthogonality","name":"Orthogonality of intelligence and goals","type":"concept","proposition":{"code":"R01","criterion":"An AI system's level of intelligence and its final goals can vary independently: a highly intelligent system could pursue more or less any final goal, so high intelligence does not by itself bring goals that are benevolent or compatible with human interests.","terms":["orthogonality thesis","orthogonality"]}}],"claims":[{"id":"rd-r01-bostrom-2003-ethical-issues","subject":"bostrom-2003-ethical-issues","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"orthogonality"},"topic":"ideas","statement":"Bostrom's paper of 2003, in its revised version on the author's site, states that a superintelligence could have a completely arbitrary sole goal, such as making as many paperclips as possible, and that artificial intellects need not share human motivational tendencies.","time":{"kind":"event","date":"2003","basis":"publication","note":"Date of publication, as the source gives it."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-r01-bostrom-2003-ethical-issues-1","evidence-rd-r01-bostrom-2003-ethical-issues-2"],"attributes":{"thought":{"roles":["earliest-located"],"register":"nonfiction","attested":{"date":"2003-08-06","note":"Read in the 'slightly revised version' on the author's site. The cross-family check of 2026-10-31 found the quoted wording, in substance, in an archive capture of the author's page of 6 August 2003, which already describes itself as slightly revised; the printed proceedings were not obtained."}}}},{"id":"rd-r02-bostrom-2003-ethical-issues","subject":"bostrom-2003-ethical-issues","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"instrumental-convergence"},"topic":"ideas","statement":"Bostrom's paper of 2003, in its revised version on the author's site, argues that an agent with a definite top goal would generally find it irrational to change that goal, since changing it would make the goal less likely to be attained.","time":{"kind":"event","date":"2003","basis":"publication","note":"Date of publication, as the source gives it."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-r02-bostrom-2003-ethical-issues-1"],"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"States that an agent would keep its top goal unchanged, but not that agents with a wide range of final goals would converge on several intermediate goals such as resources, self-preservation or self-improvement.","attested":{"note":"The wording is read in an undated 'slightly revised version' on the author's site; the 2003 printed wording was not compared, so the date from which this wording is attested is unknown."}}}},{"id":"rd-a4-bostrom-2003-ethical-issues","subject":"bostrom-2003-ethical-issues","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"fast-or-hard-takeoff"},"topic":"ideas","statement":"Bostrom's paper of 2003, in the revised version on the author's site, states that the transition from roughly human-level artificial intelligence to full superintelligence may be very rapid, perhaps a matter of days rather than years, and names this possibility the singularity hypothesis.","time":{"kind":"event","date":"2003","basis":"publication","note":"Date of publication, as the source gives it."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-a4-bostrom-2003-ethical-issues-1"],"attributes":{"thought":{"roles":["articulation"],"register":"nonfiction","attested":{"date":"2003-08-06","note":"Read in the 'slightly revised version' on the author's site. The cross-family check of 2026-10-31 found the days-rather-than-years wording, in substance, in an archive capture of the author's page of 6 August 2003; the printed proceedings were not obtained."}}}}],"sources":[{"id":"source-rd-bostrom-2003-ethical-issues","documentId":"bostrom-2003-ethical-issues","title":"Ethical Issues in Advanced Artificial Intelligence","authors":["Nick Bostrom"],"publisher":"Author's site (nickbostrom.com); first published in Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence, Vol. 2, ed. I. Smit et al., 2003","publicationDate":"2003","version":"Retrieved for the candidate pack on 2026-10-29 (SHA-256 eac7d99f224b72384bc6850a581a3dfda6da21b155ebaa5ba1843a4baf7106b8). Current web page on the author's site, retrieved 2026-10-29; its reference list cites a 2003 edition of Creating Friendly AI, so the page may have been revised after 2003. The cross-family check retrieved it again on 2026-10-31, with the same bytes. The cross-family check of 2026-10-31 found the quoted R01 passage on arbitrary goals and the A4 passage on a transition of days rather than years, in substance, in an archive capture of the author's page of 6 August 2003, which already describes itself as slightly revised. The printed proceedings (pp. 12–17) were not obtained, so this is not a claim that the wording matches them.","evidenceUrl":"https://nickbostrom.com/papers/ethical-issues-in-advanced-ai/","language":"en","archivedUrl":"https://web.archive.org/web/20030806225530/http://www.nickbostrom.com:80/ethics/ai.html","archivedAt":"2003-08-06","accessedAt":"2026-10-29"}],"evidence":[{"id":"evidence-rd-r01-bostrom-2003-ethical-issues-1","claimId":"rd-r01-bostrom-2003-ethical-issues","sourceId":"source-rd-bostrom-2003-ethical-issues","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r01-bostrom-2003-ethical-issues-2","claimId":"rd-r01-bostrom-2003-ethical-issues","sourceId":"source-rd-bostrom-2003-ethical-issues","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r02-bostrom-2003-ethical-issues-1","claimId":"rd-r02-bostrom-2003-ethical-issues","sourceId":"source-rd-bostrom-2003-ethical-issues","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-a4-bostrom-2003-ethical-issues-1","claimId":"rd-a4-bostrom-2003-ethical-issues","sourceId":"source-rd-bostrom-2003-ethical-issues","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"}],"at":{"entities":[339,1799,1852,1894],"claims":[5128,5148,5180],"sources":[1716],"evidence":[6799,6800,6829,6893]},"index":{"byClaim":[["rd-a4-bostrom-2003-ethical-issues",["propositions:superintelligence","propositions:risk-debate"]],["rd-r01-bostrom-2003-ethical-issues",["propositions:risk-debate"]],["rd-r02-bostrom-2003-ethical-issues",["propositions:risk-debate"]]],"marks":[["bostrom-2003-ethical-issues",[{"collectionId":"risk-debate","propositionId":"orthogonality","claimId":"rd-r01-bostrom-2003-ethical-issues","text":"Earliest located in Arguments about catastrophic risk from advanced AI for R01 Orthogonality of intelligence and goals"}]]],"chronicle":{"years":[{"year":1863,"topics":{"ideas":5},"stated":0},{"year":1865,"topics":{"ideas":1},"stated":0},{"year":1872,"topics":{"ideas":7},"stated":0},{"year":1943,"topics":{"ideas":1},"stated":0},{"year":1947,"topics":{"ideas":2},"stated":0},{"year":1950,"topics":{"ideas":7},"stated":0},{"year":1951,"topics":{"ideas":5},"stated":0},{"year":1952,"topics":{"ideas":2},"stated":0},{"year":1955,"topics":{"ideas":31},"stated":0},{"year":1956,"topics":{"ideas":1},"stated":0},{"year":1957,"topics":{"ideas":1},"stated":0},{"year":1958,"topics":{"ideas":3},"stated":0},{"year":1959,"topics":{"ideas":6},"stated":0},{"year":1960,"topics":{"ideas":11},"stated":0},{"year":1961,"topics":{"ideas":10},"stated":0},{"year":1965,"topics":{"ideas":8},"stated":0},{"year":1966,"topics":{"ideas":20},"stated":0},{"year":1968,"topics":{"ideas":5},"stated":0},{"year":1969,"topics":{"ideas":2},"stated":0},{"year":1971,"topics":{"ideas":4},"stated":0},{"year":1972,"topics":{"ideas":1},"stated":0},{"year":1973,"topics":{"ideas":2},"stated":0},{"year":1974,"topics":{"ideas":2},"stated":0},{"year":1975,"topics":{"ideas":1},"stated":0},{"year":1976,"topics":{"ideas":1},"stated":0},{"year":1977,"topics":{"people":2,"ideas":8},"stated":0},{"year":1979,"topics":{"ideas":4},"stated":0},{"year":1980,"topics":{"ideas":2},"stated":0},{"year":1981,"topics":{"people":2},"stated":0},{"year":1982,"topics":{"ideas":4},"stated":0},{"year":1983,"topics":{"ideas":4},"stated":0},{"year":1984,"topics":{"ideas":21},"stated":0},{"year":1985,"topics":{"ideas":9},"stated":0},{"year":1986,"topics":{"ideas":18},"stated":0},{"year":1987,"topics":{"ideas":1},"stated":0},{"year":1988,"topics":{"ideas":6},"stated":0},{"year":1989,"topics":{"ideas":8},"stated":0},{"year":1990,"topics":{"ideas":5},"stated":0},{"year":1991,"topics":{"ideas":11},"stated":0},{"year":1992,"topics":{"ideas":9},"stated":0},{"year":1993,"topics":{"ideas":14,"people":3},"stated":0},{"year":1994,"topics":{"ideas":14},"stated":0},{"year":1995,"topics":{"ideas":10,"people":1},"stated":0},{"year":1996,"topics":{"ideas":71},"stated":0},{"year":1997,"topics":{"ideas":12},"stated":0},{"year":1998,"topics":{"ideas":49},"stated":0},{"year":1999,"topics":{"ideas":27,"people":1,"institutions":1},"stated":0},{"year":2000,"topics":{"ideas":62},"stated":0},{"year":2001,"topics":{"ideas":39},"stated":0},{"year":2002,"topics":{"ideas":30},"stated":0},{"year":2003,"topics":{"ideas":8},"stated":0},{"year":2004,"topics":{"ideas":12,"people":3},"stated":0},{"year":2005,"topics":{"ideas":10,"people":1},"stated":0},{"year":2006,"topics":{"ideas":26,"people":1},"stated":0},{"year":2007,"topics":{"ideas":20},"stated":0},{"year":2008,"topics":{"ideas":57,"people":3,"institutions":1},"stated":0},{"year":2009,"topics":{"ideas":10,"people":1},"stated":0},{"year":2010,"topics":{"people":3,"ideas":11,"institutions":1,"governance":4},"stated":0},{"year":2011,"topics":{"people":3,"ideas":7,"institutions":2},"stated":0},{"year":2012,"topics":{"people":4,"ideas":17,"institutions":2},"stated":0},{"year":2013,"topics":{"ideas":46,"people":4,"institutions":2,"capital":1},"stated":0},{"year":2014,"topics":{"institutions":2,"ideas":54,"people":5},"stated":0},{"year":2015,"topics":{"people":22,"ideas":122,"institutions":3,"capital":1},"stated":0},{"year":2016,"topics":{"ideas":88,"people":2},"stated":0},{"year":2017,"topics":{"people":16,"institutions":2,"ideas":128,"governance":2},"stated":0},{"year":2018,"topics":{"people":8,"ideas":158,"institutions":8,"governance":1},"stated":0},{"year":2019,"topics":{"capital":1,"compute":1,"people":10,"governance":19,"ideas":175,"institutions":4},"stated":0},{"year":2020,"topics":{"people":6,"ideas":98,"institutions":1,"governance":2},"stated":0},{"year":2021,"topics":{"people":27,"ideas":177,"compute":1,"institutions":23,"governance":4,"capital":6},"stated":0},{"year":2022,"topics":{"people":15,"ideas":217,"institutions":6,"governance":3,"compute":2,"capital":1},"stated":0},{"year":2023,"topics":{"capital":5,"compute":7,"institutions":23,"people":51,"governance":179,"ideas":389},"stated":0},{"year":2024,"topics":{"capital":12,"compute":19,"governance":282,"people":80,"ideas":178,"institutions":28},"stated":1},{"year":2025,"topics":{"governance":547,"compute":69,"capital":34,"people":90,"institutions":18,"ideas":178},"stated":7},{"year":2026,"topics":{"governance":442,"capital":8,"compute":32,"institutions":21,"ideas":91,"people":107},"stated":19},{"year":2027,"topics":{},"stated":9},{"year":2028,"topics":{},"stated":6},{"year":2029,"topics":{},"stated":3},{"year":2030,"topics":{},"stated":3},{"year":2032,"topics":{},"stated":2},{"year":2035,"topics":{},"stated":2},{"year":2039,"topics":{},"stated":1},{"year":2042,"topics":{},"stated":1},{"year":2050,"topics":{},"stated":1}],"scheduled":[],"statedBy":[]},"collections":[{"target":{"kind":"propositions","id":"superintelligence"},"title":"Propositions about superintelligence, 1863–2022","claimIds":["rd-a4-bostrom-2003-ethical-issues"],"records":[]},{"target":{"kind":"propositions","id":"risk-debate"},"title":"Arguments about catastrophic risk from advanced AI","claimIds":["rd-r01-bostrom-2003-ethical-issues","rd-r02-bostrom-2003-ethical-issues","rd-a4-bostrom-2003-ethical-issues"],"records":[]},{"target":{"kind":"propositions","id":"yudkowsky-arguments"},"title":"Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"economics-of-machine-intelligence"},"title":"Economics of machine intelligence","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"orbital-compute-arguments"},"title":"Arguments about compute in orbit","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"ai-bio-arguments"},"title":"Arguments about AI and biological risk","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"ai-in-science"},"title":"AI in science","claimIds":[],"records":[]},{"target":{"kind":"documents"},"title":"Frontier safety frameworks","claimIds":[]},{"target":{"kind":"compare","id":"frontier-safety-frameworks"},"title":"Compare frameworks","claimIds":[]},{"target":{"kind":"control","id":"openai"},"title":"OpenAI’s control structure, 2015–2026","claimIds":[]},{"target":{"kind":"control","id":"anthropic"},"title":"Anthropic’s control structure, 2021–2026","claimIds":[]},{"target":{"kind":"commitments"},"title":"External commitments and obligations","claimIds":[]},{"target":{"kind":"question","id":"frameworks-to-obligations"},"title":"Frameworks, commitments and legal obligations, 2023–2026","claimIds":[]},{"target":{"kind":"reading"},"title":"Founding and early leadership","claimIds":[]}]},"version":"131evrz"}