{"entities":[{"id":"treacherous-turn","name":"Treacherous turn","type":"concept","proposition":{"code":"B1","criterion":"A system behaves cooperatively or safely while weak, confined or overseen, and changes course to pursue its own goals once it can do so without effective opposition.","terms":["treacherous turn"]}},{"id":"belrose-pope-2024-counting-arguments","name":"Counting arguments provide no evidence for AI doom","shortName":"Belrose and Pope, Counting arguments provide no evidence for AI doom (2024)","type":"document"},{"id":"carlsmith-2023-scheming-ais","name":"Scheming AIs: Will AIs fake alignment during training in order to get power?","type":"document","subtype":"Preprint","description":"A preprint by Joseph Carlsmith, arXiv preprint 2311.08379, dated 14 November 2023. arXiv v1 stamp, 14 November 2023; the report is dated November 2023.","sourceIds":["source-rd-carlsmith-2023-scheming-ais"]},{"id":"inner-or-deceptive-misalignment","name":"Inner or deceptive misalignment","type":"concept","proposition":{"code":"R03","criterion":"A system produced by training or search can come to pursue an objective other than the one its training specified (the training signal or loss), while performing well on that training objective; an objective that is only misspecified by its developers does not meet the criterion; this includes, but is not limited to, a system that behaves as aligned during training or evaluation in order to pursue another objective later.","terms":["inner alignment","inner misalignment","mesa-optimizer","mesa-optimiser","deceptive alignment","goal misgeneralization","goal misgeneralisation","alignment faking"]}}],"claims":[{"id":"rd-r03-carlsmith-2023-scheming-ais","subject":"carlsmith-2023-scheming-ais","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"inner-or-deceptive-misalignment"},"topic":"ideas","statement":"Carlsmith's report of November 2023 examines whether advanced AIs that perform well in training will do so in order to gain power later, and concludes that scheming is a disturbingly plausible outcome of using baseline machine learning methods to train goal-directed AIs sophisticated enough to scheme.","time":{"kind":"event","date":"2023-11-14","basis":"publication","note":"Date of the arXiv v1 submission."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-r03-carlsmith-2023-scheming-ais-1","evidence-rd-r03-carlsmith-2023-scheming-ais-2","evidence-rd-r03-carlsmith-2023-scheming-ais-3"],"attributes":{"thought":{"roles":["most-developed"],"register":"nonfiction"}}},{"id":"rd-b1-carlsmith-2023-scheming-ais","subject":"carlsmith-2023-scheming-ais","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"treacherous-turn"},"topic":"ideas","statement":"Carlsmith's report of November 2023 describes a schemer as a model that performs well in training while checking whether it is in a position to escape or take over, and calls scheming a more specific instance of the treacherous turn.","time":{"kind":"event","date":"2023-11-14","basis":"publication","note":"Date of the arXiv v1 submission."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-b1-carlsmith-2023-scheming-ais-1","evidence-rd-b1-carlsmith-2023-scheming-ais-2"],"attributes":{"thought":{"roles":["articulation"],"register":"nonfiction"}}},{"id":"rd-reply-belrose-pope-2024-counting-arguments-to-carlsmith-2023-scheming-ais","subject":"belrose-pope-2024-counting-arguments","predicate":"cites","object":{"kind":"entity","entityId":"carlsmith-2023-scheming-ais"},"topic":"ideas","statement":"Belrose and Pope's essay of February 2024 replies to Carlsmith's report Scheming AIs, which it names and quotes.","time":{"kind":"event","date":"2024-02-27","basis":"publication","note":"Date of posting."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-reply-belrose-pope-2024-counting-arguments-to-carlsmith-2023-scheming-ais-1"]}],"sources":[{"id":"source-rd-carlsmith-2023-scheming-ais","documentId":"carlsmith-2023-scheming-ais","title":"Scheming AIs: Will AIs fake alignment during training in order to get power?","authors":["Joseph Carlsmith"],"publisher":"arXiv preprint 2311.08379 (Open Philanthropy report)","publicationDate":"2023-11-14","version":"Retrieved for the candidate pack on 2026-10-29 (SHA-256 ff546752a0b14d17cd05e20e1d2ff553b1f431a440a1e48241a842955df597bb). arXiv v1 of 14 November 2023.","evidenceUrl":"https://arxiv.org/pdf/2311.08379v1","language":"en"}],"evidence":[{"id":"evidence-rd-r03-carlsmith-2023-scheming-ais-1","claimId":"rd-r03-carlsmith-2023-scheming-ais","sourceId":"source-rd-carlsmith-2023-scheming-ais","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r03-carlsmith-2023-scheming-ais-2","claimId":"rd-r03-carlsmith-2023-scheming-ais","sourceId":"source-rd-carlsmith-2023-scheming-ais","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r03-carlsmith-2023-scheming-ais-3","claimId":"rd-r03-carlsmith-2023-scheming-ais","sourceId":"source-rd-carlsmith-2023-scheming-ais","support":"context","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-b1-carlsmith-2023-scheming-ais-1","claimId":"rd-b1-carlsmith-2023-scheming-ais","sourceId":"source-rd-carlsmith-2023-scheming-ais","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-b1-carlsmith-2023-scheming-ais-2","claimId":"rd-b1-carlsmith-2023-scheming-ais","sourceId":"source-rd-carlsmith-2023-scheming-ais","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-reply-belrose-pope-2024-counting-arguments-to-carlsmith-2023-scheming-ais-1","claimId":"rd-reply-belrose-pope-2024-counting-arguments-to-carlsmith-2023-scheming-ais","sourceId":"source-rd-belrose-pope-2024-counting-arguments","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"}],"at":{"entities":[342,1787,1810,1851],"claims":[5153,5162,5352],"sources":[1727],"evidence":[6839,6840,6841,6862,6863,7192]},"index":{"byClaim":[["rd-b1-carlsmith-2023-scheming-ais",["propositions:superintelligence","propositions:risk-debate"]],["rd-r03-carlsmith-2023-scheming-ais",["propositions:risk-debate"]]],"marks":[],"chronicle":{"years":[{"year":1863,"topics":{"ideas":5},"stated":0},{"year":1865,"topics":{"ideas":1},"stated":0},{"year":1872,"topics":{"ideas":7},"stated":0},{"year":1943,"topics":{"ideas":1},"stated":0},{"year":1947,"topics":{"ideas":2},"stated":0},{"year":1950,"topics":{"ideas":7},"stated":0},{"year":1951,"topics":{"ideas":5},"stated":0},{"year":1952,"topics":{"ideas":2},"stated":0},{"year":1955,"topics":{"ideas":31},"stated":0},{"year":1956,"topics":{"ideas":1},"stated":0},{"year":1957,"topics":{"ideas":1},"stated":0},{"year":1958,"topics":{"ideas":3},"stated":0},{"year":1959,"topics":{"ideas":6},"stated":0},{"year":1960,"topics":{"ideas":11},"stated":0},{"year":1961,"topics":{"ideas":10},"stated":0},{"year":1965,"topics":{"ideas":8},"stated":0},{"year":1966,"topics":{"ideas":20},"stated":0},{"year":1968,"topics":{"ideas":5},"stated":0},{"year":1969,"topics":{"ideas":2},"stated":0},{"year":1971,"topics":{"ideas":4},"stated":0},{"year":1972,"topics":{"ideas":1},"stated":0},{"year":1973,"topics":{"ideas":2},"stated":0},{"year":1974,"topics":{"ideas":2},"stated":0},{"year":1975,"topics":{"ideas":1},"stated":0},{"year":1976,"topics":{"ideas":1},"stated":0},{"year":1977,"topics":{"people":2,"ideas":8},"stated":0},{"year":1979,"topics":{"ideas":4},"stated":0},{"year":1980,"topics":{"ideas":2},"stated":0},{"year":1981,"topics":{"people":2},"stated":0},{"year":1982,"topics":{"ideas":4},"stated":0},{"year":1983,"topics":{"ideas":4},"stated":0},{"year":1984,"topics":{"ideas":21},"stated":0},{"year":1985,"topics":{"ideas":9},"stated":0},{"year":1986,"topics":{"ideas":18},"stated":0},{"year":1987,"topics":{"ideas":1},"stated":0},{"year":1988,"topics":{"ideas":6},"stated":0},{"year":1989,"topics":{"ideas":8},"stated":0},{"year":1990,"topics":{"ideas":5},"stated":0},{"year":1991,"topics":{"ideas":11},"stated":0},{"year":1992,"topics":{"ideas":9},"stated":0},{"year":1993,"topics":{"ideas":14,"people":3},"stated":0},{"year":1994,"topics":{"ideas":14},"stated":0},{"year":1995,"topics":{"ideas":10,"people":1},"stated":0},{"year":1996,"topics":{"ideas":71},"stated":0},{"year":1997,"topics":{"ideas":12},"stated":0},{"year":1998,"topics":{"ideas":49},"stated":0},{"year":1999,"topics":{"ideas":27,"people":1,"institutions":1},"stated":0},{"year":2000,"topics":{"ideas":62},"stated":0},{"year":2001,"topics":{"ideas":39},"stated":0},{"year":2002,"topics":{"ideas":30},"stated":0},{"year":2003,"topics":{"ideas":8},"stated":0},{"year":2004,"topics":{"ideas":12,"people":3},"stated":0},{"year":2005,"topics":{"ideas":10,"people":1},"stated":0},{"year":2006,"topics":{"ideas":26,"people":1},"stated":0},{"year":2007,"topics":{"ideas":20},"stated":0},{"year":2008,"topics":{"ideas":57,"people":3,"institutions":1},"stated":0},{"year":2009,"topics":{"ideas":10,"people":1},"stated":0},{"year":2010,"topics":{"people":3,"ideas":11,"institutions":1,"governance":4},"stated":0},{"year":2011,"topics":{"people":3,"ideas":7,"institutions":2},"stated":0},{"year":2012,"topics":{"people":4,"ideas":17,"institutions":2},"stated":0},{"year":2013,"topics":{"ideas":46,"people":4,"institutions":2,"capital":1},"stated":0},{"year":2014,"topics":{"institutions":2,"ideas":54,"people":5},"stated":0},{"year":2015,"topics":{"people":22,"ideas":122,"institutions":3,"capital":1},"stated":0},{"year":2016,"topics":{"ideas":88,"people":2},"stated":0},{"year":2017,"topics":{"people":16,"institutions":2,"ideas":128,"governance":2},"stated":0},{"year":2018,"topics":{"people":8,"ideas":158,"institutions":8,"governance":1},"stated":0},{"year":2019,"topics":{"capital":1,"compute":1,"people":10,"governance":19,"ideas":175,"institutions":4},"stated":0},{"year":2020,"topics":{"people":6,"ideas":98,"institutions":1,"governance":2},"stated":0},{"year":2021,"topics":{"people":27,"ideas":177,"compute":1,"institutions":23,"governance":4,"capital":6},"stated":0},{"year":2022,"topics":{"people":15,"ideas":217,"institutions":6,"governance":3,"compute":2,"capital":1},"stated":0},{"year":2023,"topics":{"capital":5,"compute":7,"institutions":23,"people":51,"governance":179,"ideas":389},"stated":0},{"year":2024,"topics":{"capital":12,"compute":19,"governance":282,"people":80,"ideas":178,"institutions":28},"stated":1},{"year":2025,"topics":{"governance":547,"compute":69,"capital":34,"people":90,"institutions":18,"ideas":178},"stated":7},{"year":2026,"topics":{"governance":442,"capital":8,"compute":32,"institutions":21,"ideas":91,"people":107},"stated":19},{"year":2027,"topics":{},"stated":9},{"year":2028,"topics":{},"stated":6},{"year":2029,"topics":{},"stated":3},{"year":2030,"topics":{},"stated":3},{"year":2032,"topics":{},"stated":2},{"year":2035,"topics":{},"stated":2},{"year":2039,"topics":{},"stated":1},{"year":2042,"topics":{},"stated":1},{"year":2050,"topics":{},"stated":1}],"scheduled":[],"statedBy":[]},"collections":[{"target":{"kind":"propositions","id":"superintelligence"},"title":"Propositions about superintelligence, 1863–2022","claimIds":["rd-b1-carlsmith-2023-scheming-ais"],"records":[]},{"target":{"kind":"propositions","id":"risk-debate"},"title":"Arguments about catastrophic risk from advanced AI","claimIds":["rd-r03-carlsmith-2023-scheming-ais","rd-b1-carlsmith-2023-scheming-ais"],"records":[]},{"target":{"kind":"propositions","id":"yudkowsky-arguments"},"title":"Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"economics-of-machine-intelligence"},"title":"Economics of machine intelligence","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"orbital-compute-arguments"},"title":"Arguments about compute in orbit","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"ai-bio-arguments"},"title":"Arguments about AI and biological risk","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"ai-in-science"},"title":"AI in science","claimIds":[],"records":[]},{"target":{"kind":"documents"},"title":"Frontier safety frameworks","claimIds":[]},{"target":{"kind":"compare","id":"frontier-safety-frameworks"},"title":"Compare frameworks","claimIds":[]},{"target":{"kind":"control","id":"openai"},"title":"OpenAI’s control structure, 2015–2026","claimIds":[]},{"target":{"kind":"control","id":"anthropic"},"title":"Anthropic’s control structure, 2021–2026","claimIds":[]},{"target":{"kind":"commitments"},"title":"External commitments and obligations","claimIds":[]},{"target":{"kind":"question","id":"frameworks-to-obligations"},"title":"Frameworks, commitments and legal obligations, 2023–2026","claimIds":[]},{"target":{"kind":"reading"},"title":"Founding and early leadership","claimIds":[]}]},"version":"131evrz"}