{"entities":[{"id":"treacherous-turn","name":"Treacherous turn","type":"concept","proposition":{"code":"B1","criterion":"A system behaves cooperatively or safely while weak, confined or overseen, and changes course to pursue its own goals once it can do so without effective opposition.","terms":["treacherous turn"]}},{"id":"alignment-is-hard","name":"Difficulty of alignment","type":"concept","proposition":{"code":"R06","criterion":"Making an advanced AI system reliably pursue the goals its developers intend (aligning it) is a hard, unsolved technical problem that will not be solved by default in the ordinary course of development.","terms":["alignment problem","control problem","value alignment"]}},{"id":"belrose-pope-2023-ai-easy-to-control","name":"AI is easy to control","type":"document","subtype":"Essay","description":"An essay by Nora Belrose and Quintin Pope, AI Optimism, dated 28 November 2023. The address carries 2023/11/28; the page metadata gives datePublished 2023-11-29T02:38:05 UTC.","sourceIds":["source-rd-belrose-pope-2023-ai-easy-to-control"]},{"id":"control-by-design","name":"Advanced AI can be kept under control by design","type":"concept","proposition":{"code":"R10b","criterion":"People can build advanced AI systems with objectives, safeguards or oversight that keep them under human control, so that loss of control is avoidable in ordinary engineering practice.","terms":[]}},{"id":"inner-or-deceptive-misalignment","name":"Inner or deceptive misalignment","type":"concept","proposition":{"code":"R03","criterion":"A system produced by training or search can come to pursue an objective other than the one its training specified (the training signal or loss), while performing well on that training objective; an objective that is only misspecified by its developers does not meet the criterion; this includes, but is not limited to, a system that behaves as aligned during training or evaluation in order to pursue another objective later.","terms":["inner alignment","inner misalignment","mesa-optimizer","mesa-optimiser","deceptive alignment","goal misgeneralization","goal misgeneralisation","alignment faking"]}}],"claims":[{"id":"rd-r03-belrose-pope-2023-ai-easy-to-control","subject":"belrose-pope-2023-ai-easy-to-control","predicate":"contests_proposition","object":{"kind":"entity","entityId":"inner-or-deceptive-misalignment"},"topic":"ideas","statement":"Belrose and Pope's essay of November 2023 argues that an AI will learn human values early in training, before it acquires dangerous capabilities, so that AIs will not learn to pretend to be ethical during training while holding other motives.","time":{"kind":"event","date":"2023-11-28","basis":"publication","note":"Date of posting."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-r03-belrose-pope-2023-ai-easy-to-control-1","evidence-rd-r03-belrose-pope-2023-ai-easy-to-control-2"],"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"It disputes the premise that a capable system would acquire other objectives before it learns the intended values."}}}},{"id":"rd-b1-belrose-pope-2023-ai-easy-to-control","subject":"belrose-pope-2023-ai-easy-to-control","predicate":"contests_proposition","object":{"kind":"entity","entityId":"treacherous-turn"},"topic":"ideas","statement":"Belrose and Pope's essay of November 2023 argues that gradient descent sees all of an AI's computation, so that a secret plan to harm its overseers would be noticed and trained away, and that such plans do not improve performance on the tasks AIs are trained for.","time":{"kind":"event","date":"2023-11-28","basis":"publication","note":"Date of posting."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-b1-belrose-pope-2023-ai-easy-to-control-1","evidence-rd-b1-belrose-pope-2023-ai-easy-to-control-2"],"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"It disputes the premise that a system could keep a concealed goal through training, since gradient descent acts on all of its computation."}}}},{"id":"rd-r06-belrose-pope-2023-ai-easy-to-control","subject":"belrose-pope-2023-ai-easy-to-control","predicate":"contests_proposition","object":{"kind":"entity","entityId":"alignment-is-hard"},"topic":"ideas","statement":"Belrose and Pope's essay of November 2023 argues that AI, even superhuman AI, will remain much more controllable than humans for the foreseeable future, and that it will be easy to instill human values into an AI.","time":{"kind":"event","date":"2023-11-28","basis":"publication","note":"Date of posting."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-r06-belrose-pope-2023-ai-easy-to-control-1","evidence-rd-r06-belrose-pope-2023-ai-easy-to-control-2","evidence-rd-r06-belrose-pope-2023-ai-easy-to-control-3"],"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"It disputes the premise that training cannot reliably shape a system's goals, arguing that white-box access and gradient descent give fine-grained control of values."}}}},{"id":"rd-r10b-belrose-pope-2023-ai-easy-to-control","subject":"belrose-pope-2023-ai-easy-to-control","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"control-by-design"},"topic":"ideas","statement":"Belrose and Pope's essay of 28 November 2023 argues that AI, even superhuman AI, will remain much more controllable than humans for the foreseeable future, because neural networks are white boxes open to gradient descent and other control methods, and that it will be easy to instil human values into AI.","time":{"kind":"event","date":"2023-11-28","basis":"publication","note":"Date of posting."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-r10b-belrose-pope-2023-ai-easy-to-control-1","evidence-rd-r10b-belrose-pope-2023-ai-easy-to-control-2"],"attributes":{"thought":{"roles":["most-developed"],"register":"nonfiction"}}}],"sources":[{"id":"source-rd-belrose-pope-2023-ai-easy-to-control","documentId":"belrose-pope-2023-ai-easy-to-control","title":"AI is easy to control","authors":["Nora Belrose","Quintin Pope"],"publisher":"AI Optimism (optimists.ai)","publicationDate":"2023-11-28","version":"Retrieved for the candidate pack on 2026-10-29 (SHA-256 ebe2a9bfc1eaffeefb19ec191f3db1dfa9276ec09cb2bc20257fd46dafd3da5c). current optimists.ai page, address dated 2023/11/28, datePublished 2023-11-29T02:38 UTC in the page metadata; later edits not checked. The cross-family check retrieved it again on 2026-10-31; the bytes differ, and the check compared the quoted passages.","evidenceUrl":"https://optimists.ai/2023/11/28/ai-is-easy-to-control/","language":"en"}],"evidence":[{"id":"evidence-rd-r03-belrose-pope-2023-ai-easy-to-control-1","claimId":"rd-r03-belrose-pope-2023-ai-easy-to-control","sourceId":"source-rd-belrose-pope-2023-ai-easy-to-control","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r03-belrose-pope-2023-ai-easy-to-control-2","claimId":"rd-r03-belrose-pope-2023-ai-easy-to-control","sourceId":"source-rd-belrose-pope-2023-ai-easy-to-control","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-b1-belrose-pope-2023-ai-easy-to-control-1","claimId":"rd-b1-belrose-pope-2023-ai-easy-to-control","sourceId":"source-rd-belrose-pope-2023-ai-easy-to-control","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-b1-belrose-pope-2023-ai-easy-to-control-2","claimId":"rd-b1-belrose-pope-2023-ai-easy-to-control","sourceId":"source-rd-belrose-pope-2023-ai-easy-to-control","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r06-belrose-pope-2023-ai-easy-to-control-1","claimId":"rd-r06-belrose-pope-2023-ai-easy-to-control","sourceId":"source-rd-belrose-pope-2023-ai-easy-to-control","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r06-belrose-pope-2023-ai-easy-to-control-2","claimId":"rd-r06-belrose-pope-2023-ai-easy-to-control","sourceId":"source-rd-belrose-pope-2023-ai-easy-to-control","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r06-belrose-pope-2023-ai-easy-to-control-3","claimId":"rd-r06-belrose-pope-2023-ai-easy-to-control","sourceId":"source-rd-belrose-pope-2023-ai-easy-to-control","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r10b-belrose-pope-2023-ai-easy-to-control-1","claimId":"rd-r10b-belrose-pope-2023-ai-easy-to-control","sourceId":"source-rd-belrose-pope-2023-ai-easy-to-control","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r10b-belrose-pope-2023-ai-easy-to-control-2","claimId":"rd-r10b-belrose-pope-2023-ai-easy-to-control","sourceId":"source-rd-belrose-pope-2023-ai-easy-to-control","support":"supports","verification":"machine_checked","retrievalRoute":"direct"}],"at":{"entities":[342,1776,1786,1812,1851],"claims":[5154,5163,5192,5245],"sources":[1705],"evidence":[6842,6843,6864,6865,6917,6918,6919,7010,7011]},"index":{"byClaim":[["rd-b1-belrose-pope-2023-ai-easy-to-control",["propositions:superintelligence","propositions:risk-debate"]],["rd-r03-belrose-pope-2023-ai-easy-to-control",["propositions:risk-debate"]],["rd-r06-belrose-pope-2023-ai-easy-to-control",["propositions:risk-debate"]],["rd-r10b-belrose-pope-2023-ai-easy-to-control",["propositions:risk-debate"]]],"marks":[["belrose-pope-2023-ai-easy-to-control",[{"collectionId":"risk-debate","propositionId":"inner-or-deceptive-misalignment","claimId":"rd-r03-belrose-pope-2023-ai-easy-to-control","text":"Earliest located text contesting R03 Inner or deceptive misalignment in Arguments about catastrophic risk from advanced AI"}]]],"chronicle":{"years":[{"year":1863,"topics":{"ideas":5},"stated":0},{"year":1865,"topics":{"ideas":1},"stated":0},{"year":1872,"topics":{"ideas":7},"stated":0},{"year":1943,"topics":{"ideas":1},"stated":0},{"year":1947,"topics":{"ideas":2},"stated":0},{"year":1950,"topics":{"ideas":7},"stated":0},{"year":1951,"topics":{"ideas":5},"stated":0},{"year":1952,"topics":{"ideas":2},"stated":0},{"year":1955,"topics":{"ideas":31},"stated":0},{"year":1956,"topics":{"ideas":1},"stated":0},{"year":1957,"topics":{"ideas":1},"stated":0},{"year":1958,"topics":{"ideas":3},"stated":0},{"year":1959,"topics":{"ideas":6},"stated":0},{"year":1960,"topics":{"ideas":11},"stated":0},{"year":1961,"topics":{"ideas":10},"stated":0},{"year":1965,"topics":{"ideas":8},"stated":0},{"year":1966,"topics":{"ideas":20},"stated":0},{"year":1968,"topics":{"ideas":5},"stated":0},{"year":1969,"topics":{"ideas":2},"stated":0},{"year":1971,"topics":{"ideas":4},"stated":0},{"year":1972,"topics":{"ideas":1},"stated":0},{"year":1973,"topics":{"ideas":2},"stated":0},{"year":1974,"topics":{"ideas":2},"stated":0},{"year":1975,"topics":{"ideas":1},"stated":0},{"year":1976,"topics":{"ideas":1},"stated":0},{"year":1977,"topics":{"people":2,"ideas":8},"stated":0},{"year":1979,"topics":{"ideas":4},"stated":0},{"year":1980,"topics":{"ideas":2},"stated":0},{"year":1981,"topics":{"people":2},"stated":0},{"year":1982,"topics":{"ideas":4},"stated":0},{"year":1983,"topics":{"ideas":4},"stated":0},{"year":1984,"topics":{"ideas":21},"stated":0},{"year":1985,"topics":{"ideas":9},"stated":0},{"year":1986,"topics":{"ideas":18},"stated":0},{"year":1987,"topics":{"ideas":1},"stated":0},{"year":1988,"topics":{"ideas":6},"stated":0},{"year":1989,"topics":{"ideas":8},"stated":0},{"year":1990,"topics":{"ideas":5},"stated":0},{"year":1991,"topics":{"ideas":11},"stated":0},{"year":1992,"topics":{"ideas":9},"stated":0},{"year":1993,"topics":{"ideas":14,"people":3},"stated":0},{"year":1994,"topics":{"ideas":14},"stated":0},{"year":1995,"topics":{"ideas":10,"people":1},"stated":0},{"year":1996,"topics":{"ideas":71},"stated":0},{"year":1997,"topics":{"ideas":12},"stated":0},{"year":1998,"topics":{"ideas":49},"stated":0},{"year":1999,"topics":{"ideas":27,"people":1,"institutions":1},"stated":0},{"year":2000,"topics":{"ideas":62},"stated":0},{"year":2001,"topics":{"ideas":39},"stated":0},{"year":2002,"topics":{"ideas":30},"stated":0},{"year":2003,"topics":{"ideas":8},"stated":0},{"year":2004,"topics":{"ideas":12,"people":3},"stated":0},{"year":2005,"topics":{"ideas":10,"people":1},"stated":0},{"year":2006,"topics":{"ideas":26,"people":1},"stated":0},{"year":2007,"topics":{"ideas":20},"stated":0},{"year":2008,"topics":{"ideas":57,"people":3,"institutions":1},"stated":0},{"year":2009,"topics":{"ideas":10,"people":1},"stated":0},{"year":2010,"topics":{"people":3,"ideas":11,"institutions":1,"governance":4},"stated":0},{"year":2011,"topics":{"people":3,"ideas":7,"institutions":2},"stated":0},{"year":2012,"topics":{"people":4,"ideas":17,"institutions":2},"stated":0},{"year":2013,"topics":{"ideas":46,"people":4,"institutions":2,"capital":1},"stated":0},{"year":2014,"topics":{"institutions":2,"ideas":54,"people":5},"stated":0},{"year":2015,"topics":{"people":22,"ideas":122,"institutions":3,"capital":1},"stated":0},{"year":2016,"topics":{"ideas":88,"people":2},"stated":0},{"year":2017,"topics":{"people":16,"institutions":2,"ideas":128,"governance":2},"stated":0},{"year":2018,"topics":{"people":8,"ideas":158,"institutions":8,"governance":1},"stated":0},{"year":2019,"topics":{"capital":1,"compute":1,"people":10,"governance":19,"ideas":175,"institutions":4},"stated":0},{"year":2020,"topics":{"people":6,"ideas":98,"institutions":1,"governance":2},"stated":0},{"year":2021,"topics":{"people":27,"ideas":177,"compute":1,"institutions":23,"governance":4,"capital":6},"stated":0},{"year":2022,"topics":{"people":15,"ideas":217,"institutions":6,"governance":3,"compute":2,"capital":1},"stated":0},{"year":2023,"topics":{"capital":5,"compute":7,"institutions":23,"people":51,"governance":179,"ideas":389},"stated":0},{"year":2024,"topics":{"capital":12,"compute":19,"governance":282,"people":80,"ideas":178,"institutions":28},"stated":1},{"year":2025,"topics":{"governance":547,"compute":69,"capital":34,"people":90,"institutions":18,"ideas":178},"stated":7},{"year":2026,"topics":{"governance":442,"capital":8,"compute":32,"institutions":21,"ideas":91,"people":107},"stated":19},{"year":2027,"topics":{},"stated":9},{"year":2028,"topics":{},"stated":6},{"year":2029,"topics":{},"stated":3},{"year":2030,"topics":{},"stated":3},{"year":2032,"topics":{},"stated":2},{"year":2035,"topics":{},"stated":2},{"year":2039,"topics":{},"stated":1},{"year":2042,"topics":{},"stated":1},{"year":2050,"topics":{},"stated":1}],"scheduled":[],"statedBy":[]},"collections":[{"target":{"kind":"propositions","id":"superintelligence"},"title":"Propositions about superintelligence, 1863–2022","claimIds":["rd-b1-belrose-pope-2023-ai-easy-to-control"],"records":[]},{"target":{"kind":"propositions","id":"risk-debate"},"title":"Arguments about catastrophic risk from advanced AI","claimIds":["rd-r03-belrose-pope-2023-ai-easy-to-control","rd-b1-belrose-pope-2023-ai-easy-to-control","rd-r06-belrose-pope-2023-ai-easy-to-control","rd-r10b-belrose-pope-2023-ai-easy-to-control"],"records":[]},{"target":{"kind":"propositions","id":"yudkowsky-arguments"},"title":"Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"economics-of-machine-intelligence"},"title":"Economics of machine intelligence","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"orbital-compute-arguments"},"title":"Arguments about compute in orbit","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"ai-bio-arguments"},"title":"Arguments about AI and biological risk","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"ai-in-science"},"title":"AI in science","claimIds":[],"records":[]},{"target":{"kind":"documents"},"title":"Frontier safety frameworks","claimIds":[]},{"target":{"kind":"compare","id":"frontier-safety-frameworks"},"title":"Compare frameworks","claimIds":[]},{"target":{"kind":"control","id":"openai"},"title":"OpenAI’s control structure, 2015–2026","claimIds":[]},{"target":{"kind":"control","id":"anthropic"},"title":"Anthropic’s control structure, 2021–2026","claimIds":[]},{"target":{"kind":"commitments"},"title":"External commitments and obligations","claimIds":[]},{"target":{"kind":"question","id":"frameworks-to-obligations"},"title":"Frameworks, commitments and legal obligations, 2023–2026","claimIds":[]},{"target":{"kind":"reading"},"title":"Founding and early leadership","claimIds":[]}]},"version":"131evrz"}