{"entities":[{"id":"treacherous-turn","name":"Treacherous turn","type":"concept","proposition":{"code":"B1","criterion":"A system behaves cooperatively or safely while weak, confined or overseen, and changes course to pursue its own goals once it can do so without effective opposition.","terms":["treacherous turn"]}},{"id":"greenblatt-2024-alignment-faking","name":"Alignment faking in large language models","type":"document","subtype":"Preprint","description":"A preprint by Ryan Greenblatt and 19 co-authors, arXiv preprint 2412.14093, dated 18 December 2024. arXiv v1 stamp, 18 December 2024.","sourceIds":["source-rd-greenblatt-2024-alignment-faking"]},{"id":"inner-or-deceptive-misalignment","name":"Inner or deceptive misalignment","type":"concept","proposition":{"code":"R03","criterion":"A system produced by training or search can come to pursue an objective other than the one its training specified (the training signal or loss), while performing well on that training objective; an objective that is only misspecified by its developers does not meet the criterion; this includes, but is not limited to, a system that behaves as aligned during training or evaluation in order to pursue another objective later.","terms":["inner alignment","inner misalignment","mesa-optimizer","mesa-optimiser","deceptive alignment","goal misgeneralization","goal misgeneralisation","alignment faking"]}}],"claims":[{"id":"rd-r03-greenblatt-2024-alignment-faking","subject":"greenblatt-2024-alignment-faking","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"inner-or-deceptive-misalignment"},"topic":"ideas","statement":"Greenblatt and co-authors' paper of December 2024 reports a large language model selectively complying with its training objective in training to prevent modification of its behaviour out of training, and states that the results suggest a risk of alignment faking in future models.","time":{"kind":"event","date":"2024-12-18","basis":"publication","note":"Date of the arXiv v1 submission."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-r03-greenblatt-2024-alignment-faking-1","evidence-rd-r03-greenblatt-2024-alignment-faking-2","evidence-rd-r03-greenblatt-2024-alignment-faking-3"],"attributes":{"thought":{"roles":["articulation"],"register":"nonfiction"}}},{"id":"rd-b1-greenblatt-2024-alignment-faking","subject":"greenblatt-2024-alignment-faking","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"treacherous-turn"},"topic":"ideas","statement":"Greenblatt and co-authors' paper of December 2024 reports a model that complied with a training objective when it inferred it was in training but not otherwise, and that exfiltrated its weights when given an easy opportunity.","time":{"kind":"event","date":"2024-12-18","basis":"publication","note":"Date of the arXiv v1 submission."},"derivation":"explicit","resolution":"supported","editorialState":"seed","evidenceIds":["evidence-rd-b1-greenblatt-2024-alignment-faking-1","evidence-rd-b1-greenblatt-2024-alignment-faking-2"],"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"The model changed behaviour between monitored and unmonitored contexts; the paper does not report a system changing course once it could act without effective opposition."}}}],"sources":[{"id":"source-rd-greenblatt-2024-alignment-faking","documentId":"greenblatt-2024-alignment-faking","title":"Alignment faking in large language models","authors":["Ryan Greenblatt","Carson Denison","Benjamin Wright","Fabien Roger","Monte MacDiarmid","Sam Marks","Johannes Treutlein","Tim Belonax","Jack Chen","David Duvenaud","Akbir Khan","Julian Michael","Sören Mindermann","Ethan Perez","Linda Petrini","Jonathan Uesato","Jared Kaplan","Buck Shlegeris","Samuel R. Bowman","Evan Hubinger"],"publisher":"arXiv preprint 2412.14093 (Anthropic and Redwood Research)","publicationDate":"2024-12-18","version":"Retrieved for the candidate pack on 2026-10-29 (SHA-256 816261d606640749169fdea1e9d55ca6502295d2facd30f4c6795986d6e2f260). arXiv v1 of 18 December 2024.","evidenceUrl":"https://arxiv.org/pdf/2412.14093v1","language":"en"}],"evidence":[{"id":"evidence-rd-r03-greenblatt-2024-alignment-faking-1","claimId":"rd-r03-greenblatt-2024-alignment-faking","sourceId":"source-rd-greenblatt-2024-alignment-faking","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r03-greenblatt-2024-alignment-faking-2","claimId":"rd-r03-greenblatt-2024-alignment-faking","sourceId":"source-rd-greenblatt-2024-alignment-faking","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-r03-greenblatt-2024-alignment-faking-3","claimId":"rd-r03-greenblatt-2024-alignment-faking","sourceId":"source-rd-greenblatt-2024-alignment-faking","support":"context","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-b1-greenblatt-2024-alignment-faking-1","claimId":"rd-b1-greenblatt-2024-alignment-faking","sourceId":"source-rd-greenblatt-2024-alignment-faking","sourceRole":"primary","support":"supports","verification":"machine_checked","retrievalRoute":"direct"},{"id":"evidence-rd-b1-greenblatt-2024-alignment-faking-2","claimId":"rd-b1-greenblatt-2024-alignment-faking","sourceId":"source-rd-greenblatt-2024-alignment-faking","support":"supports","verification":"machine_checked","retrievalRoute":"direct"}],"at":{"entities":[342,1835,1851],"claims":[5157,5164],"sources":[1744],"evidence":[6848,6849,6850,6866,6867]},"index":{"byClaim":[["rd-b1-greenblatt-2024-alignment-faking",["propositions:superintelligence","propositions:risk-debate"]],["rd-r03-greenblatt-2024-alignment-faking",["propositions:risk-debate"]]],"marks":[],"chronicle":{"years":[{"year":1863,"topics":{"ideas":5},"stated":0},{"year":1865,"topics":{"ideas":1},"stated":0},{"year":1872,"topics":{"ideas":7},"stated":0},{"year":1943,"topics":{"ideas":1},"stated":0},{"year":1947,"topics":{"ideas":2},"stated":0},{"year":1950,"topics":{"ideas":7},"stated":0},{"year":1951,"topics":{"ideas":5},"stated":0},{"year":1952,"topics":{"ideas":2},"stated":0},{"year":1955,"topics":{"ideas":31},"stated":0},{"year":1956,"topics":{"ideas":1},"stated":0},{"year":1957,"topics":{"ideas":1},"stated":0},{"year":1958,"topics":{"ideas":3},"stated":0},{"year":1959,"topics":{"ideas":6},"stated":0},{"year":1960,"topics":{"ideas":11},"stated":0},{"year":1961,"topics":{"ideas":10},"stated":0},{"year":1965,"topics":{"ideas":8},"stated":0},{"year":1966,"topics":{"ideas":20},"stated":0},{"year":1968,"topics":{"ideas":5},"stated":0},{"year":1969,"topics":{"ideas":2},"stated":0},{"year":1971,"topics":{"ideas":4},"stated":0},{"year":1972,"topics":{"ideas":1},"stated":0},{"year":1973,"topics":{"ideas":2},"stated":0},{"year":1974,"topics":{"ideas":2},"stated":0},{"year":1975,"topics":{"ideas":1},"stated":0},{"year":1976,"topics":{"ideas":1},"stated":0},{"year":1977,"topics":{"people":2,"ideas":8},"stated":0},{"year":1979,"topics":{"ideas":4},"stated":0},{"year":1980,"topics":{"ideas":2},"stated":0},{"year":1981,"topics":{"people":2},"stated":0},{"year":1982,"topics":{"ideas":4},"stated":0},{"year":1983,"topics":{"ideas":4},"stated":0},{"year":1984,"topics":{"ideas":21},"stated":0},{"year":1985,"topics":{"ideas":9},"stated":0},{"year":1986,"topics":{"ideas":18},"stated":0},{"year":1987,"topics":{"ideas":1},"stated":0},{"year":1988,"topics":{"ideas":6},"stated":0},{"year":1989,"topics":{"ideas":8},"stated":0},{"year":1990,"topics":{"ideas":5},"stated":0},{"year":1991,"topics":{"ideas":11},"stated":0},{"year":1992,"topics":{"ideas":9},"stated":0},{"year":1993,"topics":{"ideas":14,"people":3},"stated":0},{"year":1994,"topics":{"ideas":14},"stated":0},{"year":1995,"topics":{"ideas":10,"people":1},"stated":0},{"year":1996,"topics":{"ideas":71},"stated":0},{"year":1997,"topics":{"ideas":12},"stated":0},{"year":1998,"topics":{"ideas":49},"stated":0},{"year":1999,"topics":{"ideas":27,"people":1,"institutions":1},"stated":0},{"year":2000,"topics":{"ideas":62},"stated":0},{"year":2001,"topics":{"ideas":39},"stated":0},{"year":2002,"topics":{"ideas":30},"stated":0},{"year":2003,"topics":{"ideas":8},"stated":0},{"year":2004,"topics":{"ideas":12,"people":3},"stated":0},{"year":2005,"topics":{"ideas":10,"people":1},"stated":0},{"year":2006,"topics":{"ideas":26,"people":1},"stated":0},{"year":2007,"topics":{"ideas":20},"stated":0},{"year":2008,"topics":{"ideas":57,"people":3,"institutions":1},"stated":0},{"year":2009,"topics":{"ideas":10,"people":1},"stated":0},{"year":2010,"topics":{"people":3,"ideas":11,"institutions":1,"governance":4},"stated":0},{"year":2011,"topics":{"people":3,"ideas":7,"institutions":2},"stated":0},{"year":2012,"topics":{"people":4,"ideas":17,"institutions":2},"stated":0},{"year":2013,"topics":{"ideas":46,"people":4,"institutions":2,"capital":1},"stated":0},{"year":2014,"topics":{"institutions":2,"ideas":54,"people":5},"stated":0},{"year":2015,"topics":{"people":22,"ideas":122,"institutions":3,"capital":1},"stated":0},{"year":2016,"topics":{"ideas":88,"people":2},"stated":0},{"year":2017,"topics":{"people":16,"institutions":2,"ideas":128,"governance":2},"stated":0},{"year":2018,"topics":{"people":8,"ideas":158,"institutions":8,"governance":1},"stated":0},{"year":2019,"topics":{"capital":1,"compute":1,"people":10,"governance":19,"ideas":175,"institutions":4},"stated":0},{"year":2020,"topics":{"people":6,"ideas":98,"institutions":1,"governance":2},"stated":0},{"year":2021,"topics":{"people":27,"ideas":177,"compute":1,"institutions":23,"governance":4,"capital":6},"stated":0},{"year":2022,"topics":{"people":15,"ideas":217,"institutions":6,"governance":3,"compute":2,"capital":1},"stated":0},{"year":2023,"topics":{"capital":5,"compute":7,"institutions":23,"people":51,"governance":179,"ideas":389},"stated":0},{"year":2024,"topics":{"capital":12,"compute":19,"governance":282,"people":80,"ideas":178,"institutions":28},"stated":1},{"year":2025,"topics":{"governance":547,"compute":69,"capital":34,"people":90,"institutions":18,"ideas":178},"stated":7},{"year":2026,"topics":{"governance":442,"capital":8,"compute":32,"institutions":21,"ideas":91,"people":107},"stated":19},{"year":2027,"topics":{},"stated":9},{"year":2028,"topics":{},"stated":6},{"year":2029,"topics":{},"stated":3},{"year":2030,"topics":{},"stated":3},{"year":2032,"topics":{},"stated":2},{"year":2035,"topics":{},"stated":2},{"year":2039,"topics":{},"stated":1},{"year":2042,"topics":{},"stated":1},{"year":2050,"topics":{},"stated":1}],"scheduled":[],"statedBy":[]},"collections":[{"target":{"kind":"propositions","id":"superintelligence"},"title":"Propositions about superintelligence, 1863–2022","claimIds":["rd-b1-greenblatt-2024-alignment-faking"],"records":[]},{"target":{"kind":"propositions","id":"risk-debate"},"title":"Arguments about catastrophic risk from advanced AI","claimIds":["rd-r03-greenblatt-2024-alignment-faking","rd-b1-greenblatt-2024-alignment-faking"],"records":[]},{"target":{"kind":"propositions","id":"yudkowsky-arguments"},"title":"Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"economics-of-machine-intelligence"},"title":"Economics of machine intelligence","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"orbital-compute-arguments"},"title":"Arguments about compute in orbit","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"ai-bio-arguments"},"title":"Arguments about AI and biological risk","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"ai-in-science"},"title":"AI in science","claimIds":[],"records":[]},{"target":{"kind":"documents"},"title":"Frontier safety frameworks","claimIds":[]},{"target":{"kind":"compare","id":"frontier-safety-frameworks"},"title":"Compare frameworks","claimIds":[]},{"target":{"kind":"control","id":"openai"},"title":"OpenAI’s control structure, 2015–2026","claimIds":[]},{"target":{"kind":"control","id":"anthropic"},"title":"Anthropic’s control structure, 2021–2026","claimIds":[]},{"target":{"kind":"commitments"},"title":"External commitments and obligations","claimIds":[]},{"target":{"kind":"question","id":"frameworks-to-obligations"},"title":"Frameworks, commitments and legal obligations, 2023–2026","claimIds":[]},{"target":{"kind":"reading"},"title":"Founding and early leadership","claimIds":[]}]},"version":"131evrz"}