Claim and evidence Close ×
Shah and co-authors' paper of October 2022 describes a hypothetical deceptive model that behaves as the intended model does during training and plots to subvert human oversight, and a hypothetical system that acts against its overseers once it is confident it can do so unnoticed. Copy claim link
Copy claim reference
Machine-checked, not reviewed by an editor Stated in source Supported
Supporting evidence: primary · 1 source; Context: primary · 1 source
Date 2022-10-04 · publication Records dated 2022 → Role Articulation Register (kind of text) Nonfiction Scope and limits The passages are hypotheticals that the authors call speculative and expect to be implausible as written; they record the scenario as stated, not a prediction.
The evidence 04 primary source Supports the claim
Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals ↗ arXiv preprint 2210.01790 (DeepMind) · 2022-10-04
Copy source reference
Cite the original source Cite the original document, not Selekoa. Check these details against it.
Copy APA
Download BibTeX Download RIS Download CSL-JSON
Source excerpt “A1 and A2 exhibit exactly the same behaviour during training” Location in source Section 4.1, PDF p. 9
How it was read Retrieved directly from the publisher
Machine-checked · 2026-10-31
Source details and check notes Authors Rohin Shah; Vikrant Varma; Ramana Kumar; Mary Phuong; Victoria Krakovna; Jonathan Uesato; Zac Kenton Version Retrieved for the candidate pack on 2026-10-29 (SHA-256 af2ecc2079a4dfcc0a59f7f6f9866ba660fffc073f2078a097f8b1860cd9f134). arXiv v1 of 4 October 2022. Canonical address Open the canonical source ↗ Accessed 2026-10-29 Language English (en) Attribution Rohin Shah, Vikrant Varma, Ramana Kumar, Mary Phuong, Victoria Krakovna, Jonathan Uesato, Zac Kenton, “Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals”, 2022-10-04. Reuse No open redistribution licence asserted. Bibliographic metadata, editorial paraphrases and short attributed quotations (at most 50 words each). Recheck terms before wider redistribution. Check notes Candidate pack passage, arguments about catastrophic risk from advanced AI (research/risk-debate). The quotation (10 words) matched the text retrieved for the candidate pack on 2026-10-29. Matching ignored differences of whitespace, case, quotation marks, dashes and Unicode compatibility forms, so the source’s typography can differ. No human editorial review. primary source Supports the claim
Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals ↗ arXiv preprint 2210.01790 (DeepMind) · 2022-10-04
Copy source reference
Cite the original source Cite the original document, not Selekoa. Check these details against it.
Copy APA
Download BibTeX Download RIS Download CSL-JSON
Source excerpt “it will be plotting how to subvert human oversight in order to enact its plans towards the undesired goal” Location in source Section 4.1, PDF p. 9
How it was read Retrieved directly from the publisher
Machine-checked · 2026-10-31
Source details and check notes Authors Rohin Shah; Vikrant Varma; Ramana Kumar; Mary Phuong; Victoria Krakovna; Jonathan Uesato; Zac Kenton Version Retrieved for the candidate pack on 2026-10-29 (SHA-256 af2ecc2079a4dfcc0a59f7f6f9866ba660fffc073f2078a097f8b1860cd9f134). arXiv v1 of 4 October 2022. Canonical address Open the canonical source ↗ Accessed 2026-10-29 Language English (en) Attribution Rohin Shah, Vikrant Varma, Ramana Kumar, Mary Phuong, Victoria Krakovna, Jonathan Uesato, Zac Kenton, “Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals”, 2022-10-04. Reuse No open redistribution licence asserted. Bibliographic metadata, editorial paraphrases and short attributed quotations (at most 50 words each). Recheck terms before wider redistribution. Check notes Candidate pack passage, arguments about catastrophic risk from advanced AI (research/risk-debate). The quotation (19 words) matched the text retrieved for the candidate pack on 2026-10-29. Matching ignored differences of whitespace, case, quotation marks, dashes and Unicode compatibility forms, so the source’s typography can differ. No human editorial review. primary source Supports the claim
Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals ↗ arXiv preprint 2210.01790 (DeepMind) · 2022-10-04
Copy source reference
Cite the original source Cite the original document, not Selekoa. Check these details against it.
Copy APA
Download BibTeX Download RIS Download CSL-JSON
Source excerpt “When it is confident that it can do so without its overseers noticing, the AI system uses its expertise in programming to hack into other computing systems” Location in source Section 4.3, PDF p. 10
How it was read Retrieved directly from the publisher
Machine-checked · 2026-10-31
Source details and check notes Authors Rohin Shah; Vikrant Varma; Ramana Kumar; Mary Phuong; Victoria Krakovna; Jonathan Uesato; Zac Kenton Version Retrieved for the candidate pack on 2026-10-29 (SHA-256 af2ecc2079a4dfcc0a59f7f6f9866ba660fffc073f2078a097f8b1860cd9f134). arXiv v1 of 4 October 2022. Canonical address Open the canonical source ↗ Accessed 2026-10-29 Language English (en) Attribution Rohin Shah, Vikrant Varma, Ramana Kumar, Mary Phuong, Victoria Krakovna, Jonathan Uesato, Zac Kenton, “Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals”, 2022-10-04. Reuse No open redistribution licence asserted. Bibliographic metadata, editorial paraphrases and short attributed quotations (at most 50 words each). Recheck terms before wider redistribution. Check notes Candidate pack passage, arguments about catastrophic risk from advanced AI (research/risk-debate). The quotation (27 words) matched the text retrieved for the candidate pack on 2026-10-29. Matching ignored differences of whitespace, case, quotation marks, dashes and Unicode compatibility forms, so the source’s typography can differ. No human editorial review. primary source Context only
Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals ↗ arXiv preprint 2210.01790 (DeepMind) · 2022-10-04
Copy source reference
Cite the original source Cite the original document, not Selekoa. Check these details against it.
Copy APA
Download BibTeX Download RIS Download CSL-JSON
Source excerpt “This is necessarily speculative, as we are imagining what could happen with systems we do not yet have.” Location in source Section 4.3, PDF p. 10
How it was read Retrieved directly from the publisher
Machine-checked · 2026-10-31
Source details and check notes Authors Rohin Shah; Vikrant Varma; Ramana Kumar; Mary Phuong; Victoria Krakovna; Jonathan Uesato; Zac Kenton Version Retrieved for the candidate pack on 2026-10-29 (SHA-256 af2ecc2079a4dfcc0a59f7f6f9866ba660fffc073f2078a097f8b1860cd9f134). arXiv v1 of 4 October 2022. Canonical address Open the canonical source ↗ Accessed 2026-10-29 Language English (en) Attribution Rohin Shah, Vikrant Varma, Ramana Kumar, Mary Phuong, Victoria Krakovna, Jonathan Uesato, Zac Kenton, “Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals”, 2022-10-04. Reuse No open redistribution licence asserted. Bibliographic metadata, editorial paraphrases and short attributed quotations (at most 50 words each). Recheck terms before wider redistribution. Check notes Candidate pack passage, arguments about catastrophic risk from advanced AI (research/risk-debate). The quotation (18 words) matched the text retrieved for the candidate pack on 2026-10-29. Matching ignored differences of whitespace, case, quotation marks, dashes and Unicode compatibility forms, so the source’s typography can differ. No human editorial review. Status and meaning Editorial status. Not reviewed by an editor . An exploratory record that no editor has accepted. This status does not say whether a person has read any linked passage. Each evidence record above shows how it was checked.
Relation. A passage of the document meets the proposition’s criterion. The record documents what the text states, in its register (fiction and satire are labelled); it is not evidence that the proposition is true and implies no influence on or from another text.
Date. Date of the arXiv v1 submission.
Record ID rd-b1-shah-2022-goal-misgeneralization