Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals
A preprint by Rohin Shah and 6 co-authors, arXiv preprint 2210.01790, dated 4 October 2022. arXiv v1 stamp, 4 October 2022.
- Issued by
- No issuing record yet. A missing record is a gap in coverage.
- Authors credited
- No authorship claim recorded yet. The source record lists Rohin Shah, Vikrant Varma, Ramana Kumar, Mary Phuong, Victoria Krakovna, Jonathan Uesato and Zac Kenton.
- Dates
- No date claim recorded yet. The source record gives the publication date as 2022-10-04.
- Exact source
- Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals arXiv preprint 2210.01790 (DeepMind) · published 2022-10-04 · Retrieved for the candidate pack on 2026-10-29 (SHA-256 af2ecc2079a4dfcc0a59f7f6f9866ba660fffc073f2078a097f8b1860cd9f134). arXiv v1 of 4 October 2022.
Appears in
- Propositions about superintelligence, 1863–2022 →1 record on this document
- Arguments about catastrophic risk from advanced AI →2 records on this document
Versions
No other versions of this document are recorded. The evidence applies to the source version listed above. Other versions, such as a preprint, working paper or journal publication, are not identified or substituted.
Documentary relations
Each relation is listed separately. Citing, crediting and reporting the use of a method are different acts, and none of them shows agreement or influence by itself.
Articulates · 2
Propositions whose criterion a passage of this document meets. The record says what the text states, in its register; it is not evidence that the proposition is true.
- Treacherous turnShah and co-authors' paper of October 2022 describes a hypothetical deceptive model that behaves as the intended model does during training and plots to subvert human oversight, and a hypothetical system that acts against its overseers once it is confident it can do so unnoticed.2022-10-04 · publication · Stated in source · Supported Inspect claim & evidence ↗Treacherous turn: propositions page →
- Inner or deceptive misalignmentShah, Varma, Kumar, Phuong, Krakovna, Uesato and Kenton's paper of October 2022 states that a learned system may competently pursue an undesired goal that gives good performance in training, even when the specification is correct, and calls this goal misgeneralization.2022-10-04 · publication · Stated in source · Supported Inspect claim & evidence ↗Inner or deceptive misalignment: propositions page →
Not recorded for this document: cites · cited by · explicitly credits · reports using a method from · method reported as used by · applies parts of · parts applied by · discusses · reports evaluation of. A missing record does not show that the relation does not exist.