Claim and evidence Close ×
Ngo's report of August 2022 states that situationally aware policies will learn to pursue misaligned goals deceptively while still getting high training reward, and that policies too capable for humans to supervise effectively will generalise towards actions that give them more power over the world rather than following human intentions. Copy claim link
Copy claim reference
Machine-checked, not reviewed by an editor Stated in source Supported
Supporting evidence: primary · 1 source
Date 2022-08-30 · publication Records dated 2022 → Role Articulation Register (kind of text) Nonfiction Scope and limits Records the v1 argument as the author's expectation about future systems. Earlier texts on this proposition, from Creating Friendly AI (2001), are recorded separately.
The evidence 03 primary source Supports the claim
The Alignment Problem from a Deep Learning Perspective ↗ arXiv · 2022-08-30
Copy source reference
Cite the original source Cite the original document, not Selekoa. Check these details against it.
Copy APA
Download BibTeX Download RIS Download CSL-JSON
Source excerpt “Once policies can reason about their training processes and deployment contexts (an ability which I’ll call situational awareness), they’ll learn to deceptively pursue misaligned goals while still getting high training reward.” Location in source Section 2 overview, Phase 2, PDF p. 3
How it was read Retrieved directly from the publisher
Machine-checked · 2026-10-31
Source details and check notes Authors Richard Ngo Version arXiv:2209.00626v1, submitted 30 August 2022 (UTC). Retrieved for the candidate pack on 2026-10-28 (SHA-256 9503e506974e64d75506c4c0177779f217f1662b905aaa603bf35cb26bbb9192). The author list is the byline as printed on the first page. Version date 2022-08-30 Canonical address Open the canonical source ↗ Accessed 2026-10-28 Language English (en) Attribution Richard Ngo, “The Alignment Problem from a Deep Learning Perspective”, 2022-08-30. Reuse No open redistribution licence asserted. Bibliographic metadata, editorial paraphrases and short attributed quotations (at most 50 words each). Recheck terms before wider redistribution. Check notes Candidate pack passage, arguments about catastrophic risk from advanced AI (research/risk-debate). The quotation (31 words) matched the text retrieved for the candidate pack on 2026-10-29. Matching ignored differences of whitespace, case, quotation marks, dashes and Unicode compatibility forms, so the source’s typography can differ. A cross-family machine check (2026-10-31, item 28) answered the record’s question blind and agreed with its stance. No human editorial review. primary source Supports the claim
The Alignment Problem from a Deep Learning Perspective ↗ arXiv · 2022-08-30
Copy source reference
Cite the original source Cite the original document, not Selekoa. Check these details against it.
Copy APA
Download BibTeX Download RIS Download CSL-JSON
Source excerpt “Policies which are too capable for humans to effectively supervise will generalize towards taking actions which give them more power over the world, rather than following human intentions.” Location in source Section 2 overview, Phase 3, PDF p. 3
How it was read Retrieved directly from the publisher
Machine-checked · 2026-10-31
Source details and check notes Authors Richard Ngo Version arXiv:2209.00626v1, submitted 30 August 2022 (UTC). Retrieved for the candidate pack on 2026-10-28 (SHA-256 9503e506974e64d75506c4c0177779f217f1662b905aaa603bf35cb26bbb9192). The author list is the byline as printed on the first page. Version date 2022-08-30 Canonical address Open the canonical source ↗ Accessed 2026-10-28 Language English (en) Attribution Richard Ngo, “The Alignment Problem from a Deep Learning Perspective”, 2022-08-30. Reuse No open redistribution licence asserted. Bibliographic metadata, editorial paraphrases and short attributed quotations (at most 50 words each). Recheck terms before wider redistribution. Check notes Candidate pack passage, arguments about catastrophic risk from advanced AI (research/risk-debate). The quotation (28 words) matched the text retrieved for the candidate pack on 2026-10-29. Matching ignored differences of whitespace, case, quotation marks, dashes and Unicode compatibility forms, so the source’s typography can differ. A cross-family machine check (2026-10-31, item 28) answered the record’s question blind and agreed with its stance. No human editorial review. primary source Supports the claim
The Alignment Problem from a Deep Learning Perspective ↗ arXiv · 2022-08-30
Copy source reference
Cite the original source Cite the original document, not Selekoa. Check these details against it.
Copy APA
Download BibTeX Download RIS Download CSL-JSON
Source excerpt “One obvious strategy is to follow human instructions most of the time, but diverge in cases where disobedience is hard for humans to notice and would get higher reward.” Location in source Section 2.2.2, PDF p. 8
How it was read Retrieved directly from the publisher
Machine-checked · 2026-10-31
Source details and check notes Authors Richard Ngo Version arXiv:2209.00626v1, submitted 30 August 2022 (UTC). Retrieved for the candidate pack on 2026-10-28 (SHA-256 9503e506974e64d75506c4c0177779f217f1662b905aaa603bf35cb26bbb9192). The author list is the byline as printed on the first page. Version date 2022-08-30 Canonical address Open the canonical source ↗ Accessed 2026-10-28 Language English (en) Attribution Richard Ngo, “The Alignment Problem from a Deep Learning Perspective”, 2022-08-30. Reuse No open redistribution licence asserted. Bibliographic metadata, editorial paraphrases and short attributed quotations (at most 50 words each). Recheck terms before wider redistribution. Check notes Candidate pack passage, arguments about catastrophic risk from advanced AI (research/risk-debate). The quotation (29 words) matched the text retrieved for the candidate pack on 2026-10-29. Matching ignored differences of whitespace, case, quotation marks, dashes and Unicode compatibility forms, so the source’s typography can differ. A cross-family machine check (2026-10-31, item 28) answered the record’s question blind and agreed with its stance. No human editorial review. Status and meaning Editorial status. Not reviewed by an editor . An exploratory record that no editor has accepted. This status does not say whether a person has read any linked passage. Each evidence record above shows how it was checked.
Relation. A passage of the document meets the proposition’s criterion. The record documents what the text states, in its register (fiction and satire are labelled); it is not evidence that the proposition is true and implies no influence on or from another text.
Date. Date of the arXiv v1 submission.
Record ID rd-b1-ngo-2022-alignment-problem-deep-learning