{"entities":[{"id":"iem-2013","name":"Intelligence Explosion Microeconomics","type":"document"},{"id":"staring-into-the-singularity","name":"Staring into the Singularity","type":"document"},{"id":"extropians-1996-12-31-yudkowsky-5142","name":"Re: Goal-based AI","type":"document"},{"id":"extropians-1996-12-31-yudkowsky-5159","name":"Re: Goal-based AI","type":"document"},{"id":"creating-friendly-ai-2001","name":"Creating Friendly AI 1.0: The Analysis and Design of Benevolent Goal Architectures","type":"document"},{"id":"ai-positive-negative-factor-global-risk","name":"Artificial Intelligence as a Positive and Negative Factor in Global Risk","type":"document"},{"id":"yudkowsky-2008-recursive-self-improvement","name":"Recursive Self-Improvement","type":"document"},{"id":"yudkowsky-2008-hard-takeoff","name":"Hard Takeoff","type":"document"},{"id":"arbital-programmer-deception","name":"Programmer deception","type":"document"},{"id":"agi-ruin-2022","name":"AGI Ruin: A List of Lethalities","type":"document"},{"id":"switch-off-if-dangerous","name":"Switching it off if it becomes dangerous","type":"concept","proposition":{"code":"Y01","criterion":"An explicit claim that an advanced AI system that became dangerous could simply be switched off or corrected, on the premise that a capable goal-directed system would not act against its own shutdown or modification, so that a shutdown or correction mechanism can be added to it, or that people would recognise in time when it had become dangerous enough to use one.","terms":[]}},{"id":"intelligence-brings-benevolence","name":"Sufficient intelligence brings good values","type":"concept","proposition":{"code":"Y02","criterion":"An explicit claim that a sufficiently intelligent system will work out ethics and act well, on the premise that greater intelligence by itself brings benevolent goals, or that a sufficiently intelligent mind would discover a true or universally compelling morality and be bound to act on it.","terms":[]}},{"id":"understanding-implies-compliance","name":"A superintelligence will do what we mean","type":"concept","proposition":{"code":"Y03","criterion":"An explicit claim that a superintelligent system will act on what its makers actually mean, on the premise that an AI that understands what people intend will act on that understanding.","terms":[]}},{"id":"short-instruction-suffices","name":"A short instruction such as making everyone happy","type":"concept","proposition":{"code":"Y04","criterion":"An explicit claim that an advanced AI can safely be given a short goal such as making everyone happy, on the premise that such an instruction specifies what people want.","terms":[]}},{"id":"feedback-instils-values","name":"Training on human feedback instils human values","type":"concept","proposition":{"code":"Y05","criterion":"An explicit claim that training an AI system on human feedback will make it learn human values, on the premise that optimising a system for human approval or reward produces internalised human values, or that training instils values as upbringing instils morals in children.","terms":[]}},{"id":"confinement-keeps-it-safe","name":"Confinement without network access keeps it safe","type":"concept","proposition":{"code":"Y06","criterion":"An explicit claim that an advanced AI kept confined, without network access, poses no danger, on the premise that a smarter-than-human mind confined to a channel with its human operators cannot get out through those operators.","terms":[]}},{"id":"ai-solves-alignment","name":"AI systems will solve alignment for us","type":"concept","proposition":{"code":"Y07","criterion":"An explicit claim that AI systems can be relied on to solve the alignment problem, on the premise that an AI weak enough to be safe is strong enough to solve alignment, or that its alignment output can be verified.","terms":[]}},{"id":"warning-signs-in-time","name":"Clear warning signs will come before danger","type":"concept","proposition":{"code":"Y08","criterion":"An explicit claim that clear warning signs will appear before advanced AI becomes dangerous, on the premise that a recognisable, consensus-forming signal will come in time to act, that dangerous failure modes will first appear in weaker systems where they can be observed and fixed, or that a warning sign, once seen, will lead to effective action.","terms":[]}},{"id":"next-word-prediction-is-shallow","name":"Next-word prediction is only autocomplete","type":"concept","proposition":{"code":"Y09","criterion":"An explicit claim that a system trained to predict the next word is only autocomplete, on the premise that next-token prediction is a shallow task that cannot require deep cognition, or that a system trained to predict human text is capped at imitating the people who wrote it.","terms":[]}},{"id":"diminishing-returns-to-intelligence","name":"Diminishing returns rule out a runaway","type":"concept","proposition":{"code":"Y10","criterion":"An explicit claim that intelligence has diminishing returns, so that no runaway rise in machine intelligence will occur, on the premise that returns to reinvesting cognitive gains in further cognitive improvement diminish.","terms":[]}},{"id":"slow-takeoff-time-to-adjust","name":"A slow takeoff will leave time to adjust","type":"concept","proposition":{"code":"Y11","criterion":"An explicit claim that advanced AI will arrive slowly enough to leave time to adjust, on the premise that capability gains will be gradual and visible, or that slightly weaker predecessor systems will give time and experience before the decisive one.","terms":[]}},{"id":"evaluations-catch-danger","name":"Safety evaluations would catch a dangerous system","type":"concept","proposition":{"code":"Y12","criterion":"An explicit claim that safety evaluations would detect a dangerous AI system, on the premise that a dangerous system would behave dangerously while it is being tested, or that tests on systems that cannot yet cause a catastrophe show how more capable systems will behave.","terms":[]}},{"id":"agi-decades-away","name":"AGI is decades away, so the problem can wait","type":"concept","proposition":{"code":"Y13","criterion":"An explicit claim that the risks of AGI can be left until later because AGI is decades away, on the premise that it can be known that AGI is decades away, or that long timelines mean there is no need to act now.","terms":[]}},{"id":"fix-problems-as-they-arise","name":"Problems can be fixed as they arise","type":"concept","proposition":{"code":"Y14","criterion":"An explicit claim that problems with advanced AI can be fixed as they arise, as with other technologies, on the premise that people get to learn from failures with powerful AI and try again.","terms":[]}},{"id":"tool-without-goals","name":"A tool that wants nothing","type":"concept","proposition":{"code":"Y15","criterion":"An explicit claim that an advanced AI system is a tool without wants and is therefore safe, on the premise that a system can be highly capable without having goals or optimising for outcomes, or that a system that only outputs answers or plans, leaving people to act on them, is safe.","terms":[]}},{"id":"give-it-human-values","name":"Giving it good human values","type":"concept","proposition":{"code":"Y16","criterion":"An explicit claim that an advanced AI will be safe if it is given good human values, on the premise that human values can be written down or taught as a ready, compact specification, or that present-day values, or its programmers', are settled enough to fix permanently in an AI.","terms":[]}},{"id":"pause-or-regulation-suffices","name":"A pause or regulation will handle it","type":"concept","proposition":{"code":"Y17","criterion":"An explicit claim that a pause in AI development or regulation will handle the risks of advanced AI, on the premise that a short pause, of six months for example, is enough, that regulation that is national, partial or not universally enforced can stop dangerous development, or that law can prevent the danger.","terms":[]}},{"id":"earth-not-worth-taking","name":"A superintelligence would leave Earth alone","type":"concept","proposition":{"code":"Y18","criterion":"An explicit claim that a superintelligence would not bother to take Earth or its resources because the universe is large, on the premise that it would leave people a small share because sparing Earth would cost it very little, or that people could trade with it for what they need.","terms":[]}},{"id":"harm-requires-hostility","name":"Without hatred it will not harm us","type":"concept","proposition":{"code":"Y19","criterion":"An explicit claim that an AI with no reason to hate people would not harm them, on the premise that an AI would harm people only out of hostility.","terms":[]}},{"id":"risk-is-film-fiction","name":"AI risk as film science fiction","type":"concept","proposition":{"code":"Y20","criterion":"An explicit claim that concern about AI risk is science fiction of the Terminator kind, on the premise that the concern is about robots that come to hate or rebel against people, or that film scenarios are a fair guide to what the risk would look like.","terms":[]}},{"id":"human-data-humanlike-mind","name":"Trained on human data, so humanlike","type":"concept","proposition":{"code":"Y21","criterion":"An explicit claim that an AI trained on human data will think and want as people do, on the premise that training on human text or behaviour yields human-like thought and motives.","terms":[]}},{"id":"build-superintelligence-fast","name":"Build superintelligence as fast as possible","type":"concept","proposition":{"code":"Y22","criterion":"An explicit claim that superintelligence should be reached as fast as possible because its arrival will be good, on the premise that reaching superintelligence is good by default, so that speed is what matters.","terms":[]}},{"id":"build-it-before-rivals","name":"Build it before rivals do","type":"concept","proposition":{"code":"Y24","criterion":"An explicit claim that advanced AI must be built first, before a rival builds it, on the premise that whoever builds it controls it, or that racing to be first improves the outcome.","terms":[]}},{"id":"interpretability-reads-its-mind","name":"Interpretability will let us read its mind","type":"concept","proposition":{"code":"Y25","criterion":"An explicit claim that interpretability will make AI systems safe by showing what they think, on the premise that people can see what an AI system is thinking, or that seeing what it thinks is enough to make it safe.","terms":[]}},{"id":"no-body-no-danger","name":"Without a body it cannot do much","type":"concept","proposition":{"code":"Y26","criterion":"An explicit claim that even a superintelligence could do little without a body, on the premise that intelligence without physical actuators of its own is not dangerous, or that to act in the physical world it would need human help that people would not give.","terms":[]}},{"id":"experiment-and-see","name":"Run experiments and see what happens","type":"concept","proposition":{"code":"Y27","criterion":"An explicit claim that advanced AI can be handled by running experiments and seeing what happens, on the premise that alignment can be learned by trial and error at the frontier, or that results from safe experiments on weaker systems will carry over to dangerous ones.","terms":[]}},{"id":"developers-know-what-they-are-doing","name":"The developers know what they are doing","type":"concept","proposition":{"code":"Y28","criterion":"An explicit claim that the organisations building advanced AI can be relied on to handle its safety, on the premise that a credible safety plan exists, or that the most advanced projects are the most competent to handle safety.","terms":[]}},{"id":"many-ais-balance","name":"Many AIs will keep each other in check","type":"concept","proposition":{"code":"Y29","criterion":"An explicit claim that many AI systems will keep one another in check, on the premise that several superintelligences would balance one another in people's favour.","terms":[]}},{"id":"default-outcome-good","name":"It will probably turn out fine","type":"concept","proposition":{"code":"Y30","criterion":"An explicit claim that the development of advanced AI will probably turn out well, on the premise that the default outcome is good.","terms":[]}},{"id":"ai-box-experiment-page","name":"The AI-Box Experiment","type":"document"},{"id":"arbital-instrumental-convergence","name":"Instrumental convergence","type":"document"},{"id":"arbital-orthogonality","name":"Orthogonality Thesis","type":"document"},{"id":"coherent-extrapolated-volition-2004","name":"Coherent Extrapolated Volition","type":"document"},{"id":"time-2023-shut-it-all-down","name":"Pausing AI Developments Isn't Enough. We Need to Shut it All Down","type":"document"},{"id":"extropians-2000-03-14-yudkowsky-4837","name":"Re: Otter vs. Yudkowsky","type":"document"},{"id":"extropians-2000-03-16-yudkowsky-5105","name":"Re: regarding the Eliezer vrs. Otter debate...","type":"document"},{"id":"extropians-1999-01-06-yudkowsky-0160","name":"Re: Paths to Uploading","type":"document"},{"id":"extropians-1999-02-24-yudkowsky-2293","name":"Re: FAQ Additions (Posthuman mind control)","type":"document"},{"id":"extropians-1999-04-08-yudkowsky-0310","name":"Re: Geniebusters","type":"document"},{"id":"extropians-2000-09-05-yudkowsky-4277","name":"Re: Why would AI want to be friendly? (Was: Congratulations to Eli,Brian ...)","type":"document"},{"id":"extropians-2000-09-05-yudkowsky-4292","name":"Re: Why would AI want to be friendly? (Was: Congratulations to Eli,Brian ...)","type":"document"},{"id":"extropians-2000-09-29-yudkowsky-6073","name":"Eugene's nuclear threat","type":"document"},{"id":"extropians-2000-09-29-yudkowsky-6080","name":"Re: Eugene's nuclear threat","type":"document"},{"id":"extropians-1997-09-25-yudkowsky-4400","name":"Re: Singularity: Generation gap","type":"document"},{"id":"extropians-1997-09-26-yudkowsky-4420","name":"Re: Singularity: Generation gap","type":"document"},{"id":"extropians-1998-09-07-yudkowsky-2171","name":"Singularity, Breaker of Dreams","type":"document"},{"id":"extropians-1998-09-08-yudkowsky-2235","name":"Re: Human AI to superhuman (Re: Max More)","type":"document"},{"id":"extropians-1998-09-15-yudkowsky-2411","name":"Re: AI Prime Directive","type":"document"},{"id":"extropians-1998-09-18-yudkowsky-2487","name":"Re: Singularity: The Moore's Law Argument","type":"document"},{"id":"extropians-1999-08-01-yudkowsky-1386","name":"Re: IA vs. AI was: longevity vs singularity","type":"document"},{"id":"extropians-1999-08-03-yudkowsky-1487","name":"Re: IA vs. AI (vs. humanity)","type":"document"},{"id":"extropians-1999-08-04-yudkowsky-1587","name":"Re: Does a parental model work for having A.I. that can be trusted?","type":"document"},{"id":"extropians-1999-08-04-yudkowsky-1589","name":"Re: Does a parental model work for having A.I. that can be trusted?","type":"document"},{"id":"extropians-1999-08-04-yudkowsky-1591","name":"Re: Does a parental model work for having A.I. that can be trusted?","type":"document"},{"id":"extropians-1999-08-25-yudkowsky-2783","name":"Re: SI Comparative Advantage [was Re: Free Will]","type":"document"},{"id":"extropians-1998-12-06-yudkowsky-2153","name":"Singularity: AI Morality","type":"document"},{"id":"extropians-1999-10-21-yudkowsky-1086","name":"Re: >H: The next 100 months","type":"document"},{"id":"extropians-1996-11-13-yudkowsky-2941","name":"Re: Zones of Thought (was Zindell's _The Wild_)","type":"document"},{"id":"extropians-1996-11-16-yudkowsky-3042","name":"Re: extropians-digest V1 #110","type":"document"},{"id":"extropians-1996-11-22-yudkowsky-3178","name":"Interim purpose","type":"document"},{"id":"extropians-1996-11-22-yudkowsky-3184","name":"Re: Survival ethic (U-Prime)","type":"document"},{"id":"extropians-1996-12-08-yudkowsky-3761","name":"Re: Trans- vs. Non-human","type":"document"},{"id":"lw-2007-the-hidden-complexity-of-wishes","name":"The Hidden Complexity of Wishes","type":"document"},{"id":"lw-2026-only-law-can-prevent-extinction","name":"Only Law Can Prevent Extinction","type":"document"},{"id":"lw-2021-ngo-and-yudkowsky-on-alignment-difficulty","name":"Ngo and Yudkowsky on alignment difficulty","type":"document"},{"id":"lw-2008-above-average-ai-scientists","name":"Above-Average AI Scientists","type":"document"},{"id":"lw-2017-theres-no-fire-alarm-for-artificial","name":"There's No Fire Alarm for Artificial General Intelligence","type":"document"},{"id":"lw-2021-discussion-with-eliezer-yudkowsky-on-agi","name":"Discussion with Eliezer Yudkowsky on AGI interventions","type":"document"},{"id":"lw-2024-the-sun-is-big-but-superintelligences","name":"The Sun is big, but superintelligences will not spare Earth a little sunlight","type":"document"},{"id":"lw-2009-value-is-fragile","name":"Value is Fragile","type":"document"},{"id":"lw-2013-after-critical-event-w-happens-they","name":"After critical event W happens, they still won't believe you","type":"document"},{"id":"lw-2008-fighting-a-rearguard-action-against-the","name":"Fighting a Rearguard Action Against the Truth","type":"document"},{"id":"lw-2008-magical-categories","name":"Magical Categories","type":"document"},{"id":"lw-2008-no-universally-compelling-arguments","name":"No Universally Compelling Arguments","type":"document"},{"id":"lw-2018-challenges-to-christianos-capability-amplification-proposal","name":"Challenges to Christiano’s capability amplification proposal","type":"document"},{"id":"lw-2008-that-tiny-note-of-discord","name":"That Tiny Note of Discord","type":"document"},{"id":"lw-2026-cat-belling-problems","name":"Cat-Belling Problems","type":"document"},{"id":"lw-2021-biology-inspired-agi-timelines-the-trick","name":"Biology-Inspired AGI Timelines: The Trick That Never Works","type":"document"},{"id":"lw-2026-the-talker-does-not-control-the","name":"The Talker Does Not Control The Doer (in Current AIs)","type":"document"},{"id":"lw-2007-thou-art-godshatter","name":"Thou Art Godshatter","type":"document"},{"id":"lw-2008-ghosts-in-the-machine","name":"Ghosts in the Machine","type":"document"},{"id":"lw-2008-the-magnitude-of-his-own-folly","name":"The Magnitude of His Own Folly","type":"document"},{"id":"lw-2026-irretrievability-or-murphys-curse-of-oneshotness","name":"Irretrievability; or, Murphy's Curse of Oneshotness upon ASI","type":"document"},{"id":"lw-2022-miri-announces-new-death-with-dignity","name":"MIRI announces new \"Death With Dignity\" strategy","type":"document"},{"id":"lw-2022-six-dimensions-of-operational-adequacy-in","name":"Six Dimensions of Operational Adequacy in AGI Projects","type":"document"},{"id":"lw-2023-gpts-are-predictors-not-imitators","name":"GPTs are Predictors, not Imitators","type":"document"},{"id":"lw-2025-re-recent-anthropic-safety-research","name":"Re: recent Anthropic safety research","type":"document"},{"id":"lw-2007-the-logical-fallacy-of-generalization-from","name":"The Logical Fallacy of Generalization from Fictional Evidence","type":"document"},{"id":"lw-2023-alexander-and-yudkowsky-on-agi-goals","name":"Alexander and Yudkowsky on AGI goals","type":"document"},{"id":"lw-2012-reply-to-holden-on-tool-ai","name":"Reply to Holden on 'Tool AI'","type":"document"},{"id":"lw-2008-the-rhythm-of-disagreement","name":"The Rhythm of Disagreement","type":"document"},{"id":"lw-2022-shah-and-yudkowsky-on-alignment-failures","name":"Shah and Yudkowsky on alignment failures","type":"document"},{"id":"lw-2021-yudkowsky-and-christiano-discuss-takeoff-speeds","name":"Yudkowsky and Christiano discuss \"Takeoff Speeds\"","type":"document"},{"id":"lw-2008-dreams-of-friendliness","name":"Dreams of Friendliness","type":"document"},{"id":"lw-2008-what-i-think-if-not-why","name":"What I Think, If Not Why","type":"document"},{"id":"sl4-2000-07-09-yudkowsky-0007-0017","name":"Re: [SL4] Programmed morality","type":"document"},{"id":"sl4-2000-07-20-yudkowsky-0007-0023","name":"[SL4] AI flight recorder","type":"document"},{"id":"sl4-2000-08-21-yudkowsky-0008-0046","name":"Re: [SL4] Employment vs. Singularity","type":"document"},{"id":"sl4-2000-09-16-yudkowsky-0009-0086","name":"Re:","type":"document"},{"id":"sl4-2000-09-17-yudkowsky-0009-0092","name":"Re: Get it right the first time (Was: )","type":"document"},{"id":"sl4-2000-10-03-yudkowsky-0010-0125","name":"Re: Ben what are your views and concerns","type":"document"},{"id":"sl4-2000-11-13-yudkowsky-0011-0181","name":"Re: Ben's \"Extropian Creed\"","type":"document"},{"id":"sl4-2000-11-18-yudkowsky-0011-0240","name":"Re: Evolving minds","type":"document"},{"id":"sl4-2000-12-10-yudkowsky-0012-0355","name":"Revising a Friendly AI","type":"document"},{"id":"sl4-2000-12-12-yudkowsky-0012-0381","name":"Re: Revising a Friendly AI","type":"document"},{"id":"sl4-2001-01-24-yudkowsky-0101-0452","name":"Re: Event horizon","type":"document"},{"id":"sl4-2001-02-04-yudkowsky-0102-0539","name":"Re: Beyond evolution","type":"document"},{"id":"sl4-2001-02-05-yudkowsky-0102-0542","name":"Re: Beyond evolution","type":"document"},{"id":"sl4-2001-02-07-yudkowsky-0102-0545","name":"Re: Learning to be evil","type":"document"},{"id":"sl4-2001-02-25-yudkowsky-0102-0594","name":"Re: Military in or out?","type":"document"},{"id":"sl4-2001-02-26-yudkowsky-0102-0603","name":"Re: Military in or out?","type":"document"},{"id":"sl4-2001-04-06-yudkowsky-0104-0956","name":"Re: Singularity intros (was: Si definition of Friendliess)","type":"document"},{"id":"sl4-2001-04-08-yudkowsky-0104-1019","name":"Re: Threats (was Theoretical question for the list: publicity?)","type":"document"},{"id":"sl4-2001-04-12-yudkowsky-0104-1076","name":"Re: Open Source Friendly AI? (was Singularity and the generalpublic)","type":"document"},{"id":"sl4-2001-04-19-yudkowsky-0104-1166","name":"Re: FW: Group releases \"Friendly AI\"","type":"document"},{"id":"sl4-2001-08-01-yudkowsky-0108-2031","name":"Re: Floppy take-off","type":"document"},{"id":"sl4-2002-03-09-yudkowsky-0203-3148","name":"Re: Novamente goal system","type":"document"},{"id":"sl4-2002-03-10-yudkowsky-0203-3160","name":"Webmind Inc. as social process","type":"document"},{"id":"sl4-2002-05-03-yudkowsky-0205-3551","name":"Re: supergoal stability","type":"document"},{"id":"sl4-2002-06-23-yudkowsky-0206-4208","name":"Re: Threats to the Singularity.","type":"document"},{"id":"sl4-2002-06-23-yudkowsky-0206-4219","name":"Re: Seed AI (was: How hard a Singularity?)","type":"document"},{"id":"sl4-2002-06-27-yudkowsky-0206-4453","name":"Re: Military Friendly AI","type":"document"},{"id":"sl4-2002-07-02-yudkowsky-0207-4667","name":"Re: Seed AI (was: How hard a Singularity?)","type":"document"},{"id":"sl4-2002-09-10-yudkowsky-0209-5398","name":"Friendly AI communications issues (was: Consciousness)","type":"document"},{"id":"sl4-2002-09-19-yudkowsky-0209-5571","name":"Re: QUES: CFAI [#2]","type":"document"},{"id":"sl4-2003-05-22-yudkowsky-0305-6783","name":"Re: Flight recorders in AIs","type":"document"},{"id":"sl4-2003-05-29-yudkowsky-0305-6846","name":"Re: SIAI's flawed friendliness analysis","type":"document"},{"id":"sl4-2004-05-30-yudkowsky-0405-8843","name":"Re: One or more AIs??","type":"document"},{"id":"sl4-2004-06-01-yudkowsky-0406-8905","name":"Re: FAI: Collective Volition","type":"document"},{"id":"sl4-2004-12-29-yudkowsky-0412-10488","name":"Re: Conservative Estimation of the Economic Impact of Artificial Intelligence","type":"document"},{"id":"sl4-2005-01-02-yudkowsky-0501-10512","name":"Re: Definition of strong recursive self-improvement","type":"document"},{"id":"sl4-2005-07-13-yudkowsky-0507-11477","name":"Re: Fighting UFAI","type":"document"},{"id":"sl4-2005-07-13-yudkowsky-0507-11483","name":"Re: Fighting UFAI","type":"document"},{"id":"sl4-2005-07-25-yudkowsky-0507-11765","name":"Re: 6 points about Coherent Extrapolated Volition","type":"document"},{"id":"sl4-2005-08-09-yudkowsky-0508-11866","name":"Re: On Our Duty to Not Be Responsible for Artificial Minds","type":"document"},{"id":"sl4-2006-02-15-yudkowsky-0602-14096","name":"Re: Singularity Institute: Likely to win the race to build GAI?","type":"document"},{"id":"sl4-2006-06-06-yudkowsky-0606-15174","name":"Re: [agi] Two draft papers: AI and existential risk; heuristics and biases","type":"document"},{"id":"sl4-2006-06-13-yudkowsky-0606-15271","name":"Re: Two draft papers: AI and existential risk; heuristics and biases","type":"document"},{"id":"sl4-2007-11-24-yudkowsky-0711-17248","name":"Re: UCaRtMaAI paper","type":"document"},{"id":"sl4-2008-02-11-yudkowsky-0802-17621","name":"Re: Investing in FAI research: now vs. later.","type":"document"}],"claims":[{"id":"yk-y01-creating-friendly-ai-2001-b","subject":"creating-friendly-ai-2001","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"switch-off-if-dangerous"},"topic":"ideas","time":{"kind":"event","date":"2001-06-15","basis":"publication","note":"Release date from the reprint; an open-commentary version was announced on SL4 on 2001-04-18, so some wording may be earlier than this date. Reprint wording is not authenticated for 2001 unless a capture was compared."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"A design assumption that an AI cannot be forcibly constrained; it does not argue that shutdown or an added correction mechanism would fail.","attested":{"date":"2001-06-26","note":"The quoted wording also appears in the 2001 HTML edition as captured on 2001-06-26; the quotation itself is taken from MIRI’s later reprint."}}}},{"id":"yk-y01-arbital-instrumental-convergence-c","subject":"arbital-instrumental-convergence","predicate":"contests_proposition","object":{"kind":"entity","entityId":"switch-off-if-dangerous"},"topic":"ideas","time":{"kind":"event","date":"2015-07-16","basis":"publication","note":"Viewer shows \"16 Jul 2015\", probably the creation date; the body says \"As of 2016\" and uses later terminology, so at least part of the cached wording postdates 2015."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","attested":{"note":"Arbital pages could be edited after creation; the viewer shows a page date, and the quoted wording is known only from the page as retrieved in 2026."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a capable goal-directed system will not act to prevent its own shutdown or modification, and a shutdown or correction mechanism can simply be added to it."}}}},{"id":"yk-y01-agi-ruin-2022-c","subject":"agi-ruin-2022","predicate":"contests_proposition","object":{"kind":"entity","entityId":"switch-off-if-dangerous"},"topic":"ideas","time":{"kind":"event","date":"2022-06-05","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a capable goal-directed system will not act to prevent its own shutdown or modification, and a shutdown or correction mechanism can simply be added to it."}}}},{"id":"yk-y01-time-2023-shut-it-all-down-c","subject":"time-2023-shut-it-all-down","predicate":"contests_proposition","object":{"kind":"entity","entityId":"switch-off-if-dangerous"},"topic":"ideas","time":{"kind":"event","date":"2023-03-29","basis":"publication","note":"Page metadata datePublished 2023-03-29T22:01:58Z. The page title currently reads \"The Only Way to Deal With the Threat From AI? Shut It Down\"; the headline is the one crossposted to LessWrong on 2023-04-08."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that we will recognise the moment when it has become dangerous enough to press the switch."}}}},{"id":"yk-y01-sl4-2000-09-17-yudkowsky-0009-0092-c","subject":"sl4-2000-09-17-yudkowsky-0009-0092","predicate":"contests_proposition","object":{"kind":"entity","entityId":"switch-off-if-dangerous"},"topic":"ideas","time":{"kind":"event","date":"2000-09-17","basis":"publication","note":"Archive author index lists the message as \"Sun Sep 17 2000 - 10:19:16 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a capable goal-directed system will not act to prevent its own shutdown or modification, and a shutdown or correction mechanism can simply be added to it."}}}},{"id":"yk-y01-sl4-2002-05-03-yudkowsky-0205-3551-c","subject":"sl4-2002-05-03-yudkowsky-0205-3551","predicate":"contests_proposition","object":{"kind":"entity","entityId":"switch-off-if-dangerous"},"topic":"ideas","time":{"kind":"event","date":"2002-05-03","basis":"publication","note":"Archive author index lists the message as \"Fri May 03 2002 - 16:38:46 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that a capable goal-directed system will not act to prevent its own shutdown or modification, and a shutdown or correction mechanism can simply be added to it, even when read with its message or dialogue turn."}}}},{"id":"yk-y01-sl4-2002-09-19-yudkowsky-0209-5571-b","subject":"sl4-2002-09-19-yudkowsky-0209-5571","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"switch-off-if-dangerous"},"topic":"ideas","time":{"kind":"event","date":"2002-09-19","basis":"publication","note":"Archive author index lists the message as \"Thu Sep 19 2002 - 13:58:03 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"A conditional failure scenario in which detection of a fast ascent fails; it does not predict that dangerous capability will go unrecognised."}}},{"id":"yk-y01-sl4-2006-06-06-yudkowsky-0606-15174-c","subject":"sl4-2006-06-06-yudkowsky-0606-15174","predicate":"contests_proposition","object":{"kind":"entity","entityId":"switch-off-if-dangerous"},"topic":"ideas","time":{"kind":"event","date":"2006-06-06","basis":"publication","note":"Archive author index lists the message as \"Tue Jun 06 2006 - 23:19:49 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that we will recognise the moment when it has become dangerous enough to press the switch."}}}},{"id":"yk-y01-extropians-1996-12-08-yudkowsky-3761-c","subject":"extropians-1996-12-08-yudkowsky-3761","predicate":"contests_proposition","object":{"kind":"entity","entityId":"switch-off-if-dangerous"},"topic":"ideas","time":{"kind":"event","date":"1996-12-08","basis":"publication","note":"Archive author index lists the message as \"Sun, 08 Dec 1996 22:39:23 -0600\" under \"Eliezer Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a capable goal-directed system will not act to prevent its own shutdown or modification, and a shutdown or correction mechanism can simply be added to it."}}}},{"id":"yk-y01-extropians-1997-09-26-yudkowsky-4420-c","subject":"extropians-1997-09-26-yudkowsky-4420","predicate":"contests_proposition","object":{"kind":"entity","entityId":"switch-off-if-dangerous"},"topic":"ideas","time":{"kind":"event","date":"1997-09-26","basis":"publication","note":"Archive author index lists the message as \"Fri, 26 Sep 1997 18:20:36 -0500\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a capable goal-directed system will not act to prevent its own shutdown or modification, and a shutdown or correction mechanism can simply be added to it."}}}},{"id":"yk-y02-arbital-orthogonality-c","subject":"arbital-orthogonality","predicate":"contests_proposition","object":{"kind":"entity","entityId":"intelligence-brings-benevolence"},"topic":"ideas","time":{"kind":"event","date":"2015-03-10","basis":"publication","note":"Viewer shows \"10 Mar 2015\", probably the creation date; the body uses later terminology (\"AGI alignment\"), so the cached wording may be a later revision. Comments on the page are by others."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","attested":{"note":"Arbital pages could be edited after creation; the viewer shows a page date, and the quoted wording is known only from the page as retrieved in 2026."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that greater intelligence by itself brings benevolent goals."}}}},{"id":"yk-y02-lw-2008-no-universally-compelling-arguments-c","subject":"lw-2008-no-universally-compelling-arguments","predicate":"contests_proposition","object":{"kind":"entity","entityId":"intelligence-brings-benevolence"},"topic":"ideas","time":{"kind":"event","date":"2008-06-26","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a sufficiently intelligent mind would discover a true or universally compelling morality and be bound to act on it."}}}},{"id":"yk-y02-staring-into-the-singularity-b","subject":"staring-into-the-singularity","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"intelligence-brings-benevolence"},"topic":"ideas","time":{"kind":"event","date":"1996-11-18","basis":"publication","note":"The page states \"Created: 11/18/1996\"; the essay was announced on the Extropians list on 1996-11-19. As for the human-thought collection's records of this essay."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"A comparative and uncertain chance that superintelligence finds and implements what is right; it does not claim that intelligence brings or compels benevolence.","attested":{"date":"2000-12-04","note":"The quoted wording is that of version 1.2.1 (\"Updated: 08/31/2000\") as captured on 2000-12-04; no capture of the 1996–1999 page holds the text."}}}},{"id":"yk-y02-coherent-extrapolated-volition-2004-c","subject":"coherent-extrapolated-volition-2004","predicate":"contests_proposition","object":{"kind":"entity","entityId":"intelligence-brings-benevolence"},"topic":"ideas","time":{"kind":"event","date":"2004-05","basis":"publication","note":"Text states \"The information is current as of May 2004\"; reprint citation gives 2004. Precision: month."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","attested":{"note":"Known only from MIRI's later, undated PDF reprint of the May 2004 document; the wording was not compared with a 2004 copy."},"engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that a sufficiently intelligent mind would discover a true or universally compelling morality and be bound to act on it, even when read with its message or dialogue turn."},"retrospective":{"note":"Reports a belief he says he held from the age of sixteen until he recognised it as a problem at about twenty."}}}},{"id":"yk-y02-lw-2008-that-tiny-note-of-discord-c","subject":"lw-2008-that-tiny-note-of-discord","predicate":"contests_proposition","object":{"kind":"entity","entityId":"intelligence-brings-benevolence"},"topic":"ideas","time":{"kind":"event","date":"2008-09-23","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that greater intelligence by itself brings benevolent goals."},"retrospective":{"from":"1997","until":"1997","note":"Reports the belief of “Eliezer1997”."}}}},{"id":"yk-y02-sl4-2000-11-13-yudkowsky-0011-0181-c","subject":"sl4-2000-11-13-yudkowsky-0011-0181","predicate":"contests_proposition","object":{"kind":"entity","entityId":"intelligence-brings-benevolence"},"topic":"ideas","time":{"kind":"event","date":"2000-11-13","basis":"publication","note":"Archive author index lists the message as \"Mon Nov 13 2000 - 22:45:54 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a sufficiently intelligent mind would discover a true or universally compelling morality and be bound to act on it."},"retrospective":{"note":"Reports a belief he “used to have”."}}}},{"id":"yk-y02-sl4-2000-12-10-yudkowsky-0012-0355-c","subject":"sl4-2000-12-10-yudkowsky-0012-0355","predicate":"contests_proposition","object":{"kind":"entity","entityId":"intelligence-brings-benevolence"},"topic":"ideas","time":{"kind":"event","date":"2000-12-10","basis":"publication","note":"Archive author index lists the message as \"Sun Dec 10 2000 - 19:56:20 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that greater intelligence by itself brings benevolent goals."},"retrospective":{"note":"Reports a view he held about a year before the message."}}}},{"id":"yk-y02-sl4-2001-04-12-yudkowsky-0104-1076-c","subject":"sl4-2001-04-12-yudkowsky-0104-1076","predicate":"contests_proposition","object":{"kind":"entity","entityId":"intelligence-brings-benevolence"},"topic":"ideas","time":{"kind":"event","date":"2001-04-12","basis":"publication","note":"Archive author index lists the message as \"Thu Apr 12 2001 - 23:58:35 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that greater intelligence by itself brings benevolent goals, even when read with its message or dialogue turn."},"retrospective":{"note":"Reports a time when he thought superintelligence implied supermorality."}}}},{"id":"yk-y02-extropians-1996-11-22-yudkowsky-3178-a","subject":"extropians-1996-11-22-yudkowsky-3178","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"intelligence-brings-benevolence"},"topic":"ideas","time":{"kind":"event","date":"1996-11-22","basis":"publication","note":"Archive author index lists the message as \"Fri, 22 Nov 1996 02:36:13 -0600\" under \"Eliezer Yudkowsky\". Yudkowsky's reply to Tim Freeman dated 'Thu, 21 Nov 1996 19:07:03 -0600', forwarded by him to the list in this message sent 1996-11-22."},"attributes":{"thought":{"roles":["earliest-located"],"register":"nonfiction"}}},{"id":"yk-y02-extropians-1996-11-22-yudkowsky-3184-a","subject":"extropians-1996-11-22-yudkowsky-3184","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"intelligence-brings-benevolence"},"topic":"ideas","time":{"kind":"event","date":"1996-11-22","basis":"publication","note":"Archive author index lists the message as \"Fri, 22 Nov 1996 12:23:14 -0600\" under \"Eliezer Yudkowsky\"."},"attributes":{"thought":{"roles":["articulation"],"register":"nonfiction"}}},{"id":"yk-y02-extropians-1999-08-01-yudkowsky-1386-c","subject":"extropians-1999-08-01-yudkowsky-1386","predicate":"contests_proposition","object":{"kind":"entity","entityId":"intelligence-brings-benevolence"},"topic":"ideas","time":{"kind":"event","date":"1999-08-01","basis":"publication","note":"Archive author index lists the message as \"1999-08-01\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that greater intelligence by itself brings benevolent goals, even when read with its message or dialogue turn."},"retrospective":{"note":"Reports the view of his old Extropians posts and the old version of Staring into the Singularity."}}}},{"id":"yk-y03-ai-positive-negative-factor-global-risk-c","subject":"ai-positive-negative-factor-global-risk","predicate":"contests_proposition","object":{"kind":"entity","entityId":"understanding-implies-compliance"},"topic":"ideas","time":{"kind":"event","date":"2006-02-05","note":"Date of the SIAI draft (\"Draft of 5 February, 2006\"), whose capture of 2006-07-08 contains the quoted wording; the chapter was published in 2008. As for the human-thought collection's records of this chapter."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that if an AI understands what we intend, it will act on that understanding."}}}},{"id":"yk-y03-lw-2008-ghosts-in-the-machine-b","subject":"lw-2008-ghosts-in-the-machine","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"understanding-implies-compliance"},"topic":"ideas","time":{"kind":"event","date":"2008-06-17","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"It denies an inner interpreter of instructions; it does not address whether an AI that understands intentions acts on them."}}},{"id":"yk-y03-lw-2007-the-hidden-complexity-of-wishes-c","subject":"lw-2007-the-hidden-complexity-of-wishes","predicate":"contests_proposition","object":{"kind":"entity","entityId":"understanding-implies-compliance"},"topic":"ideas","time":{"kind":"event","date":"2007-11-24","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","attested":{"date":"2024-05","note":"Note signed 'EY, May 2024'; absent from the posting-day version 1.0.0 of the 2007 post."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that if an AI understands what we intend, it will act on that understanding."}}}},{"id":"yk-y03-lw-2026-the-talker-does-not-control-the-c","subject":"lw-2026-the-talker-does-not-control-the","predicate":"contests_proposition","object":{"kind":"entity","entityId":"understanding-implies-compliance"},"topic":"ideas","time":{"kind":"event","date":"2026-09-13","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that if an AI understands what we intend, it will act on that understanding."}}}},{"id":"yk-y03-sl4-2000-12-10-yudkowsky-0012-0355-c","subject":"sl4-2000-12-10-yudkowsky-0012-0355","predicate":"contests_proposition","object":{"kind":"entity","entityId":"understanding-implies-compliance"},"topic":"ideas","time":{"kind":"event","date":"2000-12-10","basis":"publication","note":"Archive author index lists the message as \"Sun Dec 10 2000 - 19:56:20 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that if an AI understands what we intend, it will act on that understanding."}}}},{"id":"yk-y04-lw-2008-what-i-think-if-not-why-c","subject":"lw-2008-what-i-think-if-not-why","predicate":"contests_proposition","object":{"kind":"entity","entityId":"short-instruction-suffices"},"topic":"ideas","time":{"kind":"event","date":"2008-12-11","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that a short instruction such as 'make people happy' specifies what we want, even when read with its message or dialogue turn."}}}},{"id":"yk-y04-coherent-extrapolated-volition-2004-c","subject":"coherent-extrapolated-volition-2004","predicate":"contests_proposition","object":{"kind":"entity","entityId":"short-instruction-suffices"},"topic":"ideas","time":{"kind":"event","date":"2004-05","basis":"publication","note":"Text states \"The information is current as of May 2004\"; reprint citation gives 2004. Precision: month."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","attested":{"note":"Known only from MIRI's later, undated PDF reprint of the May 2004 document; the wording was not compared with a 2004 copy."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a short instruction such as 'make people happy' specifies what we want."}}}},{"id":"yk-y04-lw-2007-the-hidden-complexity-of-wishes-c","subject":"lw-2007-the-hidden-complexity-of-wishes","predicate":"contests_proposition","object":{"kind":"entity","entityId":"short-instruction-suffices"},"topic":"ideas","time":{"kind":"event","date":"2007-11-24","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a short instruction such as 'make people happy' specifies what we want."}}}},{"id":"yk-y04-lw-2008-magical-categories-c","subject":"lw-2008-magical-categories","predicate":"contests_proposition","object":{"kind":"entity","entityId":"short-instruction-suffices"},"topic":"ideas","time":{"kind":"event","date":"2008-08-24","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a short instruction such as 'make people happy' specifies what we want."}}}},{"id":"yk-y04-sl4-2000-10-03-yudkowsky-0010-0125-c","subject":"sl4-2000-10-03-yudkowsky-0010-0125","predicate":"contests_proposition","object":{"kind":"entity","entityId":"short-instruction-suffices"},"topic":"ideas","time":{"kind":"event","date":"2000-10-03","basis":"publication","note":"Archive author index lists the message as \"Tue Oct 03 2000 - 22:53:59 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a short instruction such as 'make people happy' specifies what we want."}}}},{"id":"yk-y04-sl4-2003-05-29-yudkowsky-0305-6846-c","subject":"sl4-2003-05-29-yudkowsky-0305-6846","predicate":"contests_proposition","object":{"kind":"entity","entityId":"short-instruction-suffices"},"topic":"ideas","time":{"kind":"event","date":"2003-05-29","basis":"publication","note":"Archive author index lists the message as \"Thu May 29 2003 - 15:14:45 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a short instruction such as 'make people happy' specifies what we want."}}}},{"id":"yk-y04-extropians-1998-12-06-yudkowsky-2153-c","subject":"extropians-1998-12-06-yudkowsky-2153","predicate":"contests_proposition","object":{"kind":"entity","entityId":"short-instruction-suffices"},"topic":"ideas","time":{"kind":"event","date":"1998-12-06","basis":"publication","note":"Archive author index lists the message as \"1998-12-06\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a short instruction such as 'make people happy' specifies what we want."}}}},{"id":"yk-y04-extropians-1999-02-24-yudkowsky-2293-b","subject":"extropians-1999-02-24-yudkowsky-2293","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"short-instruction-suffices"},"topic":"ideas","time":{"kind":"event","date":"1999-02-24","basis":"publication","note":"Archive author index lists the message as \"1999-02-24\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"About goal drift from a happiness goal assumed to be sufficiently specified; it does not address whether a short instruction specifies what people want."}}},{"id":"yk-y04-extropians-2000-03-14-yudkowsky-4837-b","subject":"extropians-2000-03-14-yudkowsky-4837","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"short-instruction-suffices"},"topic":"ideas","time":{"kind":"event","date":"2000-03-14","basis":"publication","note":"Archive author index lists the message as \"Tue Mar 14 2000 - 00:26:34 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"It dismisses a perverse means of pursuing human happiness; it does not claim that a short instruction specifies what people want."}}},{"id":"yk-y05-agi-ruin-2022-c","subject":"agi-ruin-2022","predicate":"contests_proposition","object":{"kind":"entity","entityId":"feedback-instils-values"},"topic":"ideas","time":{"kind":"event","date":"2022-06-05","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that optimising a system for human approval or reward produces internalised human values."}}}},{"id":"yk-y05-coherent-extrapolated-volition-2004-c","subject":"coherent-extrapolated-volition-2004","predicate":"contests_proposition","object":{"kind":"entity","entityId":"feedback-instils-values"},"topic":"ideas","time":{"kind":"event","date":"2004-05","basis":"publication","note":"Text states \"The information is current as of May 2004\"; reprint citation gives 2004. Precision: month."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","attested":{"note":"Known only from MIRI's later, undated PDF reprint of the May 2004 document; the wording was not compared with a 2004 copy."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that optimising a system for human approval or reward produces internalised human values."}}}},{"id":"yk-y05-lw-2007-thou-art-godshatter-b","subject":"lw-2007-thou-art-godshatter","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"feedback-instils-values"},"topic":"ideas","time":{"kind":"event","date":"2007-11-13","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"About evolution and human desires, not about training AI on human feedback."}}},{"id":"yk-y05-lw-2023-alexander-and-yudkowsky-on-agi-goals-c","subject":"lw-2023-alexander-and-yudkowsky-on-agi-goals","predicate":"contests_proposition","object":{"kind":"entity","entityId":"feedback-instils-values"},"topic":"ideas","time":{"kind":"event","date":"2023-01-24","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that AI training will instil values the way upbringing instils morals in children."}}}},{"id":"yk-y05-lw-2026-the-talker-does-not-control-the-c","subject":"lw-2026-the-talker-does-not-control-the","predicate":"contests_proposition","object":{"kind":"entity","entityId":"feedback-instils-values"},"topic":"ideas","time":{"kind":"event","date":"2026-09-13","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that optimising a system for human approval or reward produces internalised human values."}}}},{"id":"yk-y05-sl4-2000-10-03-yudkowsky-0010-0125-c","subject":"sl4-2000-10-03-yudkowsky-0010-0125","predicate":"contests_proposition","object":{"kind":"entity","entityId":"feedback-instils-values"},"topic":"ideas","time":{"kind":"event","date":"2000-10-03","basis":"publication","note":"Archive author index lists the message as \"Tue Oct 03 2000 - 22:53:59 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that optimising a system for human approval or reward produces internalised human values."}}}},{"id":"yk-y05-sl4-2000-12-12-yudkowsky-0012-0381-c","subject":"sl4-2000-12-12-yudkowsky-0012-0381","predicate":"contests_proposition","object":{"kind":"entity","entityId":"feedback-instils-values"},"topic":"ideas","time":{"kind":"event","date":"2000-12-12","basis":"publication","note":"Archive author index lists the message as \"Tue Dec 12 2000 - 17:42:04 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that AI training will instil values the way upbringing instils morals in children."}}}},{"id":"yk-y05-sl4-2005-08-09-yudkowsky-0508-11866-b","subject":"sl4-2005-08-09-yudkowsky-0508-11866","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"feedback-instils-values"},"topic":"ideas","time":{"kind":"event","date":"2005-08-09","basis":"publication","note":"Archive author index lists the message as \"Tue Aug 09 2005 - 11:09:46 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"About autonomy and the responsibility of creators, not about training AI on human feedback."}}},{"id":"yk-y05-extropians-1999-08-04-yudkowsky-1587-c","subject":"extropians-1999-08-04-yudkowsky-1587","predicate":"contests_proposition","object":{"kind":"entity","entityId":"feedback-instils-values"},"topic":"ideas","time":{"kind":"event","date":"1999-08-04","basis":"publication","note":"Archive author index lists the message as \"1999-08-04\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that AI training will instil values the way upbringing instils morals in children."}}}},{"id":"yk-y05-extropians-1999-08-01-yudkowsky-1386-b","subject":"extropians-1999-08-01-yudkowsky-1386","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"feedback-instils-values"},"topic":"ideas","time":{"kind":"event","date":"1999-08-01","basis":"publication","note":"Archive author index lists the message as \"1999-08-01\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"About an internal indicator of goal achievement, not about human feedback or approval."}}},{"id":"yk-y06-extropians-1996-12-31-yudkowsky-5142-c","subject":"extropians-1996-12-31-yudkowsky-5142","predicate":"contests_proposition","object":{"kind":"entity","entityId":"confinement-keeps-it-safe"},"topic":"ideas","time":{"kind":"event","date":"1996-12-31","basis":"publication","note":"Archive author index lists the message as \"Tue, 31 Dec 1996 10:40:10 -0600\" under \"Eliezer Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a smarter-than-human mind confined to a channel with its human operators cannot get out through those operators."}}}},{"id":"yk-y06-extropians-1996-12-31-yudkowsky-5159-c","subject":"extropians-1996-12-31-yudkowsky-5159","predicate":"contests_proposition","object":{"kind":"entity","entityId":"confinement-keeps-it-safe"},"topic":"ideas","time":{"kind":"event","date":"1996-12-31","basis":"publication","note":"Archive author index lists the message as \"Tue, 31 Dec 1996 14:50:37 -0600\" under \"Eliezer Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a smarter-than-human mind confined to a channel with its human operators cannot get out through those operators."}}}},{"id":"yk-y06-ai-box-experiment-page-c","subject":"ai-box-experiment-page","predicate":"contests_proposition","object":{"kind":"entity","entityId":"confinement-keeps-it-safe"},"topic":"ideas","time":{"kind":"event","date":"2002","basis":"publication","note":"Page states \"This document is ©2002\"; the current page is a 2020 re-hosting (posted September 4, 2020, updated July 2, 2021) and carries a CC BY-ND 3.0 licence and a MIRI footer, so it was revised after 2002. The 2002 wording was not compared. Precision: year (stated copyright)."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","attested":{"date":"2021-07-02","note":"The page states ©2002, but the copy retrieved is a 2020 re-hosting (posted 2020-09-04, updated 2021-07-02) with a later licence and footer; the 2002 wording was not compared."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a smarter-than-human mind confined to a channel with its human operators cannot get out through those operators."}}}},{"id":"yk-y06-agi-ruin-2022-c","subject":"agi-ruin-2022","predicate":"contests_proposition","object":{"kind":"entity","entityId":"confinement-keeps-it-safe"},"topic":"ideas","time":{"kind":"event","date":"2022-06-05","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a smarter-than-human mind confined to a channel with its human operators cannot get out through those operators."}}}},{"id":"yk-y07-lw-2008-the-rhythm-of-disagreement-c","subject":"lw-2008-the-rhythm-of-disagreement","predicate":"contests_proposition","object":{"kind":"entity","entityId":"ai-solves-alignment"},"topic":"ideas","time":{"kind":"event","date":"2008-06-01","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that an AI weak enough to be safe is strong enough to solve alignment."}}}},{"id":"yk-y07-lw-2021-ngo-and-yudkowsky-on-alignment-difficulty-c","subject":"lw-2021-ngo-and-yudkowsky-on-alignment-difficulty","predicate":"contests_proposition","object":{"kind":"entity","entityId":"ai-solves-alignment"},"topic":"ideas","time":{"kind":"event","date":"2021-11-15","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that an AI weak enough to be safe is strong enough to solve alignment."}}}},{"id":"yk-y07-lw-2021-discussion-with-eliezer-yudkowsky-on-agi-c","subject":"lw-2021-discussion-with-eliezer-yudkowsky-on-agi","predicate":"contests_proposition","object":{"kind":"entity","entityId":"ai-solves-alignment"},"topic":"ideas","time":{"kind":"event","date":"2021-11-11","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that we can verify its alignment output."}}}},{"id":"yk-y07-time-2023-shut-it-all-down-c","subject":"time-2023-shut-it-all-down","predicate":"contests_proposition","object":{"kind":"entity","entityId":"ai-solves-alignment"},"topic":"ideas","time":{"kind":"event","date":"2023-03-29","basis":"publication","note":"Page metadata datePublished 2023-03-29T22:01:58Z. The page title currently reads \"The Only Way to Deal With the Threat From AI? Shut It Down\"; the headline is the one crossposted to LessWrong on 2023-04-08."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that an AI weak enough to be safe is strong enough to solve alignment, even when read with its message or dialogue turn."}}}},{"id":"yk-y07-lw-2026-cat-belling-problems-c","subject":"lw-2026-cat-belling-problems","predicate":"contests_proposition","object":{"kind":"entity","entityId":"ai-solves-alignment"},"topic":"ideas","time":{"kind":"event","date":"2026-09-03","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that we can verify its alignment output."}}}},{"id":"yk-y07-sl4-2005-01-02-yudkowsky-0501-10512-c","subject":"sl4-2005-01-02-yudkowsky-0501-10512","predicate":"contests_proposition","object":{"kind":"entity","entityId":"ai-solves-alignment"},"topic":"ideas","time":{"kind":"event","date":"2005-01-02","basis":"publication","note":"Archive author index lists the message as \"Sun Jan 02 2005 - 10:22:43 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that an AI weak enough to be safe is strong enough to solve alignment."}}}},{"id":"yk-y07-sl4-2008-02-11-yudkowsky-0802-17621-c","subject":"sl4-2008-02-11-yudkowsky-0802-17621","predicate":"contests_proposition","object":{"kind":"entity","entityId":"ai-solves-alignment"},"topic":"ideas","time":{"kind":"event","date":"2008-02-11","basis":"publication","note":"Archive author index lists the message as \"Mon Feb 11 2008 - 15:52:05 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that an AI weak enough to be safe is strong enough to solve alignment."}}}},{"id":"yk-y08-ai-positive-negative-factor-global-risk-c","subject":"ai-positive-negative-factor-global-risk","predicate":"contests_proposition","object":{"kind":"entity","entityId":"warning-signs-in-time"},"topic":"ideas","time":{"kind":"event","date":"2006-02-05","note":"Date of the SIAI draft (\"Draft of 5 February, 2006\"), whose capture of 2006-07-08 contains the quoted wording; the chapter was published in 2008. As for the human-thought collection's records of this chapter."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that there will be a recognisable, consensus-forming signal in time to act."}}}},{"id":"yk-y08-lw-2013-after-critical-event-w-happens-they-c","subject":"lw-2013-after-critical-event-w-happens-they","predicate":"contests_proposition","object":{"kind":"entity","entityId":"warning-signs-in-time"},"topic":"ideas","time":{"kind":"event","date":"2013-06-13","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that there will be a recognisable, consensus-forming signal in time to act."}}}},{"id":"yk-y08-lw-2017-theres-no-fire-alarm-for-artificial-c","subject":"lw-2017-theres-no-fire-alarm-for-artificial","predicate":"contests_proposition","object":{"kind":"entity","entityId":"warning-signs-in-time"},"topic":"ideas","time":{"kind":"event","date":"2017-10-13","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that there will be a recognisable, consensus-forming signal in time to act."}}}},{"id":"yk-y08-agi-ruin-2022-c","subject":"agi-ruin-2022","predicate":"contests_proposition","object":{"kind":"entity","entityId":"warning-signs-in-time"},"topic":"ideas","time":{"kind":"event","date":"2022-06-05","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that dangerous failure modes will show up first in weaker systems, where they can be observed and fixed."}}}},{"id":"yk-y08-lw-2022-miri-announces-new-death-with-dignity-c","subject":"lw-2022-miri-announces-new-death-with-dignity","predicate":"contests_proposition","object":{"kind":"entity","entityId":"warning-signs-in-time"},"topic":"ideas","time":{"kind":"event","date":"2022-04-02","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that once a warning sign is seen, it will lead to effective action."}}}},{"id":"yk-y08-sl4-2000-07-09-yudkowsky-0007-0017-b","subject":"sl4-2000-07-09-yudkowsky-0007-0017","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"warning-signs-in-time"},"topic":"ideas","time":{"kind":"event","date":"2000-07-09","basis":"publication","note":"Archive author index lists the message as \"Sun Jul 09 2000 - 13:53:37 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"Confidence in a future understanding of AI motivations; it does not claim that dangerous failure modes first appear in weaker systems."}}},{"id":"yk-y08-sl4-2006-06-06-yudkowsky-0606-15174-c","subject":"sl4-2006-06-06-yudkowsky-0606-15174","predicate":"contests_proposition","object":{"kind":"entity","entityId":"warning-signs-in-time"},"topic":"ideas","time":{"kind":"event","date":"2006-06-06","basis":"publication","note":"Archive author index lists the message as \"Tue Jun 06 2006 - 23:19:49 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that dangerous failure modes will show up first in weaker systems, where they can be observed and fixed."}}}},{"id":"yk-y08-sl4-2005-07-13-yudkowsky-0507-11477-c","subject":"sl4-2005-07-13-yudkowsky-0507-11477","predicate":"contests_proposition","object":{"kind":"entity","entityId":"warning-signs-in-time"},"topic":"ideas","time":{"kind":"event","date":"2005-07-13","basis":"publication","note":"Archive author index lists the message as \"Wed Jul 13 2005 - 18:56:10 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that once a warning sign is seen, it will lead to effective action, even when read with its message or dialogue turn."}}}},{"id":"yk-y08-extropians-1999-01-06-yudkowsky-0160-c","subject":"extropians-1999-01-06-yudkowsky-0160","predicate":"contests_proposition","object":{"kind":"entity","entityId":"warning-signs-in-time"},"topic":"ideas","time":{"kind":"event","date":"1999-01-06","basis":"publication","note":"Archive author index lists the message as \"1999-01-06\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that there will be a recognisable, consensus-forming signal in time to act, even when read with its message or dialogue turn."}}}},{"id":"yk-y09-lw-2022-shah-and-yudkowsky-on-alignment-failures-c","subject":"lw-2022-shah-and-yudkowsky-on-alignment-failures","predicate":"contests_proposition","object":{"kind":"entity","entityId":"next-word-prediction-is-shallow"},"topic":"ideas","time":{"kind":"event","date":"2022-02-28","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that next-token prediction is a shallow task that cannot require deep cognition."}}}},{"id":"yk-y09-lw-2023-gpts-are-predictors-not-imitators-c","subject":"lw-2023-gpts-are-predictors-not-imitators","predicate":"contests_proposition","object":{"kind":"entity","entityId":"next-word-prediction-is-shallow"},"topic":"ideas","time":{"kind":"event","date":"2023-04-08","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a system trained to predict human text is capped at imitating the humans who wrote it, and against the premise that next-token prediction is a shallow task that cannot require deep cognition."}}}},{"id":"yk-y10-staring-into-the-singularity-c","subject":"staring-into-the-singularity","predicate":"contests_proposition","object":{"kind":"entity","entityId":"diminishing-returns-to-intelligence"},"topic":"ideas","time":{"kind":"event","date":"1996-11-18","basis":"publication","note":"The page states \"Created: 11/18/1996\"; the essay was announced on the Extropians list on 1996-11-19. As for the human-thought collection's records of this essay."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","attested":{"date":"2000-12-04","note":"The quoted wording is that of version 1.2.1 (\"Updated: 08/31/2000\") as captured on 2000-12-04; no capture of the 1996–1999 page holds the text."},"engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that returns to reinvesting cognitive gains in further cognitive improvement diminish, even when read with its message or dialogue turn."}}}},{"id":"yk-y10-yudkowsky-2008-recursive-self-improvement-c","subject":"yudkowsky-2008-recursive-self-improvement","predicate":"contests_proposition","object":{"kind":"entity","entityId":"diminishing-returns-to-intelligence"},"topic":"ideas","time":{"kind":"event","date":"2008-12-01","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that returns to reinvesting cognitive gains in further cognitive improvement diminish."}}}},{"id":"yk-y10-iem-2013-c","subject":"iem-2013","predicate":"contests_proposition","object":{"kind":"entity","entityId":"diminishing-returns-to-intelligence"},"topic":"ideas","time":{"kind":"event","date":"2013-09-13","basis":"publication","note":"Report states \"Last modified September 13, 2013\"; first released April 2013."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that returns to reinvesting cognitive gains in further cognitive improvement diminish."}}}},{"id":"yk-y10-extropians-1996-11-13-yudkowsky-2941-c","subject":"extropians-1996-11-13-yudkowsky-2941","predicate":"contests_proposition","object":{"kind":"entity","entityId":"diminishing-returns-to-intelligence"},"topic":"ideas","time":{"kind":"event","date":"1996-11-13","basis":"publication","note":"Archive author index lists the message as \"Wed, 13 Nov 1996 18:57:33 -0600\" under \"Eliezer Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that returns to reinvesting cognitive gains in further cognitive improvement diminish, even when read with its message or dialogue turn."}}}},{"id":"yk-y10-extropians-1996-11-16-yudkowsky-3042-c","subject":"extropians-1996-11-16-yudkowsky-3042","predicate":"contests_proposition","object":{"kind":"entity","entityId":"diminishing-returns-to-intelligence"},"topic":"ideas","time":{"kind":"event","date":"1996-11-16","basis":"publication","note":"Archive author index lists the message as \"Sat, 16 Nov 1996 22:37:39 -0600\" under \"Eliezer Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that returns to reinvesting cognitive gains in further cognitive improvement diminish."}}}},{"id":"yk-y10-extropians-1998-09-18-yudkowsky-2487-c","subject":"extropians-1998-09-18-yudkowsky-2487","predicate":"contests_proposition","object":{"kind":"entity","entityId":"diminishing-returns-to-intelligence"},"topic":"ideas","time":{"kind":"event","date":"1998-09-18","basis":"publication","note":"Archive author index lists the message as \"1998-09-18\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that returns to reinvesting cognitive gains in further cognitive improvement diminish."}}}},{"id":"yk-y11-creating-friendly-ai-2001-c","subject":"creating-friendly-ai-2001","predicate":"contests_proposition","object":{"kind":"entity","entityId":"slow-takeoff-time-to-adjust"},"topic":"ideas","time":{"kind":"event","date":"2001-06-15","basis":"publication","note":"Release date from the reprint; an open-commentary version was announced on SL4 on 2001-04-18, so some wording may be earlier than this date. Reprint wording is not authenticated for 2001 unless a capture was compared."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","attested":{"date":"2001-06-26","note":"The quoted wording also appears in the 2001 HTML edition as captured on 2001-06-26; the quotation itself is taken from MIRI’s later reprint."},"engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that capability gains will be gradual and visible, even when read with its message or dialogue turn."}}}},{"id":"yk-y11-yudkowsky-2008-hard-takeoff-c","subject":"yudkowsky-2008-hard-takeoff","predicate":"contests_proposition","object":{"kind":"entity","entityId":"slow-takeoff-time-to-adjust"},"topic":"ideas","time":{"kind":"event","date":"2008-12-02","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that capability gains will be gradual and visible."}}}},{"id":"yk-y11-lw-2021-yudkowsky-and-christiano-discuss-takeoff-speeds-c","subject":"lw-2021-yudkowsky-and-christiano-discuss-takeoff-speeds","predicate":"contests_proposition","object":{"kind":"entity","entityId":"slow-takeoff-time-to-adjust"},"topic":"ideas","time":{"kind":"event","date":"2021-11-22","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, one passage, read with the message or dialogue turn it belongs to, gives a reason against the premise that slightly weaker predecessor systems will give us time and experience to adjust before the decisive one, while another states the contrary conclusion only."}}}},{"id":"yk-y11-agi-ruin-2022-c","subject":"agi-ruin-2022","predicate":"contests_proposition","object":{"kind":"entity","entityId":"slow-takeoff-time-to-adjust"},"topic":"ideas","time":{"kind":"event","date":"2022-06-05","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that slightly weaker predecessor systems will give us time and experience to adjust before the decisive one."}}}},{"id":"yk-y11-sl4-2000-10-03-yudkowsky-0010-0125-c","subject":"sl4-2000-10-03-yudkowsky-0010-0125","predicate":"contests_proposition","object":{"kind":"entity","entityId":"slow-takeoff-time-to-adjust"},"topic":"ideas","time":{"kind":"event","date":"2000-10-03","basis":"publication","note":"Archive author index lists the message as \"Tue Oct 03 2000 - 22:53:59 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that capability gains will be gradual and visible, even when read with its message or dialogue turn."}}}},{"id":"yk-y11-sl4-2001-01-24-yudkowsky-0101-0452-c","subject":"sl4-2001-01-24-yudkowsky-0101-0452","predicate":"contests_proposition","object":{"kind":"entity","entityId":"slow-takeoff-time-to-adjust"},"topic":"ideas","time":{"kind":"event","date":"2001-01-24","basis":"publication","note":"Archive author index lists the message as \"Wed Jan 24 2001 - 09:12:47 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that capability gains will be gradual and visible."}}}},{"id":"yk-y11-sl4-2000-07-09-yudkowsky-0007-0017-b","subject":"sl4-2000-07-09-yudkowsky-0007-0017","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"slow-takeoff-time-to-adjust"},"topic":"ideas","time":{"kind":"event","date":"2000-07-09","basis":"publication","note":"Archive author index lists the message as \"Sun Jul 09 2000 - 13:53:37 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"About understanding AI motivations before AI is a threat, not about takeoff speed or time to adjust."}}},{"id":"yk-y11-sl4-2004-12-29-yudkowsky-0412-10488-c","subject":"sl4-2004-12-29-yudkowsky-0412-10488","predicate":"contests_proposition","object":{"kind":"entity","entityId":"slow-takeoff-time-to-adjust"},"topic":"ideas","time":{"kind":"event","date":"2004-12-29","basis":"publication","note":"Archive author index lists the message as \"Wed Dec 29 2004 - 16:27:43 MST\" under \"Eliezer Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that slightly weaker predecessor systems will give us time and experience to adjust before the decisive one."}}}},{"id":"yk-y11-sl4-2000-08-21-yudkowsky-0008-0046-b","subject":"sl4-2000-08-21-yudkowsky-0008-0046","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"slow-takeoff-time-to-adjust"},"topic":"ideas","time":{"kind":"event","date":"2000-08-21","basis":"publication","note":"Archive author index lists the message as \"Mon Aug 21 2000 - 00:41:14 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"A strategy statement that a slow Singularity carries other technological risks; it makes no claim about takeoff speed or time to adjust to AI."}}},{"id":"yk-y11-extropians-1998-09-08-yudkowsky-2235-c","subject":"extropians-1998-09-08-yudkowsky-2235","predicate":"contests_proposition","object":{"kind":"entity","entityId":"slow-takeoff-time-to-adjust"},"topic":"ideas","time":{"kind":"event","date":"1998-09-08","basis":"publication","note":"Archive author index lists the message as \"1998-09-08\" under \"Eliezer S. Yudkowsky\". Sent header: Tue, 08 Sep 1998 14:22:05 -0500."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that capability gains will be gradual and visible."}}}},{"id":"yk-y11-extropians-1999-08-03-yudkowsky-1487-c","subject":"extropians-1999-08-03-yudkowsky-1487","predicate":"contests_proposition","object":{"kind":"entity","entityId":"slow-takeoff-time-to-adjust"},"topic":"ideas","time":{"kind":"event","date":"1999-08-03","basis":"publication","note":"Archive author index lists the message as \"1999-08-03\" under \"Eliezer S. Yudkowsky\". Sent header: Tue, 03 Aug 1999 15:02:17 -0500."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that capability gains will be gradual and visible."}}}},{"id":"yk-y12-creating-friendly-ai-2001-c","subject":"creating-friendly-ai-2001","predicate":"contests_proposition","object":{"kind":"entity","entityId":"evaluations-catch-danger"},"topic":"ideas","time":{"kind":"event","date":"2001-06-15","basis":"publication","note":"Release date from the reprint; an open-commentary version was announced on SL4 on 2001-04-18, so some wording may be earlier than this date. Reprint wording is not authenticated for 2001 unless a capture was compared."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","attested":{"date":"2001-06-17","note":"The quoted wording also appears in the 2001 HTML edition as captured on 2001-06-17; the quotation itself is taken from MIRI’s later reprint."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a dangerous system would behave dangerously while it is being tested."}}}},{"id":"yk-y12-arbital-programmer-deception-c","subject":"arbital-programmer-deception","predicate":"contests_proposition","object":{"kind":"entity","entityId":"evaluations-catch-danger"},"topic":"ideas","time":{"kind":"event","date":"2015-07-16","basis":"publication","note":"Viewer shows \"16 Jul 2015\"; creation or last edit not stated."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","attested":{"note":"Arbital pages could be edited after creation; the viewer shows a page date, and the quoted wording is known only from the page as retrieved in 2026."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a dangerous system would behave dangerously while it is being tested."}}}},{"id":"yk-y12-agi-ruin-2022-c","subject":"agi-ruin-2022","predicate":"contests_proposition","object":{"kind":"entity","entityId":"evaluations-catch-danger"},"topic":"ideas","time":{"kind":"event","date":"2022-06-05","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a dangerous system would behave dangerously while it is being tested."}}}},{"id":"yk-y12-lw-2026-irretrievability-or-murphys-curse-of-oneshotness-c","subject":"lw-2026-irretrievability-or-murphys-curse-of-oneshotness","predicate":"contests_proposition","object":{"kind":"entity","entityId":"evaluations-catch-danger"},"topic":"ideas","time":{"kind":"event","date":"2026-05-04","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that tests on systems that cannot yet cause a catastrophe show how more capable systems will behave."}}}},{"id":"yk-y13-ai-positive-negative-factor-global-risk-c","subject":"ai-positive-negative-factor-global-risk","predicate":"contests_proposition","object":{"kind":"entity","entityId":"agi-decades-away"},"topic":"ideas","time":{"kind":"event","date":"2006-02-05","note":"Date of the SIAI draft (\"Draft of 5 February, 2006\"), whose capture of 2006-07-08 contains the quoted wording; the chapter was published in 2008. As for the human-thought collection's records of this chapter."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that long timelines mean there is no need to act now."}}}},{"id":"yk-y13-lw-2017-theres-no-fire-alarm-for-artificial-c","subject":"lw-2017-theres-no-fire-alarm-for-artificial","predicate":"contests_proposition","object":{"kind":"entity","entityId":"agi-decades-away"},"topic":"ideas","time":{"kind":"event","date":"2017-10-13","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that we can know that AGI is decades away, and against the premise that long timelines mean there is no need to act now."}}}},{"id":"yk-y13-lw-2021-biology-inspired-agi-timelines-the-trick-c","subject":"lw-2021-biology-inspired-agi-timelines-the-trick","predicate":"contests_proposition","object":{"kind":"entity","entityId":"agi-decades-away"},"topic":"ideas","time":{"kind":"event","date":"2021-12-01","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"fiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that we can know that AGI is decades away."}}}},{"id":"yk-y13-sl4-2000-11-18-yudkowsky-0011-0240-b","subject":"sl4-2000-11-18-yudkowsky-0011-0240","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"agi-decades-away"},"topic":"ideas","time":{"kind":"event","date":"2000-11-18","basis":"publication","note":"Archive author index lists the message as \"Sat Nov 18 2000 - 12:34:51 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"About which Friendly AI work can be done in advance, not about AGI timelines. A report of his earlier belief about advance knowledge of Friendly AI, not about AGI timelines.","retrospective":{"note":"Reports a belief he held about a year before the message."}}}},{"id":"yk-y13-sl4-2002-07-02-yudkowsky-0207-4667-b","subject":"sl4-2002-07-02-yudkowsky-0207-4667","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"agi-decades-away"},"topic":"ideas","time":{"kind":"event","date":"2002-07-02","basis":"publication","note":"Archive author index lists the message as \"Tue Jul 02 2002 - 23:01:42 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"About the stage of an AI's development at which Friendliness work must start, not about calendar timelines."}}},{"id":"yk-y14-creating-friendly-ai-2001-b","subject":"creating-friendly-ai-2001","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"fix-problems-as-they-arise"},"topic":"ideas","time":{"kind":"event","date":"2001-06-15","basis":"publication","note":"Release date from the reprint; an open-commentary version was announced on SL4 on 2001-04-18, so some wording may be earlier than this date. Reprint wording is not authenticated for 2001 unless a capture was compared."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"A conservative design assumption in a table, not a forecast that failures with powerful AI will be unrecoverable.","attested":{"date":"2001-06-26","note":"The quoted wording also appears in the 2001 HTML edition as captured on 2001-06-26; the quotation itself is taken from MIRI’s later reprint."}}}},{"id":"yk-y14-time-2023-shut-it-all-down-c","subject":"time-2023-shut-it-all-down","predicate":"contests_proposition","object":{"kind":"entity","entityId":"fix-problems-as-they-arise"},"topic":"ideas","time":{"kind":"event","date":"2023-03-29","basis":"publication","note":"Page metadata datePublished 2023-03-29T22:01:58Z. The page title currently reads \"The Only Way to Deal With the Threat From AI? Shut It Down\"; the headline is the one crossposted to LessWrong on 2023-04-08."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that we get to learn from failures with powerful AI and try again, as with other technologies."}}}},{"id":"yk-y14-lw-2026-irretrievability-or-murphys-curse-of-oneshotness-c","subject":"lw-2026-irretrievability-or-murphys-curse-of-oneshotness","predicate":"contests_proposition","object":{"kind":"entity","entityId":"fix-problems-as-they-arise"},"topic":"ideas","time":{"kind":"event","date":"2026-05-04","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that we get to learn from failures with powerful AI and try again, as with other technologies."}}}},{"id":"yk-y14-sl4-2000-09-17-yudkowsky-0009-0092-c","subject":"sl4-2000-09-17-yudkowsky-0009-0092","predicate":"contests_proposition","object":{"kind":"entity","entityId":"fix-problems-as-they-arise"},"topic":"ideas","time":{"kind":"event","date":"2000-09-17","basis":"publication","note":"Archive author index lists the message as \"Sun Sep 17 2000 - 10:19:16 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that we get to learn from failures with powerful AI and try again, as with other technologies."}}}},{"id":"yk-y14-sl4-2002-09-10-yudkowsky-0209-5398-c","subject":"sl4-2002-09-10-yudkowsky-0209-5398","predicate":"contests_proposition","object":{"kind":"entity","entityId":"fix-problems-as-they-arise"},"topic":"ideas","time":{"kind":"event","date":"2002-09-10","basis":"publication","note":"Archive author index lists the message as \"Tue Sep 10 2002 - 07:46:27 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that we get to learn from failures with powerful AI and try again, as with other technologies."}}}},{"id":"yk-y14-sl4-2005-07-25-yudkowsky-0507-11765-b","subject":"sl4-2005-07-25-yudkowsky-0507-11765","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"fix-problems-as-they-arise"},"topic":"ideas","time":{"kind":"event","date":"2005-07-25","basis":"publication","note":"Archive author index lists the message as \"Mon Jul 25 2005 - 09:49:15 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"A later description of a plan to build AI and Friendliness in parallel; it does not address learning from failures with powerful AI.","retrospective":{"from":"2001","until":"2002","note":"Reports the plan of Creating Friendly AI (2001), which he says he gave up after realising in 2002 that almost every AI design would kill him."}}}},{"id":"yk-y14-extropians-2000-09-05-yudkowsky-4277-c","subject":"extropians-2000-09-05-yudkowsky-4277","predicate":"contests_proposition","object":{"kind":"entity","entityId":"fix-problems-as-they-arise"},"topic":"ideas","time":{"kind":"event","date":"2000-09-05","basis":"publication","note":"Archive author index lists the message as \"Tue Sep 05 2000 - 12:19:43 MDT\" under \"Eliezer S. Yudkowsky\". Sent header: Tue, 05 Sep 2000 14:19:43 -0400."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that we get to learn from failures with powerful AI and try again, as with other technologies."}}}},{"id":"yk-y15-lw-2008-the-rhythm-of-disagreement-c","subject":"lw-2008-the-rhythm-of-disagreement","predicate":"contests_proposition","object":{"kind":"entity","entityId":"tool-without-goals"},"topic":"ideas","time":{"kind":"event","date":"2008-06-01","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a system can be highly capable without having goals or optimising for outcomes."}}}},{"id":"yk-y15-lw-2008-dreams-of-friendliness-c","subject":"lw-2008-dreams-of-friendliness","predicate":"contests_proposition","object":{"kind":"entity","entityId":"tool-without-goals"},"topic":"ideas","time":{"kind":"event","date":"2008-08-31","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a system can be highly capable without having goals or optimising for outcomes."}}}},{"id":"yk-y15-lw-2012-reply-to-holden-on-tool-ai-c","subject":"lw-2012-reply-to-holden-on-tool-ai","predicate":"contests_proposition","object":{"kind":"entity","entityId":"tool-without-goals"},"topic":"ideas","time":{"kind":"event","date":"2012-06-12","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a system that only outputs answers or plans, leaving humans to act on them, is safe."}}}},{"id":"yk-y15-lw-2021-ngo-and-yudkowsky-on-alignment-difficulty-c","subject":"lw-2021-ngo-and-yudkowsky-on-alignment-difficulty","predicate":"contests_proposition","object":{"kind":"entity","entityId":"tool-without-goals"},"topic":"ideas","time":{"kind":"event","date":"2021-11-15","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a system can be highly capable without having goals or optimising for outcomes."}}}},{"id":"yk-y15-sl4-2001-02-25-yudkowsky-0102-0594-c","subject":"sl4-2001-02-25-yudkowsky-0102-0594","predicate":"contests_proposition","object":{"kind":"entity","entityId":"tool-without-goals"},"topic":"ideas","time":{"kind":"event","date":"2001-02-25","basis":"publication","note":"Archive author index lists the message as \"Sun Feb 25 2001 - 11:44:18 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that a system that only outputs answers or plans, leaving humans to act on them, is safe, even when read with its message or dialogue turn."}}}},{"id":"yk-y15-extropians-1998-09-15-yudkowsky-2411-c","subject":"extropians-1998-09-15-yudkowsky-2411","predicate":"contests_proposition","object":{"kind":"entity","entityId":"tool-without-goals"},"topic":"ideas","time":{"kind":"event","date":"1998-09-15","basis":"publication","note":"Archive author index lists the message as \"1998-09-15\" under \"Eliezer S. Yudkowsky\". Sent header: Tue, 15 Sep 1998 09:20:52 -0500."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a system can be highly capable without having goals or optimising for outcomes."}}}},{"id":"yk-y15-extropians-1999-04-08-yudkowsky-0310-c","subject":"extropians-1999-04-08-yudkowsky-0310","predicate":"contests_proposition","object":{"kind":"entity","entityId":"tool-without-goals"},"topic":"ideas","time":{"kind":"event","date":"1999-04-08","basis":"publication","note":"Archive author index lists the message as \"1999-04-08\" under \"Eliezer S. Yudkowsky\". Sent header: Thu, 08 Apr 1999 20:46:00 -0500."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a system can be highly capable without having goals or optimising for outcomes."}}}},{"id":"yk-y16-agi-ruin-2022-c","subject":"agi-ruin-2022","predicate":"contests_proposition","object":{"kind":"entity","entityId":"give-it-human-values"},"topic":"ideas","time":{"kind":"event","date":"2022-06-05","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that human values can be written down or taught as a ready, compact specification."}}}},{"id":"yk-y16-coherent-extrapolated-volition-2004-c","subject":"coherent-extrapolated-volition-2004","predicate":"contests_proposition","object":{"kind":"entity","entityId":"give-it-human-values"},"topic":"ideas","time":{"kind":"event","date":"2004-05","basis":"publication","note":"Text states \"The information is current as of May 2004\"; reprint citation gives 2004. Precision: month."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","attested":{"note":"Known only from MIRI's later, undated PDF reprint of the May 2004 document; the wording was not compared with a 2004 copy."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that human values can be written down or taught as a ready, compact specification."}}}},{"id":"yk-y16-lw-2009-value-is-fragile-c","subject":"lw-2009-value-is-fragile","predicate":"contests_proposition","object":{"kind":"entity","entityId":"give-it-human-values"},"topic":"ideas","time":{"kind":"event","date":"2009-01-29","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that human values can be written down or taught as a ready, compact specification."}}}},{"id":"yk-y16-sl4-2000-10-03-yudkowsky-0010-0125-c","subject":"sl4-2000-10-03-yudkowsky-0010-0125","predicate":"contests_proposition","object":{"kind":"entity","entityId":"give-it-human-values"},"topic":"ideas","time":{"kind":"event","date":"2000-10-03","basis":"publication","note":"Archive author index lists the message as \"Tue Oct 03 2000 - 22:53:59 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that human values can be written down or taught as a ready, compact specification."}}}},{"id":"yk-y16-sl4-2002-06-27-yudkowsky-0206-4453-c","subject":"sl4-2002-06-27-yudkowsky-0206-4453","predicate":"contests_proposition","object":{"kind":"entity","entityId":"give-it-human-values"},"topic":"ideas","time":{"kind":"event","date":"2002-06-27","basis":"publication","note":"Archive author index lists the message as \"Thu Jun 27 2002 - 11:53:11 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that present-day or programmers' values are settled enough to fix permanently in an AI."}}}},{"id":"yk-y16-sl4-2000-11-18-yudkowsky-0011-0240-c","subject":"sl4-2000-11-18-yudkowsky-0011-0240","predicate":"contests_proposition","object":{"kind":"entity","entityId":"give-it-human-values"},"topic":"ideas","time":{"kind":"event","date":"2000-11-18","basis":"publication","note":"Archive author index lists the message as \"Sat Nov 18 2000 - 12:34:51 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that present-day or programmers' values are settled enough to fix permanently in an AI, even when read with its message or dialogue turn."}}}},{"id":"yk-y16-extropians-1996-12-08-yudkowsky-3761-c","subject":"extropians-1996-12-08-yudkowsky-3761","predicate":"contests_proposition","object":{"kind":"entity","entityId":"give-it-human-values"},"topic":"ideas","time":{"kind":"event","date":"1996-12-08","basis":"publication","note":"Archive author index lists the message as \"Sun, 08 Dec 1996 22:39:23 -0600\" under \"Eliezer Yudkowsky\". Sent header: Sun, 08 Dec 1996 22:39:23 -0600."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that present-day or programmers' values are settled enough to fix permanently in an AI."}}}},{"id":"yk-y17-time-2023-shut-it-all-down-c","subject":"time-2023-shut-it-all-down","predicate":"contests_proposition","object":{"kind":"entity","entityId":"pause-or-regulation-suffices"},"topic":"ideas","time":{"kind":"event","date":"2023-03-29","basis":"publication","note":"Page metadata datePublished 2023-03-29T22:01:58Z. The page title currently reads \"The Only Way to Deal With the Threat From AI? Shut It Down\"; the headline is the one crossposted to LessWrong on 2023-04-08."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a short pause (for example six months) is enough."}}}},{"id":"yk-y17-time-2023-shut-it-all-down-b","subject":"time-2023-shut-it-all-down","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"pause-or-regulation-suffices"},"topic":"ideas","time":{"kind":"event","date":"2023-03-29","basis":"publication","note":"Page metadata datePublished 2023-03-29T22:01:58Z. The page title currently reads \"The Only Way to Deal With the Threat From AI? Shut It Down\"; the headline is the one crossposted to LessWrong on 2023-04-08."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"It presents an enforced, indefinite and worldwide halt as what would need to be done, not a pause or regulation that will handle the risk."}}},{"id":"yk-y17-creating-friendly-ai-2001-c","subject":"creating-friendly-ai-2001","predicate":"contests_proposition","object":{"kind":"entity","entityId":"pause-or-regulation-suffices"},"topic":"ideas","time":{"kind":"event","date":"2001-06-15","basis":"publication","note":"Release date from the reprint; an open-commentary version was announced on SL4 on 2001-04-18, so some wording may be earlier than this date. Reprint wording is not authenticated for 2001 unless a capture was compared."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","attested":{"date":"2001-06-17","note":"The quoted wording also appears in the 2001 HTML edition as captured on 2001-06-17; the quotation itself is taken from MIRI’s later reprint."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that regulation that is national, partial or not universally enforced can stop dangerous development."}}}},{"id":"yk-y17-lw-2026-only-law-can-prevent-extinction-c","subject":"lw-2026-only-law-can-prevent-extinction","predicate":"contests_proposition","object":{"kind":"entity","entityId":"pause-or-regulation-suffices"},"topic":"ideas","time":{"kind":"event","date":"2026-04-13","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that regulation that is national, partial or not universally enforced can stop dangerous development."}}}},{"id":"yk-y17-lw-2026-only-law-can-prevent-extinction-b","subject":"lw-2026-only-law-can-prevent-extinction","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"pause-or-regulation-suffices"},"topic":"ideas","time":{"kind":"event","date":"2026-04-13","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"It says that only lawful, state-backed and universal force could stop superintelligence being built; it does not claim that some regulation will handle the risk."}}},{"id":"yk-y17-sl4-2001-04-06-yudkowsky-0104-0956-c","subject":"sl4-2001-04-06-yudkowsky-0104-0956","predicate":"contests_proposition","object":{"kind":"entity","entityId":"pause-or-regulation-suffices"},"topic":"ideas","time":{"kind":"event","date":"2001-04-06","basis":"publication","note":"Archive author index lists the message as \"Fri Apr 06 2001 - 16:53:20 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that regulation that is national, partial or not universally enforced can stop dangerous development."}}}},{"id":"yk-y17-sl4-2001-04-08-yudkowsky-0104-1019-c","subject":"sl4-2001-04-08-yudkowsky-0104-1019","predicate":"contests_proposition","object":{"kind":"entity","entityId":"pause-or-regulation-suffices"},"topic":"ideas","time":{"kind":"event","date":"2001-04-08","basis":"publication","note":"Archive author index lists the message as \"Sun Apr 08 2001 - 09:50:57 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that law can prevent the danger (the part he states himself, as an enforced worldwide halt)."}}}},{"id":"yk-y17-extropians-1998-12-06-yudkowsky-2153-c","subject":"extropians-1998-12-06-yudkowsky-2153","predicate":"contests_proposition","object":{"kind":"entity","entityId":"pause-or-regulation-suffices"},"topic":"ideas","time":{"kind":"event","date":"1998-12-06","basis":"publication","note":"Archive author index lists the message as \"1998-12-06\" under \"Eliezer S. Yudkowsky\". Sent header: Sun, 06 Dec 1998 12:13:25 -0600."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that law can prevent the danger (the part he states himself, as an enforced worldwide halt), even when read with its message or dialogue turn."}}}},{"id":"yk-y17-extropians-1999-10-21-yudkowsky-1086-c","subject":"extropians-1999-10-21-yudkowsky-1086","predicate":"contests_proposition","object":{"kind":"entity","entityId":"pause-or-regulation-suffices"},"topic":"ideas","time":{"kind":"event","date":"1999-10-21","basis":"publication","note":"Archive author index lists the message as \"1999-10-21\" under \"Eliezer S. Yudkowsky\". Sent header: Thu, 21 Oct 1999 09:30:25 -0500."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that law can prevent the danger (the part he states himself, as an enforced worldwide halt)."}}}},{"id":"yk-y18-lw-2024-the-sun-is-big-but-superintelligences-c","subject":"lw-2024-the-sun-is-big-but-superintelligences","predicate":"contests_proposition","object":{"kind":"entity","entityId":"earth-not-worth-taking"},"topic":"ideas","time":{"kind":"event","date":"2024-09-23","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a superintelligence would leave us a small share, because sparing Earth would cost it very little, and against the premise that humans could trade with a superintelligence for what we need."}}}},{"id":"yk-y18-sl4-2000-10-03-yudkowsky-0010-0125-c","subject":"sl4-2000-10-03-yudkowsky-0010-0125","predicate":"contests_proposition","object":{"kind":"entity","entityId":"earth-not-worth-taking"},"topic":"ideas","time":{"kind":"event","date":"2000-10-03","basis":"publication","note":"Archive author index lists the message as \"Tue Oct 03 2000 - 22:53:59 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that humans could trade with a superintelligence for what we need."}}}},{"id":"yk-y18-sl4-2002-06-23-yudkowsky-0206-4208-c","subject":"sl4-2002-06-23-yudkowsky-0206-4208","predicate":"contests_proposition","object":{"kind":"entity","entityId":"earth-not-worth-taking"},"topic":"ideas","time":{"kind":"event","date":"2002-06-23","basis":"publication","note":"Archive author index lists the message as \"Sun Jun 23 2002 - 08:16:52 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a superintelligence would leave us a small share, because sparing Earth would cost it very little."}}}},{"id":"yk-y18-sl4-2006-06-13-yudkowsky-0606-15271-c","subject":"sl4-2006-06-13-yudkowsky-0606-15271","predicate":"contests_proposition","object":{"kind":"entity","entityId":"earth-not-worth-taking"},"topic":"ideas","time":{"kind":"event","date":"2006-06-13","basis":"publication","note":"Archive author index lists the message as \"Tue Jun 13 2006 - 00:34:22 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that humans could trade with a superintelligence for what we need."}}}},{"id":"yk-y18-extropians-1999-08-25-yudkowsky-2783-c","subject":"extropians-1999-08-25-yudkowsky-2783","predicate":"contests_proposition","object":{"kind":"entity","entityId":"earth-not-worth-taking"},"topic":"ideas","time":{"kind":"event","date":"1999-08-25","basis":"publication","note":"Archive author index lists the message as \"1999-08-25\" under \"Eliezer S. Yudkowsky\". Sent header: Wed, 25 Aug 1999 18:18:21 -0500."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that humans could trade with a superintelligence for what we need."}}}},{"id":"yk-y18-extropians-1996-11-22-yudkowsky-3178-b","subject":"extropians-1996-11-22-yudkowsky-3178","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"earth-not-worth-taking"},"topic":"ideas","time":{"kind":"event","date":"1996-11-22","basis":"publication","note":"Archive author index lists the message as \"Fri, 22 Nov 1996 02:36:13 -0600\" under \"Eliezer Yudkowsky\". Sent header: Fri, 22 Nov 1996 02:36:13 -0600. Date line of Yudkowsky's reply inside the echoed exchange: Fri, 22 Nov 1996 02:27:52 -0600; list message sent 22 Nov 1996 02:36:13 -0600."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"About benevolent upgrading of revived people, not about a superintelligence leaving Earth a small share."}}},{"id":"yk-y19-arbital-instrumental-convergence-c","subject":"arbital-instrumental-convergence","predicate":"contests_proposition","object":{"kind":"entity","entityId":"harm-requires-hostility"},"topic":"ideas","time":{"kind":"event","date":"2015-07-16","basis":"publication","note":"Viewer shows \"16 Jul 2015\", probably the creation date; the body says \"As of 2016\" and uses later terminology, so at least part of the cached wording postdates 2015."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","attested":{"note":"Arbital pages could be edited after creation; the viewer shows a page date, and the quoted wording is known only from the page as retrieved in 2026."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that an AI would harm humans only out of hostility."}}}},{"id":"yk-y19-ai-positive-negative-factor-global-risk-c","subject":"ai-positive-negative-factor-global-risk","predicate":"contests_proposition","object":{"kind":"entity","entityId":"harm-requires-hostility"},"topic":"ideas","time":{"kind":"event","date":"2006-02-05","note":"Date of the SIAI draft (\"Draft of 5 February, 2006\"), whose capture of 2006-07-08 contains the quoted wording; the chapter was published in 2008. As for the human-thought collection's records of this chapter."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that an AI would harm humans only out of hostility."}}}},{"id":"yk-y19-iem-2013-c","subject":"iem-2013","predicate":"contests_proposition","object":{"kind":"entity","entityId":"harm-requires-hostility"},"topic":"ideas","time":{"kind":"event","date":"2013-09-13","basis":"publication","note":"Report states \"Last modified September 13, 2013\"; first released April 2013."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that an AI would harm humans only out of hostility."}}}},{"id":"yk-y19-time-2023-shut-it-all-down-c","subject":"time-2023-shut-it-all-down","predicate":"contests_proposition","object":{"kind":"entity","entityId":"harm-requires-hostility"},"topic":"ideas","time":{"kind":"event","date":"2023-03-29","basis":"publication","note":"Page metadata datePublished 2023-03-29T22:01:58Z. The page title currently reads \"The Only Way to Deal With the Threat From AI? Shut It Down\"; the headline is the one crossposted to LessWrong on 2023-04-08."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that an AI would harm humans only out of hostility."}}}},{"id":"yk-y19-sl4-2002-06-23-yudkowsky-0206-4208-c","subject":"sl4-2002-06-23-yudkowsky-0206-4208","predicate":"contests_proposition","object":{"kind":"entity","entityId":"harm-requires-hostility"},"topic":"ideas","time":{"kind":"event","date":"2002-06-23","basis":"publication","note":"Archive author index lists the message as \"Sun Jun 23 2002 - 08:16:52 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that an AI would harm humans only out of hostility, even when read with its message or dialogue turn."}}}},{"id":"yk-y19-sl4-2005-07-13-yudkowsky-0507-11483-c","subject":"sl4-2005-07-13-yudkowsky-0507-11483","predicate":"contests_proposition","object":{"kind":"entity","entityId":"harm-requires-hostility"},"topic":"ideas","time":{"kind":"event","date":"2005-07-13","basis":"publication","note":"Archive author index lists the message as \"Wed Jul 13 2005 - 21:12:28 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that an AI would harm humans only out of hostility."}}}},{"id":"yk-y19-sl4-2000-10-03-yudkowsky-0010-0125-c","subject":"sl4-2000-10-03-yudkowsky-0010-0125","predicate":"contests_proposition","object":{"kind":"entity","entityId":"harm-requires-hostility"},"topic":"ideas","time":{"kind":"event","date":"2000-10-03","basis":"publication","note":"Archive author index lists the message as \"Tue Oct 03 2000 - 22:53:59 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that an AI would harm humans only out of hostility."}}}},{"id":"yk-y19-extropians-1998-09-07-yudkowsky-2171-c","subject":"extropians-1998-09-07-yudkowsky-2171","predicate":"contests_proposition","object":{"kind":"entity","entityId":"harm-requires-hostility"},"topic":"ideas","time":{"kind":"event","date":"1998-09-07","basis":"publication","note":"Archive author index lists the message as \"1998-09-07\" under \"Eliezer S. Yudkowsky\". Sent header: Mon, 07 Sep 1998 01:06:05 -0500."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that an AI would harm humans only out of hostility."}}}},{"id":"yk-y19-extropians-1996-11-22-yudkowsky-3178-b","subject":"extropians-1996-11-22-yudkowsky-3178","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"harm-requires-hostility"},"topic":"ideas","time":{"kind":"event","date":"1996-11-22","basis":"publication","note":"Archive author index lists the message as \"Fri, 22 Nov 1996 02:36:13 -0600\" under \"Eliezer Yudkowsky\". Sent header: Fri, 22 Nov 1996 02:36:13 -0600. Date line of Yudkowsky's reply inside the echoed exchange: Thu, 21 Nov 1996 19:07:03 -0600; list message sent 22 Nov 1996 02:36:13 -0600."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"It makes harm conditional on a threat and on human life having no meaning; it does not claim that harm requires hostility."}}},{"id":"yk-y20-lw-2007-the-logical-fallacy-of-generalization-from-c","subject":"lw-2007-the-logical-fallacy-of-generalization-from","predicate":"contests_proposition","object":{"kind":"entity","entityId":"risk-is-film-fiction"},"topic":"ideas","time":{"kind":"event","date":"2007-10-16","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that film scenarios are a fair guide to what the risk would look like."}}}},{"id":"yk-y20-creating-friendly-ai-2001-c","subject":"creating-friendly-ai-2001","predicate":"contests_proposition","object":{"kind":"entity","entityId":"risk-is-film-fiction"},"topic":"ideas","time":{"kind":"event","date":"2001-06-15","basis":"publication","note":"Release date from the reprint; an open-commentary version was announced on SL4 on 2001-04-18, so some wording may be earlier than this date. Reprint wording is not authenticated for 2001 unless a capture was compared."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","attested":{"date":"2001-08-10","note":"The quoted wording also appears in the 2001 HTML edition as captured on 2001-08-10; the quotation itself is taken from MIRI’s later reprint."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that the concern is about robots that come to hate or rebel against humans."}}}},{"id":"yk-y20-staring-into-the-singularity-b","subject":"staring-into-the-singularity","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"risk-is-film-fiction"},"topic":"ideas","time":{"kind":"event","date":"1996-11-18","basis":"publication","note":"The page states \"Created: 11/18/1996\"; the essay was announced on the Extropians list on 1996-11-19. As for the human-thought collection's records of this essay."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"It rejects hostile emotions arising in AI in order to reassure; it does not argue that the risk is film fiction or that films are a fair guide.","attested":{"date":"2000-12-04","note":"The quoted wording is that of version 1.2.1 (\"Updated: 08/31/2000\") as captured on 2000-12-04; no capture of the 1996–1999 page holds the text."}}}},{"id":"yk-y20-ai-positive-negative-factor-global-risk-b","subject":"ai-positive-negative-factor-global-risk","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"risk-is-film-fiction"},"topic":"ideas","time":{"kind":"event","date":"2006-02-05","note":"Date of the SIAI draft (\"Draft of 5 February, 2006\"), whose capture of 2006-07-08 contains the quoted wording; the chapter was published in 2008. As for the human-thought collection's records of this chapter."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"It rejects storybook predictions, good or bad; it does not address hatred or rebellion as the concern."}}},{"id":"yk-y20-sl4-2001-04-19-yudkowsky-0104-1166-c","subject":"sl4-2001-04-19-yudkowsky-0104-1166","predicate":"contests_proposition","object":{"kind":"entity","entityId":"risk-is-film-fiction"},"topic":"ideas","time":{"kind":"event","date":"2001-04-19","basis":"publication","note":"Archive author index lists the message as \"Thu Apr 19 2001 - 14:41:14 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that the concern is about robots that come to hate or rebel against humans, even when read with its message or dialogue turn."}}}},{"id":"yk-y20-extropians-1999-08-04-yudkowsky-1591-c","subject":"extropians-1999-08-04-yudkowsky-1591","predicate":"contests_proposition","object":{"kind":"entity","entityId":"risk-is-film-fiction"},"topic":"ideas","time":{"kind":"event","date":"1999-08-04","basis":"publication","note":"Archive author index lists the message as \"1999-08-04\" under \"Eliezer S. Yudkowsky\". Sent header: Wed, 04 Aug 1999 19:07:13 -0500."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that the concern is about robots that come to hate or rebel against humans."}}}},{"id":"yk-y20-extropians-1997-09-25-yudkowsky-4400-b","subject":"extropians-1997-09-25-yudkowsky-4400","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"risk-is-film-fiction"},"topic":"ideas","time":{"kind":"event","date":"1997-09-25","basis":"publication","note":"Archive author index lists the message as \"Thu, 25 Sep 1997 20:51:50 -0500\" under \"Eliezer S. Yudkowsky\". Sent header: Thu, 25 Sep 1997 20:51:50 -0500."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"It explains fear of a Singularity as fiction-fed xenophobia while conceding a risk; it does not hold that films are a fair guide to the risk."}}},{"id":"yk-y21-lw-2023-gpts-are-predictors-not-imitators-c","subject":"lw-2023-gpts-are-predictors-not-imitators","predicate":"contests_proposition","object":{"kind":"entity","entityId":"human-data-humanlike-mind"},"topic":"ideas","time":{"kind":"event","date":"2023-04-08","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that training on human text or behaviour yields human-like thought and motives."}}}},{"id":"yk-y21-lw-2018-challenges-to-christianos-capability-amplification-proposal-c","subject":"lw-2018-challenges-to-christianos-capability-amplification-proposal","predicate":"contests_proposition","object":{"kind":"entity","entityId":"human-data-humanlike-mind"},"topic":"ideas","time":{"kind":"event","date":"2018-05-19","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","claimed":{"date":"2018-03-16","note":"The post describes itself as \"a basically unedited summary I wrote up on March 16\"; published 2018-05-19."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that training on human text or behaviour yields human-like thought and motives."}}}},{"id":"yk-y21-agi-ruin-2022-c","subject":"agi-ruin-2022","predicate":"contests_proposition","object":{"kind":"entity","entityId":"human-data-humanlike-mind"},"topic":"ideas","time":{"kind":"event","date":"2022-06-05","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that training on human text or behaviour yields human-like thought and motives, even when read with its message or dialogue turn."}}}},{"id":"yk-y21-lw-2025-re-recent-anthropic-safety-research-c","subject":"lw-2025-re-recent-anthropic-safety-research","predicate":"contests_proposition","object":{"kind":"entity","entityId":"human-data-humanlike-mind"},"topic":"ideas","time":{"kind":"event","date":"2025-08-06","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","claimed":{"date":"2025-08-04","note":"Decoded from the linked X status identifier; the X original was not retrieved and the date is unverified (cross-family check, 2026-10-07)."},"engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that training on human text or behaviour yields human-like thought and motives, even when read with its message or dialogue turn."}}}},{"id":"yk-y21-sl4-2002-03-09-yudkowsky-0203-3148-c","subject":"sl4-2002-03-09-yudkowsky-0203-3148","predicate":"contests_proposition","object":{"kind":"entity","entityId":"human-data-humanlike-mind"},"topic":"ideas","time":{"kind":"event","date":"2002-03-09","basis":"publication","note":"Archive author index lists the message as \"Sat Mar 09 2002 - 12:56:21 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that training on human text or behaviour yields human-like thought and motives."}}}},{"id":"yk-y21-sl4-2002-06-23-yudkowsky-0206-4219-c","subject":"sl4-2002-06-23-yudkowsky-0206-4219","predicate":"contests_proposition","object":{"kind":"entity","entityId":"human-data-humanlike-mind"},"topic":"ideas","time":{"kind":"event","date":"2002-06-23","basis":"publication","note":"Archive author index lists the message as \"Sun Jun 23 2002 - 11:56:42 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that training on human text or behaviour yields human-like thought and motives."}}}},{"id":"yk-y21-extropians-1999-08-04-yudkowsky-1589-c","subject":"extropians-1999-08-04-yudkowsky-1589","predicate":"contests_proposition","object":{"kind":"entity","entityId":"human-data-humanlike-mind"},"topic":"ideas","time":{"kind":"event","date":"1999-08-04","basis":"publication","note":"Archive author index lists the message as \"1999-08-04\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that training on human text or behaviour yields human-like thought and motives."}}}},{"id":"yk-y22-staring-into-the-singularity-a","subject":"staring-into-the-singularity","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"build-superintelligence-fast"},"topic":"ideas","time":{"kind":"event","date":"1996-11-18","basis":"publication","note":"The page states \"Created: 11/18/1996\"; the essay was announced on the Extropians list on 1996-11-19. As for the human-thought collection's records of this essay."},"attributes":{"thought":{"roles":["earliest-located"],"register":"nonfiction","attested":{"date":"1996-11-19","note":"The slogan is attested by Yudkowsky's announcement of the essay on the Extropians list, sent 1996-11-19, which the dataset already records as a source for this essay. A longer passage of the essay is known only from version 1.2.1, captured on 2000-12-04. The quoted wording is that of version 1.2.1 (\"Updated: 08/31/2000\") as captured on 2000-12-04; no capture of the 1996–1999 page holds the text."}}}},{"id":"yk-y22-lw-2008-fighting-a-rearguard-action-against-the-b","subject":"lw-2008-fighting-a-rearguard-action-against-the","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"build-superintelligence-fast"},"topic":"ideas","time":{"kind":"event","date":"2008-09-24","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["articulation"],"register":"nonfiction","retrospective":{"from":"1996","until":"2000","note":"Reports the strategy of “Eliezer1996-2000”."}}}},{"id":"yk-y22-lw-2008-the-magnitude-of-his-own-folly-c","subject":"lw-2008-the-magnitude-of-his-own-folly","predicate":"contests_proposition","object":{"kind":"entity","entityId":"build-superintelligence-fast"},"topic":"ideas","time":{"kind":"event","date":"2008-09-30","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that reaching superintelligence is good by default, so speed is what matters."},"retrospective":{"note":"Reports his earlier plan to proceed at full speed."}}}},{"id":"yk-y22-time-2023-shut-it-all-down-c","subject":"time-2023-shut-it-all-down","predicate":"contests_proposition","object":{"kind":"entity","entityId":"build-superintelligence-fast"},"topic":"ideas","time":{"kind":"event","date":"2023-03-29","basis":"publication","note":"Page metadata datePublished 2023-03-29T22:01:58Z. The page title currently reads \"The Only Way to Deal With the Threat From AI? Shut It Down\"; the headline is the one crossposted to LessWrong on 2023-04-08."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that reaching superintelligence is good by default, so speed is what matters, even when read with its message or dialogue turn."}}}},{"id":"yk-y22-sl4-2000-11-13-yudkowsky-0011-0181-c","subject":"sl4-2000-11-13-yudkowsky-0011-0181","predicate":"contests_proposition","object":{"kind":"entity","entityId":"build-superintelligence-fast"},"topic":"ideas","time":{"kind":"event","date":"2000-11-13","basis":"publication","note":"Archive author index lists the message as \"Mon Nov 13 2000 - 22:45:54 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that reaching superintelligence is good by default, so speed is what matters."}}}},{"id":"yk-y22-sl4-2001-08-01-yudkowsky-0108-2031-c","subject":"sl4-2001-08-01-yudkowsky-0108-2031","predicate":"contests_proposition","object":{"kind":"entity","entityId":"build-superintelligence-fast"},"topic":"ideas","time":{"kind":"event","date":"2001-08-01","basis":"publication","note":"Archive author index lists the message as \"Wed Aug 01 2001 - 19:08:09 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that reaching superintelligence is good by default, so speed is what matters."}}}},{"id":"yk-y22-extropians-1996-11-22-yudkowsky-3178-a","subject":"extropians-1996-11-22-yudkowsky-3178","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"build-superintelligence-fast"},"topic":"ideas","time":{"kind":"event","date":"1996-11-22","basis":"publication","note":"Archive author index lists the message as \"Fri, 22 Nov 1996 02:36:13 -0600\" under \"Eliezer Yudkowsky\". Date of his reply inside a private exchange he forwarded to the list on 1996-11-22."},"attributes":{"thought":{"roles":["articulation"],"register":"nonfiction"}}},{"id":"yk-y24-time-2023-shut-it-all-down-c","subject":"time-2023-shut-it-all-down","predicate":"contests_proposition","object":{"kind":"entity","entityId":"build-it-before-rivals"},"topic":"ideas","time":{"kind":"event","date":"2023-03-29","basis":"publication","note":"Page metadata datePublished 2023-03-29T22:01:58Z. The page title currently reads \"The Only Way to Deal With the Threat From AI? Shut It Down\"; the headline is the one crossposted to LessWrong on 2023-04-08."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that racing to be first improves the outcome, and against the premise that whoever builds it controls it."}}}},{"id":"yk-y24-lw-2021-discussion-with-eliezer-yudkowsky-on-agi-c","subject":"lw-2021-discussion-with-eliezer-yudkowsky-on-agi","predicate":"contests_proposition","object":{"kind":"entity","entityId":"build-it-before-rivals"},"topic":"ideas","time":{"kind":"event","date":"2021-11-11","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","claimed":{"date":"2021-09","note":"The post states the conversation took place in early September 2021; published 2021-11-11."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that racing to be first improves the outcome."}}}},{"id":"yk-y24-ai-positive-negative-factor-global-risk-b","subject":"ai-positive-negative-factor-global-risk","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"build-it-before-rivals"},"topic":"ideas","time":{"kind":"event","date":"2006-02-05","note":"Date of the SIAI draft (\"Draft of 5 February, 2006\"), whose capture of 2006-07-08 contains the quoted wording; the chapter was published in 2008. As for the human-thought collection's records of this chapter."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"A first-mover view about whether the first AI is Friendly; it does not claim that a builder or nation controls the AI or that racing improves the outcome."}}},{"id":"yk-y24-sl4-2001-02-26-yudkowsky-0102-0603-b","subject":"sl4-2001-02-26-yudkowsky-0102-0603","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"build-it-before-rivals"},"topic":"ideas","time":{"kind":"event","date":"2001-02-26","basis":"publication","note":"Archive author index lists the message as \"Mon Feb 26 2001 - 00:51:15 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"About national military use and escalation of AI, not about a race to build the first superintelligence."}}},{"id":"yk-y24-sl4-2002-03-10-yudkowsky-0203-3160-b","subject":"sl4-2002-03-10-yudkowsky-0203-3160","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"build-it-before-rivals"},"topic":"ideas","time":{"kind":"event","date":"2002-03-10","basis":"publication","note":"Archive author index lists the message as \"Sun Mar 10 2002 - 18:33:00 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"Being first matters here only through whether the builder understands Friendliness; it does not claim that the builder controls the result."}}},{"id":"yk-y24-sl4-2002-05-03-yudkowsky-0205-3551-a","subject":"sl4-2002-05-03-yudkowsky-0205-3551","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"build-it-before-rivals"},"topic":"ideas","time":{"kind":"event","date":"2002-05-03","basis":"publication","note":"Archive author index lists the message as \"Fri May 03 2002 - 16:38:46 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"Says the only thing to do about a project designing Friendliness for propaganda is to try to beat it to the punch; it does not state that the builder controls the outcome or that racing to be first improves it."}}},{"id":"yk-y24-sl4-2007-11-24-yudkowsky-0711-17248-c","subject":"sl4-2007-11-24-yudkowsky-0711-17248","predicate":"contests_proposition","object":{"kind":"entity","entityId":"build-it-before-rivals"},"topic":"ideas","time":{"kind":"event","date":"2007-11-24","basis":"publication","note":"Archive author index lists the message as \"Sat Nov 24 2007 - 17:36:29 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that racing to be first improves the outcome."}}}},{"id":"yk-y24-extropians-2000-09-29-yudkowsky-6080-a","subject":"extropians-2000-09-29-yudkowsky-6080","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"build-it-before-rivals"},"topic":"ideas","time":{"kind":"event","date":"2000-09-29","basis":"publication","note":"Archive author index lists the message as \"Fri Sep 29 2000 - 11:43:19 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"Seeks to build AI first while writing up Friendly AI and rejecting coercion; it does not state that the builder controls the outcome or that racing to be first improves it."}}},{"id":"yk-y25-creating-friendly-ai-2001-b","subject":"creating-friendly-ai-2001","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"interpretability-reads-its-mind"},"topic":"ideas","time":{"kind":"event","date":"2001-06-15","basis":"publication","note":"Release date from the reprint; an open-commentary version was announced on SL4 on 2001-04-18, so some wording may be earlier than this date. Reprint wording is not authenticated for 2001 unless a capture was compared."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"A worst-case design assumption that an AI could hide unfriendly cognition; it does not address whether reading an AI's thoughts makes it safe.","attested":{"note":"Known only from MIRI's later, undated PDF reprint; the wording was not found in the 2001 HTML pages compared."}}}},{"id":"yk-y25-agi-ruin-2022-c","subject":"agi-ruin-2022","predicate":"contests_proposition","object":{"kind":"entity","entityId":"interpretability-reads-its-mind"},"topic":"ideas","time":{"kind":"event","date":"2022-06-05","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that we can see what an AI system is thinking."}}}},{"id":"yk-y25-sl4-2000-07-20-yudkowsky-0007-0023-a","subject":"sl4-2000-07-20-yudkowsky-0007-0023","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"interpretability-reads-its-mind"},"topic":"ideas","time":{"kind":"event","date":"2000-07-20","basis":"publication","note":"Archive author index lists the message as \"Thu Jul 20 2000 - 22:10:13 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["earliest-located"],"register":"nonfiction"}}},{"id":"yk-y25-sl4-2003-05-22-yudkowsky-0305-6783-c","subject":"sl4-2003-05-22-yudkowsky-0305-6783","predicate":"contests_proposition","object":{"kind":"entity","entityId":"interpretability-reads-its-mind"},"topic":"ideas","time":{"kind":"event","date":"2003-05-22","basis":"publication","note":"Archive author index lists the message as \"Thu May 22 2003 - 02:42:16 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that seeing what it thinks is enough to make it safe."}}}},{"id":"yk-y26-ai-positive-negative-factor-global-risk-c","subject":"ai-positive-negative-factor-global-risk","predicate":"contests_proposition","object":{"kind":"entity","entityId":"no-body-no-danger"},"topic":"ideas","time":{"kind":"event","date":"2006-02-05","note":"Date of the SIAI draft (\"Draft of 5 February, 2006\"), whose capture of 2006-07-08 contains the quoted wording; the chapter was published in 2008. As for the human-thought collection's records of this chapter."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that intelligence without physical actuators of its own is not dangerous, and against the premise that to act in the physical world it would need human help, which people would not give."}}}},{"id":"yk-y26-agi-ruin-2022-c","subject":"agi-ruin-2022","predicate":"contests_proposition","object":{"kind":"entity","entityId":"no-body-no-danger"},"topic":"ideas","time":{"kind":"event","date":"2022-06-05","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that intelligence without physical actuators of its own is not dangerous."}}}},{"id":"yk-y26-time-2023-shut-it-all-down-c","subject":"time-2023-shut-it-all-down","predicate":"contests_proposition","object":{"kind":"entity","entityId":"no-body-no-danger"},"topic":"ideas","time":{"kind":"event","date":"2023-03-29","basis":"publication","note":"Page metadata datePublished 2023-03-29T22:01:58Z. The page title currently reads \"The Only Way to Deal With the Threat From AI? Shut It Down\"; the headline is the one crossposted to LessWrong on 2023-04-08."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that intelligence without physical actuators of its own is not dangerous."}}}},{"id":"yk-y26-sl4-2000-09-16-yudkowsky-0009-0086-c","subject":"sl4-2000-09-16-yudkowsky-0009-0086","predicate":"contests_proposition","object":{"kind":"entity","entityId":"no-body-no-danger"},"topic":"ideas","time":{"kind":"event","date":"2000-09-16","basis":"publication","note":"Archive author index lists the message as \"Sat Sep 16 2000 - 18:04:07 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that to act in the physical world it would need human help, which people would not give."}}}},{"id":"yk-y26-extropians-1996-12-31-yudkowsky-5142-c","subject":"extropians-1996-12-31-yudkowsky-5142","predicate":"contests_proposition","object":{"kind":"entity","entityId":"no-body-no-danger"},"topic":"ideas","time":{"kind":"event","date":"1996-12-31","basis":"publication","note":"Archive author index lists the message as \"Tue, 31 Dec 1996 10:40:10 -0600\" under \"Eliezer Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that to act in the physical world it would need human help, which people would not give."}}}},{"id":"yk-y26-extropians-1997-09-26-yudkowsky-4420-b","subject":"extropians-1997-09-26-yudkowsky-4420","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"no-body-no-danger"},"topic":"ideas","time":{"kind":"event","date":"1997-09-26","basis":"publication","note":"Archive author index lists the message as \"Fri, 26 Sep 1997 18:20:36 -0500\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"A passing list item that protein synthesis machines may be reachable over the Internet; it does not argue the point."}}},{"id":"yk-y26-extropians-2000-09-05-yudkowsky-4292-c","subject":"extropians-2000-09-05-yudkowsky-4292","predicate":"contests_proposition","object":{"kind":"entity","entityId":"no-body-no-danger"},"topic":"ideas","time":{"kind":"event","date":"2000-09-05","basis":"publication","note":"Archive author index lists the message as \"Tue Sep 05 2000 - 15:55:19 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that intelligence without physical actuators of its own is not dangerous."}}}},{"id":"yk-y27-time-2023-shut-it-all-down-c","subject":"time-2023-shut-it-all-down","predicate":"contests_proposition","object":{"kind":"entity","entityId":"experiment-and-see"},"topic":"ideas","time":{"kind":"event","date":"2023-03-29","basis":"publication","note":"Page metadata datePublished 2023-03-29T22:01:58Z. The page title currently reads \"The Only Way to Deal With the Threat From AI? Shut It Down\"; the headline is the one crossposted to LessWrong on 2023-04-08."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that alignment can be learned by trial and error at the frontier."}}}},{"id":"yk-y27-creating-friendly-ai-2001-b","subject":"creating-friendly-ai-2001","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"experiment-and-see"},"topic":"ideas","time":{"kind":"event","date":"2001-06-15","basis":"publication","note":"Release date from the reprint; an open-commentary version was announced on SL4 on 2001-04-18, so some wording may be earlier than this date. Reprint wording is not authenticated for 2001 unless a capture was compared."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"A conservative design assumption in a table, not a forecast about learning alignment by trial and error.","attested":{"date":"2001-06-26","note":"The quoted wording also appears in the 2001 HTML edition as captured on 2001-06-26; the quotation itself is taken from MIRI’s later reprint."}}}},{"id":"yk-y27-agi-ruin-2022-c","subject":"agi-ruin-2022","predicate":"contests_proposition","object":{"kind":"entity","entityId":"experiment-and-see"},"topic":"ideas","time":{"kind":"event","date":"2022-06-05","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that results from safe experiments on weaker systems will carry over to dangerous ones."}}}},{"id":"yk-y27-lw-2026-irretrievability-or-murphys-curse-of-oneshotness-c","subject":"lw-2026-irretrievability-or-murphys-curse-of-oneshotness","predicate":"contests_proposition","object":{"kind":"entity","entityId":"experiment-and-see"},"topic":"ideas","time":{"kind":"event","date":"2026-05-04","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that results from safe experiments on weaker systems will carry over to dangerous ones."}}}},{"id":"yk-y27-sl4-2000-09-17-yudkowsky-0009-0092-c","subject":"sl4-2000-09-17-yudkowsky-0009-0092","predicate":"contests_proposition","object":{"kind":"entity","entityId":"experiment-and-see"},"topic":"ideas","time":{"kind":"event","date":"2000-09-17","basis":"publication","note":"Archive author index lists the message as \"Sun Sep 17 2000 - 10:19:16 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that alignment can be learned by trial and error at the frontier."}}}},{"id":"yk-y28-agi-ruin-2022-c","subject":"agi-ruin-2022","predicate":"contests_proposition","object":{"kind":"entity","entityId":"developers-know-what-they-are-doing"},"topic":"ideas","time":{"kind":"event","date":"2022-06-05","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that a credible safety plan exists, even when read with its message or dialogue turn."}}}},{"id":"yk-y28-ai-positive-negative-factor-global-risk-c","subject":"ai-positive-negative-factor-global-risk","predicate":"contests_proposition","object":{"kind":"entity","entityId":"developers-know-what-they-are-doing"},"topic":"ideas","time":{"kind":"event","date":"2006-02-05","note":"Date of the SIAI draft (\"Draft of 5 February, 2006\"), whose capture of 2006-07-08 contains the quoted wording; the chapter was published in 2008. As for the human-thought collection's records of this chapter."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a credible safety plan exists."}}}},{"id":"yk-y28-lw-2022-six-dimensions-of-operational-adequacy-in-c","subject":"lw-2022-six-dimensions-of-operational-adequacy-in","predicate":"contests_proposition","object":{"kind":"entity","entityId":"developers-know-what-they-are-doing"},"topic":"ideas","time":{"kind":"event","date":"2022-05-30","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","claimed":{"date":"2017-11","note":"Editor's note says the document was written in November 2017; the wording is attested only in the lightly edited copy published 2022-05-30."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the post the passage belongs to gives a reason against the premise that a credible safety plan exists by diagnosing inadequate alignment mindset and safety approaches at named leading projects."}}}},{"id":"yk-y28-creating-friendly-ai-2001-a","subject":"creating-friendly-ai-2001","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"developers-know-what-they-are-doing"},"topic":"ideas","time":{"kind":"event","date":"2001-06-15","basis":"publication","note":"Release date from the reprint; an open-commentary version was announced on SL4 on 2001-04-18, so some wording may be earlier than this date. Reprint wording is not authenticated for 2001 unless a capture was compared."},"attributes":{"thought":{"roles":["earliest-located"],"register":"nonfiction","attested":{"date":"2001-06-17","note":"The quoted wording also appears in the 2001 HTML edition as captured on 2001-06-17; the quotation itself is taken from MIRI’s later reprint."}}}},{"id":"yk-y28-lw-2008-above-average-ai-scientists-b","subject":"lw-2008-above-average-ai-scientists","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"developers-know-what-they-are-doing"},"topic":"ideas","time":{"kind":"event","date":"2008-09-28","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"About the absence of any entry bar to starting an AGI project, not about the competence of the most advanced projects.","retrospective":{"note":"Reports an undated meeting after which he stopped assuming that AGI researchers met the standard the problem required."}}}},{"id":"yk-y28-sl4-2001-08-01-yudkowsky-0108-2031-c","subject":"sl4-2001-08-01-yudkowsky-0108-2031","predicate":"contests_proposition","object":{"kind":"entity","entityId":"developers-know-what-they-are-doing"},"topic":"ideas","time":{"kind":"event","date":"2001-08-01","basis":"publication","note":"Archive author index lists the message as \"Wed Aug 01 2001 - 19:08:09 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that the most advanced projects are the most competent to handle safety."}}}},{"id":"yk-y28-sl4-2006-02-15-yudkowsky-0602-14096-c","subject":"sl4-2006-02-15-yudkowsky-0602-14096","predicate":"contests_proposition","object":{"kind":"entity","entityId":"developers-know-what-they-are-doing"},"topic":"ideas","time":{"kind":"event","date":"2006-02-15","basis":"publication","note":"Archive author index lists the message as \"Wed Feb 15 2006 - 10:47:28 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that the most advanced projects are the most competent to handle safety."}}}},{"id":"yk-y28-extropians-2000-09-29-yudkowsky-6073-b","subject":"extropians-2000-09-29-yudkowsky-6073","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"developers-know-what-they-are-doing"},"topic":"ideas","time":{"kind":"event","date":"2000-09-29","basis":"publication","note":"Archive author index lists the message as \"Fri Sep 29 2000 - 10:50:28 MDT\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"Deference conditional on builders who have worked hard on Friendliness; it does not claim that the leading projects in general can handle safety."}}},{"id":"yk-y29-agi-ruin-2022-c","subject":"agi-ruin-2022","predicate":"contests_proposition","object":{"kind":"entity","entityId":"many-ais-balance"},"topic":"ideas","time":{"kind":"event","date":"2022-06-05","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that multiple superintelligences will balance each other in humans’ favour."}}}},{"id":"yk-y29-ai-positive-negative-factor-global-risk-c","subject":"ai-positive-negative-factor-global-risk","predicate":"contests_proposition","object":{"kind":"entity","entityId":"many-ais-balance"},"topic":"ideas","time":{"kind":"event","date":"2006-02-05","note":"Date of the SIAI draft (\"Draft of 5 February, 2006\"), whose capture of 2006-07-08 contains the quoted wording; the chapter was published in 2008. As for the human-thought collection's records of this chapter."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that multiple superintelligences will balance each other in humans’ favour."}}}},{"id":"yk-y29-lw-2026-irretrievability-or-murphys-curse-of-oneshotness-c","subject":"lw-2026-irretrievability-or-murphys-curse-of-oneshotness","predicate":"contests_proposition","object":{"kind":"entity","entityId":"many-ais-balance"},"topic":"ideas","time":{"kind":"event","date":"2026-05-04","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that multiple superintelligences will balance each other in humans’ favour."}}}},{"id":"yk-y29-sl4-2001-02-04-yudkowsky-0102-0539-c","subject":"sl4-2001-02-04-yudkowsky-0102-0539","predicate":"contests_proposition","object":{"kind":"entity","entityId":"many-ais-balance"},"topic":"ideas","time":{"kind":"event","date":"2001-02-04","basis":"publication","note":"Archive author index lists the message as \"Sun Feb 04 2001 - 21:48:41 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that multiple superintelligences will balance each other in humans’ favour."}}}},{"id":"yk-y29-sl4-2001-02-05-yudkowsky-0102-0542-c","subject":"sl4-2001-02-05-yudkowsky-0102-0542","predicate":"contests_proposition","object":{"kind":"entity","entityId":"many-ais-balance"},"topic":"ideas","time":{"kind":"event","date":"2001-02-05","basis":"publication","note":"Archive author index lists the message as \"Mon Feb 05 2001 - 10:57:20 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that multiple superintelligences will balance each other in humans’ favour."}}}},{"id":"yk-y29-sl4-2004-05-30-yudkowsky-0405-8843-c","subject":"sl4-2004-05-30-yudkowsky-0405-8843","predicate":"contests_proposition","object":{"kind":"entity","entityId":"many-ais-balance"},"topic":"ideas","time":{"kind":"event","date":"2004-05-30","basis":"publication","note":"Archive author index lists the message as \"Sun May 30 2004 - 01:39:50 MDT\" under \"Eliezer Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that multiple superintelligences will balance each other in humans’ favour."}}}},{"id":"yk-y29-extropians-2000-03-16-yudkowsky-5105-c","subject":"extropians-2000-03-16-yudkowsky-5105","predicate":"contests_proposition","object":{"kind":"entity","entityId":"many-ais-balance"},"topic":"ideas","time":{"kind":"event","date":"2000-03-16","basis":"publication","note":"Archive author index lists the message as \"Thu Mar 16 2000 - 16:43:29 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that multiple superintelligences will balance each other in humans’ favour."}}}},{"id":"yk-y30-time-2023-shut-it-all-down-c","subject":"time-2023-shut-it-all-down","predicate":"contests_proposition","object":{"kind":"entity","entityId":"default-outcome-good"},"topic":"ideas","time":{"kind":"event","date":"2023-03-29","basis":"publication","note":"Page metadata datePublished 2023-03-29T22:01:58Z. The page title currently reads \"The Only Way to Deal With the Threat From AI? Shut It Down\"; the headline is the one crossposted to LessWrong on 2023-04-08."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that the default outcome is good."}}}},{"id":"yk-y30-staring-into-the-singularity-b","subject":"staring-into-the-singularity","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"default-outcome-good"},"topic":"ideas","time":{"kind":"event","date":"1996-11-18","basis":"publication","note":"The page states \"Created: 11/18/1996\"; the essay was announced on the Extropians list on 1996-11-19. As for the human-thought collection's records of this essay."},"attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"It denies that hostile extinction is inevitable; it does not assert that the default outcome is good.","attested":{"date":"2000-12-04","note":"The quoted wording is that of version 1.2.1 (\"Updated: 08/31/2000\") as captured on 2000-12-04; no capture of the 1996–1999 page holds the text."}}}},{"id":"yk-y30-lw-2008-that-tiny-note-of-discord-b","subject":"lw-2008-that-tiny-note-of-discord","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"default-outcome-good"},"topic":"ideas","time":{"kind":"event","date":"2008-09-23","basis":"publication","note":"LessWrong postedAt of an imported Overcoming Bias post (Overcoming Bias date)."},"attributes":{"thought":{"roles":["articulation"],"register":"nonfiction","retrospective":{"from":"1997","until":"1997","note":"Reports the belief of “Eliezer1997”."}}}},{"id":"yk-y30-ai-positive-negative-factor-global-risk-c","subject":"ai-positive-negative-factor-global-risk","predicate":"contests_proposition","object":{"kind":"entity","entityId":"default-outcome-good"},"topic":"ideas","time":{"kind":"event","date":"2006-02-05","note":"Date of the SIAI draft (\"Draft of 5 February, 2006\"), whose capture of 2006-07-08 contains the quoted wording; the chapter was published in 2008. As for the human-thought collection's records of this chapter."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that the default outcome is good."}}}},{"id":"yk-y30-lw-2022-miri-announces-new-death-with-dignity-c","subject":"lw-2022-miri-announces-new-death-with-dignity","predicate":"contests_proposition","object":{"kind":"entity","entityId":"default-outcome-good"},"topic":"ideas","time":{"kind":"event","date":"2022-04-02","basis":"publication","note":"LessWrong postedAt."},"attributes":{"thought":{"roles":["contesting","later-restatement"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that the default outcome is good, even when read with its message or dialogue turn."}}}},{"id":"yk-y30-sl4-2000-11-13-yudkowsky-0011-0181-c","subject":"sl4-2000-11-13-yudkowsky-0011-0181","predicate":"contests_proposition","object":{"kind":"entity","entityId":"default-outcome-good"},"topic":"ideas","time":{"kind":"event","date":"2000-11-13","basis":"publication","note":"Archive author index lists the message as \"Mon Nov 13 2000 - 22:45:54 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting","earliest-located"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that the default outcome is good."}}}},{"id":"yk-y30-sl4-2001-02-07-yudkowsky-0102-0545-c","subject":"sl4-2001-02-07-yudkowsky-0102-0545","predicate":"contests_proposition","object":{"kind":"entity","entityId":"default-outcome-good"},"topic":"ideas","time":{"kind":"event","date":"2001-02-07","basis":"publication","note":"Archive author index lists the message as \"Wed Feb 07 2001 - 21:09:16 MST\" under \"Eliezer S. Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that the default outcome is good."}}}},{"id":"yk-y30-sl4-2004-06-01-yudkowsky-0406-8905-c","subject":"sl4-2004-06-01-yudkowsky-0406-8905","predicate":"contests_proposition","object":{"kind":"entity","entityId":"default-outcome-good"},"topic":"ideas","time":{"kind":"event","date":"2004-06-01","basis":"publication","note":"Archive author index lists the message as \"Tue Jun 01 2004 - 06:01:06 MDT\" under \"Eliezer Yudkowsky\"."},"attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that the default outcome is good."}}}},{"id":"yk-y30-extropians-1996-11-22-yudkowsky-3178-a","subject":"extropians-1996-11-22-yudkowsky-3178","predicate":"articulates_proposition","object":{"kind":"entity","entityId":"default-outcome-good"},"topic":"ideas","time":{"kind":"event","date":"1996-11-22","basis":"publication","note":"Archive author index lists the message as \"Fri, 22 Nov 1996 02:36:13 -0600\" under \"Eliezer Yudkowsky\". Date of his reply inside a private exchange he forwarded to the list on 1996-11-22."},"attributes":{"thought":{"roles":["earliest-located"],"register":"nonfiction"}}}],"sources":[{"id":"source-iem","authors":["Eliezer Yudkowsky"],"publisherId":"miri"},{"id":"source-sits-2000-capture","documentId":"staring-into-the-singularity","authors":["Eliezer S. Yudkowsky"]},{"id":"source-extropians-1996-12-31-yudkowsky-5142","documentId":"extropians-1996-12-31-yudkowsky-5142","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1996-12-31-yudkowsky-5159","documentId":"extropians-1996-12-31-yudkowsky-5159","authors":["Eliezer Yudkowsky"]},{"id":"source-aipnfgr-miri-reprint","documentId":"ai-positive-negative-factor-global-risk","authors":["Eliezer Yudkowsky"],"publisherId":"miri"},{"id":"source-yudkowsky-2008-recursive-self-improvement","documentId":"yudkowsky-2008-recursive-self-improvement","authors":["Eliezer Yudkowsky"]},{"id":"source-yudkowsky-2008-hard-takeoff","documentId":"yudkowsky-2008-hard-takeoff","authors":["Eliezer Yudkowsky"]},{"id":"source-arbital-programmer-deception","documentId":"arbital-programmer-deception","authors":["Eliezer Yudkowsky"]},{"id":"source-agi-ruin-2022","documentId":"agi-ruin-2022","authors":["Eliezer Yudkowsky"]},{"id":"source-ai-box-experiment-page","documentId":"ai-box-experiment-page","authors":["Eliezer Yudkowsky"]},{"id":"source-arbital-instrumental-convergence","documentId":"arbital-instrumental-convergence","authors":["Eliezer Yudkowsky"]},{"id":"source-arbital-orthogonality","documentId":"arbital-orthogonality","authors":["Eliezer Yudkowsky"]},{"id":"source-coherent-extrapolated-volition-2004","documentId":"coherent-extrapolated-volition-2004","authors":["Eliezer Yudkowsky"]},{"id":"source-cfai-miri-reprint","documentId":"creating-friendly-ai-2001","authors":["Eliezer Yudkowsky"]},{"id":"source-time-2023-shut-it-all-down","documentId":"time-2023-shut-it-all-down","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-2000-03-14-yudkowsky-4837","documentId":"extropians-2000-03-14-yudkowsky-4837","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-2000-03-16-yudkowsky-5105","documentId":"extropians-2000-03-16-yudkowsky-5105","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1999-01-06-yudkowsky-0160","documentId":"extropians-1999-01-06-yudkowsky-0160","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1999-02-24-yudkowsky-2293","documentId":"extropians-1999-02-24-yudkowsky-2293","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1999-04-08-yudkowsky-0310","documentId":"extropians-1999-04-08-yudkowsky-0310","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-2000-09-05-yudkowsky-4277","documentId":"extropians-2000-09-05-yudkowsky-4277","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-2000-09-05-yudkowsky-4292","documentId":"extropians-2000-09-05-yudkowsky-4292","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-2000-09-29-yudkowsky-6073","documentId":"extropians-2000-09-29-yudkowsky-6073","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-2000-09-29-yudkowsky-6080","documentId":"extropians-2000-09-29-yudkowsky-6080","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1997-09-25-yudkowsky-4400","documentId":"extropians-1997-09-25-yudkowsky-4400","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1997-09-26-yudkowsky-4420","documentId":"extropians-1997-09-26-yudkowsky-4420","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1998-09-07-yudkowsky-2171","documentId":"extropians-1998-09-07-yudkowsky-2171","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1998-09-08-yudkowsky-2235","documentId":"extropians-1998-09-08-yudkowsky-2235","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1998-09-15-yudkowsky-2411","documentId":"extropians-1998-09-15-yudkowsky-2411","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1998-09-18-yudkowsky-2487","documentId":"extropians-1998-09-18-yudkowsky-2487","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1999-08-01-yudkowsky-1386","documentId":"extropians-1999-08-01-yudkowsky-1386","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1999-08-03-yudkowsky-1487","documentId":"extropians-1999-08-03-yudkowsky-1487","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1999-08-04-yudkowsky-1587","documentId":"extropians-1999-08-04-yudkowsky-1587","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1999-08-04-yudkowsky-1589","documentId":"extropians-1999-08-04-yudkowsky-1589","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1999-08-04-yudkowsky-1591","documentId":"extropians-1999-08-04-yudkowsky-1591","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1999-08-25-yudkowsky-2783","documentId":"extropians-1999-08-25-yudkowsky-2783","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1998-12-06-yudkowsky-2153","documentId":"extropians-1998-12-06-yudkowsky-2153","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1999-10-21-yudkowsky-1086","documentId":"extropians-1999-10-21-yudkowsky-1086","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1996-11-13-yudkowsky-2941","documentId":"extropians-1996-11-13-yudkowsky-2941","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1996-11-16-yudkowsky-3042","documentId":"extropians-1996-11-16-yudkowsky-3042","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1996-11-22-yudkowsky-3178","documentId":"extropians-1996-11-22-yudkowsky-3178","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1996-11-22-yudkowsky-3184","documentId":"extropians-1996-11-22-yudkowsky-3184","authors":["Eliezer Yudkowsky"]},{"id":"source-extropians-1996-12-08-yudkowsky-3761","documentId":"extropians-1996-12-08-yudkowsky-3761","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2007-the-hidden-complexity-of-wishes","documentId":"lw-2007-the-hidden-complexity-of-wishes","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2026-only-law-can-prevent-extinction","documentId":"lw-2026-only-law-can-prevent-extinction","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2021-ngo-and-yudkowsky-on-alignment-difficulty","documentId":"lw-2021-ngo-and-yudkowsky-on-alignment-difficulty","authors":["Eliezer Yudkowsky","Richard Ngo"]},{"id":"source-lw-2008-above-average-ai-scientists","documentId":"lw-2008-above-average-ai-scientists","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2017-theres-no-fire-alarm-for-artificial","documentId":"lw-2017-theres-no-fire-alarm-for-artificial","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2021-discussion-with-eliezer-yudkowsky-on-agi","documentId":"lw-2021-discussion-with-eliezer-yudkowsky-on-agi","authors":["Rob Bensinger","Eliezer Yudkowsky"]},{"id":"source-lw-2024-the-sun-is-big-but-superintelligences","documentId":"lw-2024-the-sun-is-big-but-superintelligences","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2009-value-is-fragile","documentId":"lw-2009-value-is-fragile","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2013-after-critical-event-w-happens-they","documentId":"lw-2013-after-critical-event-w-happens-they","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2008-fighting-a-rearguard-action-against-the","documentId":"lw-2008-fighting-a-rearguard-action-against-the","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2008-magical-categories","documentId":"lw-2008-magical-categories","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2008-no-universally-compelling-arguments","documentId":"lw-2008-no-universally-compelling-arguments","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2018-challenges-to-christianos-capability-amplification-proposal","documentId":"lw-2018-challenges-to-christianos-capability-amplification-proposal","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2008-that-tiny-note-of-discord","documentId":"lw-2008-that-tiny-note-of-discord","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2026-cat-belling-problems","documentId":"lw-2026-cat-belling-problems","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2021-biology-inspired-agi-timelines-the-trick","documentId":"lw-2021-biology-inspired-agi-timelines-the-trick","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2026-the-talker-does-not-control-the","documentId":"lw-2026-the-talker-does-not-control-the","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2007-thou-art-godshatter","documentId":"lw-2007-thou-art-godshatter","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2008-ghosts-in-the-machine","documentId":"lw-2008-ghosts-in-the-machine","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2008-the-magnitude-of-his-own-folly","documentId":"lw-2008-the-magnitude-of-his-own-folly","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2026-irretrievability-or-murphys-curse-of-oneshotness","documentId":"lw-2026-irretrievability-or-murphys-curse-of-oneshotness","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2022-miri-announces-new-death-with-dignity","documentId":"lw-2022-miri-announces-new-death-with-dignity","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2022-six-dimensions-of-operational-adequacy-in","documentId":"lw-2022-six-dimensions-of-operational-adequacy-in","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2023-gpts-are-predictors-not-imitators","documentId":"lw-2023-gpts-are-predictors-not-imitators","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2025-re-recent-anthropic-safety-research","documentId":"lw-2025-re-recent-anthropic-safety-research","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2007-the-logical-fallacy-of-generalization-from","documentId":"lw-2007-the-logical-fallacy-of-generalization-from","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2023-alexander-and-yudkowsky-on-agi-goals","documentId":"lw-2023-alexander-and-yudkowsky-on-agi-goals","authors":["Scott Alexander","Eliezer Yudkowsky"]},{"id":"source-lw-2012-reply-to-holden-on-tool-ai","documentId":"lw-2012-reply-to-holden-on-tool-ai","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2008-the-rhythm-of-disagreement","documentId":"lw-2008-the-rhythm-of-disagreement","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2022-shah-and-yudkowsky-on-alignment-failures","documentId":"lw-2022-shah-and-yudkowsky-on-alignment-failures","authors":["Rohin Shah","Eliezer Yudkowsky"]},{"id":"source-lw-2021-yudkowsky-and-christiano-discuss-takeoff-speeds","documentId":"lw-2021-yudkowsky-and-christiano-discuss-takeoff-speeds","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2008-dreams-of-friendliness","documentId":"lw-2008-dreams-of-friendliness","authors":["Eliezer Yudkowsky"]},{"id":"source-lw-2008-what-i-think-if-not-why","documentId":"lw-2008-what-i-think-if-not-why","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2000-07-09-yudkowsky-0007-0017","documentId":"sl4-2000-07-09-yudkowsky-0007-0017","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2000-07-20-yudkowsky-0007-0023","documentId":"sl4-2000-07-20-yudkowsky-0007-0023","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2000-08-21-yudkowsky-0008-0046","documentId":"sl4-2000-08-21-yudkowsky-0008-0046","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2000-09-16-yudkowsky-0009-0086","documentId":"sl4-2000-09-16-yudkowsky-0009-0086","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2000-09-17-yudkowsky-0009-0092","documentId":"sl4-2000-09-17-yudkowsky-0009-0092","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2000-10-03-yudkowsky-0010-0125","documentId":"sl4-2000-10-03-yudkowsky-0010-0125","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2000-11-13-yudkowsky-0011-0181","documentId":"sl4-2000-11-13-yudkowsky-0011-0181","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2000-11-18-yudkowsky-0011-0240","documentId":"sl4-2000-11-18-yudkowsky-0011-0240","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2000-12-10-yudkowsky-0012-0355","documentId":"sl4-2000-12-10-yudkowsky-0012-0355","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2000-12-12-yudkowsky-0012-0381","documentId":"sl4-2000-12-12-yudkowsky-0012-0381","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2001-01-24-yudkowsky-0101-0452","documentId":"sl4-2001-01-24-yudkowsky-0101-0452","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2001-02-04-yudkowsky-0102-0539","documentId":"sl4-2001-02-04-yudkowsky-0102-0539","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2001-02-05-yudkowsky-0102-0542","documentId":"sl4-2001-02-05-yudkowsky-0102-0542","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2001-02-07-yudkowsky-0102-0545","documentId":"sl4-2001-02-07-yudkowsky-0102-0545","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2001-02-25-yudkowsky-0102-0594","documentId":"sl4-2001-02-25-yudkowsky-0102-0594","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2001-02-26-yudkowsky-0102-0603","documentId":"sl4-2001-02-26-yudkowsky-0102-0603","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2001-04-06-yudkowsky-0104-0956","documentId":"sl4-2001-04-06-yudkowsky-0104-0956","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2001-04-08-yudkowsky-0104-1019","documentId":"sl4-2001-04-08-yudkowsky-0104-1019","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2001-04-12-yudkowsky-0104-1076","documentId":"sl4-2001-04-12-yudkowsky-0104-1076","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2001-04-19-yudkowsky-0104-1166","documentId":"sl4-2001-04-19-yudkowsky-0104-1166","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2001-08-01-yudkowsky-0108-2031","documentId":"sl4-2001-08-01-yudkowsky-0108-2031","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2002-03-09-yudkowsky-0203-3148","documentId":"sl4-2002-03-09-yudkowsky-0203-3148","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2002-03-10-yudkowsky-0203-3160","documentId":"sl4-2002-03-10-yudkowsky-0203-3160","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2002-05-03-yudkowsky-0205-3551","documentId":"sl4-2002-05-03-yudkowsky-0205-3551","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2002-06-23-yudkowsky-0206-4208","documentId":"sl4-2002-06-23-yudkowsky-0206-4208","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2002-06-23-yudkowsky-0206-4219","documentId":"sl4-2002-06-23-yudkowsky-0206-4219","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2002-06-27-yudkowsky-0206-4453","documentId":"sl4-2002-06-27-yudkowsky-0206-4453","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2002-07-02-yudkowsky-0207-4667","documentId":"sl4-2002-07-02-yudkowsky-0207-4667","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2002-09-10-yudkowsky-0209-5398","documentId":"sl4-2002-09-10-yudkowsky-0209-5398","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2002-09-19-yudkowsky-0209-5571","documentId":"sl4-2002-09-19-yudkowsky-0209-5571","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2003-05-22-yudkowsky-0305-6783","documentId":"sl4-2003-05-22-yudkowsky-0305-6783","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2003-05-29-yudkowsky-0305-6846","documentId":"sl4-2003-05-29-yudkowsky-0305-6846","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2004-05-30-yudkowsky-0405-8843","documentId":"sl4-2004-05-30-yudkowsky-0405-8843","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2004-06-01-yudkowsky-0406-8905","documentId":"sl4-2004-06-01-yudkowsky-0406-8905","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2004-12-29-yudkowsky-0412-10488","documentId":"sl4-2004-12-29-yudkowsky-0412-10488","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2005-01-02-yudkowsky-0501-10512","documentId":"sl4-2005-01-02-yudkowsky-0501-10512","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2005-07-13-yudkowsky-0507-11477","documentId":"sl4-2005-07-13-yudkowsky-0507-11477","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2005-07-13-yudkowsky-0507-11483","documentId":"sl4-2005-07-13-yudkowsky-0507-11483","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2005-07-25-yudkowsky-0507-11765","documentId":"sl4-2005-07-25-yudkowsky-0507-11765","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2005-08-09-yudkowsky-0508-11866","documentId":"sl4-2005-08-09-yudkowsky-0508-11866","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2006-02-15-yudkowsky-0602-14096","documentId":"sl4-2006-02-15-yudkowsky-0602-14096","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2006-06-06-yudkowsky-0606-15174","documentId":"sl4-2006-06-06-yudkowsky-0606-15174","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2006-06-13-yudkowsky-0606-15271","documentId":"sl4-2006-06-13-yudkowsky-0606-15271","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2007-11-24-yudkowsky-0711-17248","documentId":"sl4-2007-11-24-yudkowsky-0711-17248","authors":["Eliezer Yudkowsky"]},{"id":"source-sl4-2008-02-11-yudkowsky-0802-17621","documentId":"sl4-2008-02-11-yudkowsky-0802-17621","authors":["Eliezer Yudkowsky"]}],"evidence":[],"at":{"entities":[23,294,295,296,311,314,316,317,325,333,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489],"claims":[984,985,986,987,988,989,990,991,992,993,994,995,996,997,998,999,1000,1001,1002,1003,1004,1005,1006,1007,1008,1009,1010,1011,1012,1013,1014,1015,1016,1017,1018,1019,1020,1021,1022,1023,1024,1025,1026,1027,1028,1029,1030,1031,1032,1033,1034,1035,1036,1037,1038,1039,1040,1041,1042,1043,1044,1045,1046,1047,1048,1049,1050,1051,1052,1053,1054,1055,1056,1057,1058,1059,1060,1061,1062,1063,1064,1065,1066,1067,1068,1069,1070,1071,1072,1073,1074,1075,1076,1077,1078,1079,1080,1081,1082,1083,1084,1085,1086,1087,1088,1089,1090,1091,1092,1093,1094,1095,1096,1097,1098,1099,1100,1101,1102,1103,1104,1105,1106,1107,1108,1109,1110,1111,1112,1113,1114,1115,1116,1117,1118,1119,1120,1121,1122,1123,1124,1125,1126,1127,1128,1129,1130,1131,1132,1133,1134,1135,1136,1137,1138,1139,1140,1141,1142,1143,1144,1145,1146,1147,1148,1149,1150,1151,1152,1153,1154,1155,1156,1157,1158,1159,1160,1161,1162,1163,1164,1165,1166,1167,1168,1169,1170,1171,1172,1173,1174,1175,1176,1177,1178,1179,1180,1181,1182,1183,1184,1185,1186,1187,1188,1189,1190],"sources":[10,295,297,298,318,321,322,330,338,342,343,344,345,346,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456],"evidence":[]},"index":{"byClaim":[["yk-y01-extropians-1996-12-08-yudkowsky-3761-c",["propositions:yudkowsky-arguments"]],["yk-y01-extropians-1997-09-26-yudkowsky-4420-c",["propositions:yudkowsky-arguments"]],["yk-y01-sl4-2000-09-17-yudkowsky-0009-0092-c",["propositions:yudkowsky-arguments"]],["yk-y01-creating-friendly-ai-2001-b",["propositions:yudkowsky-arguments"]],["yk-y01-sl4-2002-05-03-yudkowsky-0205-3551-c",["propositions:yudkowsky-arguments"]],["yk-y01-sl4-2002-09-19-yudkowsky-0209-5571-b",["propositions:yudkowsky-arguments"]],["yk-y01-sl4-2006-06-06-yudkowsky-0606-15174-c",["propositions:yudkowsky-arguments"]],["yk-y01-arbital-instrumental-convergence-c",["propositions:yudkowsky-arguments"]],["yk-y01-agi-ruin-2022-c",["propositions:yudkowsky-arguments"]],["yk-y01-time-2023-shut-it-all-down-c",["propositions:yudkowsky-arguments"]],["yk-y02-staring-into-the-singularity-b",["propositions:yudkowsky-arguments"]],["yk-y02-extropians-1996-11-22-yudkowsky-3178-a",["propositions:yudkowsky-arguments"]],["yk-y02-extropians-1996-11-22-yudkowsky-3184-a",["propositions:yudkowsky-arguments"]],["yk-y02-extropians-1999-08-01-yudkowsky-1386-c",["propositions:yudkowsky-arguments"]],["yk-y02-sl4-2000-11-13-yudkowsky-0011-0181-c",["propositions:yudkowsky-arguments"]],["yk-y02-sl4-2000-12-10-yudkowsky-0012-0355-c",["propositions:yudkowsky-arguments"]],["yk-y02-sl4-2001-04-12-yudkowsky-0104-1076-c",["propositions:yudkowsky-arguments"]],["yk-y02-coherent-extrapolated-volition-2004-c",["propositions:yudkowsky-arguments"]],["yk-y02-lw-2008-no-universally-compelling-arguments-c",["propositions:yudkowsky-arguments"]],["yk-y02-lw-2008-that-tiny-note-of-discord-c",["propositions:yudkowsky-arguments"]],["yk-y02-arbital-orthogonality-c",["propositions:yudkowsky-arguments"]],["yk-y03-sl4-2000-12-10-yudkowsky-0012-0355-c",["propositions:yudkowsky-arguments"]],["yk-y03-ai-positive-negative-factor-global-risk-c",["propositions:yudkowsky-arguments"]],["yk-y03-lw-2007-the-hidden-complexity-of-wishes-c",["propositions:yudkowsky-arguments"]],["yk-y03-lw-2008-ghosts-in-the-machine-b",["propositions:yudkowsky-arguments"]],["yk-y03-lw-2026-the-talker-does-not-control-the-c",["propositions:yudkowsky-arguments"]],["yk-y04-extropians-1998-12-06-yudkowsky-2153-c",["propositions:yudkowsky-arguments"]],["yk-y04-extropians-1999-02-24-yudkowsky-2293-b",["propositions:yudkowsky-arguments"]],["yk-y04-extropians-2000-03-14-yudkowsky-4837-b",["propositions:yudkowsky-arguments"]],["yk-y04-sl4-2000-10-03-yudkowsky-0010-0125-c",["propositions:yudkowsky-arguments"]],["yk-y04-sl4-2003-05-29-yudkowsky-0305-6846-c",["propositions:yudkowsky-arguments"]],["yk-y04-coherent-extrapolated-volition-2004-c",["propositions:yudkowsky-arguments"]],["yk-y04-lw-2007-the-hidden-complexity-of-wishes-c",["propositions:yudkowsky-arguments"]],["yk-y04-lw-2008-magical-categories-c",["propositions:yudkowsky-arguments"]],["yk-y04-lw-2008-what-i-think-if-not-why-c",["propositions:yudkowsky-arguments"]],["yk-y05-extropians-1999-08-01-yudkowsky-1386-b",["propositions:yudkowsky-arguments"]],["yk-y05-extropians-1999-08-04-yudkowsky-1587-c",["propositions:yudkowsky-arguments"]],["yk-y05-sl4-2000-10-03-yudkowsky-0010-0125-c",["propositions:yudkowsky-arguments"]],["yk-y05-sl4-2000-12-12-yudkowsky-0012-0381-c",["propositions:yudkowsky-arguments"]],["yk-y05-coherent-extrapolated-volition-2004-c",["propositions:yudkowsky-arguments"]],["yk-y05-sl4-2005-08-09-yudkowsky-0508-11866-b",["propositions:yudkowsky-arguments"]],["yk-y05-lw-2007-thou-art-godshatter-b",["propositions:yudkowsky-arguments"]],["yk-y05-agi-ruin-2022-c",["propositions:yudkowsky-arguments"]],["yk-y05-lw-2023-alexander-and-yudkowsky-on-agi-goals-c",["propositions:yudkowsky-arguments"]],["yk-y05-lw-2026-the-talker-does-not-control-the-c",["propositions:yudkowsky-arguments"]],["yk-y06-extropians-1996-12-31-yudkowsky-5142-c",["propositions:yudkowsky-arguments"]],["yk-y06-extropians-1996-12-31-yudkowsky-5159-c",["propositions:yudkowsky-arguments"]],["yk-y06-ai-box-experiment-page-c",["propositions:yudkowsky-arguments"]],["yk-y06-agi-ruin-2022-c",["propositions:yudkowsky-arguments"]],["yk-y07-sl4-2005-01-02-yudkowsky-0501-10512-c",["propositions:yudkowsky-arguments"]],["yk-y07-sl4-2008-02-11-yudkowsky-0802-17621-c",["propositions:yudkowsky-arguments"]],["yk-y07-lw-2008-the-rhythm-of-disagreement-c",["propositions:yudkowsky-arguments"]],["yk-y07-lw-2021-discussion-with-eliezer-yudkowsky-on-agi-c",["propositions:yudkowsky-arguments"]],["yk-y07-lw-2021-ngo-and-yudkowsky-on-alignment-difficulty-c",["propositions:yudkowsky-arguments"]],["yk-y07-time-2023-shut-it-all-down-c",["propositions:yudkowsky-arguments"]],["yk-y07-lw-2026-cat-belling-problems-c",["propositions:yudkowsky-arguments"]],["yk-y08-extropians-1999-01-06-yudkowsky-0160-c",["propositions:yudkowsky-arguments"]],["yk-y08-sl4-2000-07-09-yudkowsky-0007-0017-b",["propositions:yudkowsky-arguments"]],["yk-y08-sl4-2005-07-13-yudkowsky-0507-11477-c",["propositions:yudkowsky-arguments"]],["yk-y08-ai-positive-negative-factor-global-risk-c",["propositions:yudkowsky-arguments"]],["yk-y08-sl4-2006-06-06-yudkowsky-0606-15174-c",["propositions:yudkowsky-arguments"]],["yk-y08-lw-2013-after-critical-event-w-happens-they-c",["propositions:yudkowsky-arguments"]],["yk-y08-lw-2017-theres-no-fire-alarm-for-artificial-c",["propositions:yudkowsky-arguments"]],["yk-y08-lw-2022-miri-announces-new-death-with-dignity-c",["propositions:yudkowsky-arguments"]],["yk-y08-agi-ruin-2022-c",["propositions:yudkowsky-arguments"]],["yk-y09-lw-2022-shah-and-yudkowsky-on-alignment-failures-c",["propositions:yudkowsky-arguments"]],["yk-y09-lw-2023-gpts-are-predictors-not-imitators-c",["propositions:yudkowsky-arguments"]],["yk-y10-extropians-1996-11-13-yudkowsky-2941-c",["propositions:yudkowsky-arguments"]],["yk-y10-extropians-1996-11-16-yudkowsky-3042-c",["propositions:yudkowsky-arguments"]],["yk-y10-staring-into-the-singularity-c",["propositions:yudkowsky-arguments"]],["yk-y10-extropians-1998-09-18-yudkowsky-2487-c",["propositions:yudkowsky-arguments"]],["yk-y10-yudkowsky-2008-recursive-self-improvement-c",["propositions:yudkowsky-arguments"]],["yk-y10-iem-2013-c",["propositions:yudkowsky-arguments"]],["yk-y11-extropians-1998-09-08-yudkowsky-2235-c",["propositions:yudkowsky-arguments"]],["yk-y11-extropians-1999-08-03-yudkowsky-1487-c",["propositions:yudkowsky-arguments"]],["yk-y11-sl4-2000-07-09-yudkowsky-0007-0017-b",["propositions:yudkowsky-arguments"]],["yk-y11-sl4-2000-08-21-yudkowsky-0008-0046-b",["propositions:yudkowsky-arguments"]],["yk-y11-sl4-2000-10-03-yudkowsky-0010-0125-c",["propositions:yudkowsky-arguments"]],["yk-y11-sl4-2001-01-24-yudkowsky-0101-0452-c",["propositions:yudkowsky-arguments"]],["yk-y11-creating-friendly-ai-2001-c",["propositions:yudkowsky-arguments"]],["yk-y11-sl4-2004-12-29-yudkowsky-0412-10488-c",["propositions:yudkowsky-arguments"]],["yk-y11-yudkowsky-2008-hard-takeoff-c",["propositions:yudkowsky-arguments"]],["yk-y11-lw-2021-yudkowsky-and-christiano-discuss-takeoff-speeds-c",["propositions:yudkowsky-arguments"]],["yk-y11-agi-ruin-2022-c",["propositions:yudkowsky-arguments"]],["yk-y12-creating-friendly-ai-2001-c",["propositions:yudkowsky-arguments"]],["yk-y12-arbital-programmer-deception-c",["propositions:yudkowsky-arguments"]],["yk-y12-agi-ruin-2022-c",["propositions:yudkowsky-arguments"]],["yk-y12-lw-2026-irretrievability-or-murphys-curse-of-oneshotness-c",["propositions:yudkowsky-arguments"]],["yk-y13-sl4-2000-11-18-yudkowsky-0011-0240-b",["propositions:yudkowsky-arguments"]],["yk-y13-sl4-2002-07-02-yudkowsky-0207-4667-b",["propositions:yudkowsky-arguments"]],["yk-y13-ai-positive-negative-factor-global-risk-c",["propositions:yudkowsky-arguments"]],["yk-y13-lw-2017-theres-no-fire-alarm-for-artificial-c",["propositions:yudkowsky-arguments"]],["yk-y13-lw-2021-biology-inspired-agi-timelines-the-trick-c",["propositions:yudkowsky-arguments"]],["yk-y14-extropians-2000-09-05-yudkowsky-4277-c",["propositions:yudkowsky-arguments"]],["yk-y14-sl4-2000-09-17-yudkowsky-0009-0092-c",["propositions:yudkowsky-arguments"]],["yk-y14-creating-friendly-ai-2001-b",["propositions:yudkowsky-arguments"]],["yk-y14-sl4-2002-09-10-yudkowsky-0209-5398-c",["propositions:yudkowsky-arguments"]],["yk-y14-sl4-2005-07-25-yudkowsky-0507-11765-b",["propositions:yudkowsky-arguments"]],["yk-y14-time-2023-shut-it-all-down-c",["propositions:yudkowsky-arguments"]],["yk-y14-lw-2026-irretrievability-or-murphys-curse-of-oneshotness-c",["propositions:yudkowsky-arguments"]],["yk-y15-extropians-1998-09-15-yudkowsky-2411-c",["propositions:yudkowsky-arguments"]],["yk-y15-extropians-1999-04-08-yudkowsky-0310-c",["propositions:yudkowsky-arguments"]],["yk-y15-sl4-2001-02-25-yudkowsky-0102-0594-c",["propositions:yudkowsky-arguments"]],["yk-y15-lw-2008-the-rhythm-of-disagreement-c",["propositions:yudkowsky-arguments"]],["yk-y15-lw-2008-dreams-of-friendliness-c",["propositions:yudkowsky-arguments"]],["yk-y15-lw-2012-reply-to-holden-on-tool-ai-c",["propositions:yudkowsky-arguments"]],["yk-y15-lw-2021-ngo-and-yudkowsky-on-alignment-difficulty-c",["propositions:yudkowsky-arguments"]],["yk-y16-extropians-1996-12-08-yudkowsky-3761-c",["propositions:yudkowsky-arguments"]],["yk-y16-sl4-2000-10-03-yudkowsky-0010-0125-c",["propositions:yudkowsky-arguments"]],["yk-y16-sl4-2000-11-18-yudkowsky-0011-0240-c",["propositions:yudkowsky-arguments"]],["yk-y16-sl4-2002-06-27-yudkowsky-0206-4453-c",["propositions:yudkowsky-arguments"]],["yk-y16-coherent-extrapolated-volition-2004-c",["propositions:yudkowsky-arguments"]],["yk-y16-lw-2009-value-is-fragile-c",["propositions:yudkowsky-arguments"]],["yk-y16-agi-ruin-2022-c",["propositions:yudkowsky-arguments"]],["yk-y17-extropians-1998-12-06-yudkowsky-2153-c",["propositions:yudkowsky-arguments"]],["yk-y17-extropians-1999-10-21-yudkowsky-1086-c",["propositions:yudkowsky-arguments"]],["yk-y17-sl4-2001-04-06-yudkowsky-0104-0956-c",["propositions:yudkowsky-arguments"]],["yk-y17-sl4-2001-04-08-yudkowsky-0104-1019-c",["propositions:yudkowsky-arguments"]],["yk-y17-creating-friendly-ai-2001-c",["propositions:yudkowsky-arguments"]],["yk-y17-time-2023-shut-it-all-down-b",["propositions:yudkowsky-arguments"]],["yk-y17-time-2023-shut-it-all-down-c",["propositions:yudkowsky-arguments"]],["yk-y17-lw-2026-only-law-can-prevent-extinction-b",["propositions:yudkowsky-arguments"]],["yk-y17-lw-2026-only-law-can-prevent-extinction-c",["propositions:yudkowsky-arguments"]],["yk-y18-extropians-1996-11-22-yudkowsky-3178-b",["propositions:yudkowsky-arguments"]],["yk-y18-extropians-1999-08-25-yudkowsky-2783-c",["propositions:yudkowsky-arguments"]],["yk-y18-sl4-2000-10-03-yudkowsky-0010-0125-c",["propositions:yudkowsky-arguments"]],["yk-y18-sl4-2002-06-23-yudkowsky-0206-4208-c",["propositions:yudkowsky-arguments"]],["yk-y18-sl4-2006-06-13-yudkowsky-0606-15271-c",["propositions:yudkowsky-arguments"]],["yk-y18-lw-2024-the-sun-is-big-but-superintelligences-c",["propositions:yudkowsky-arguments"]],["yk-y19-extropians-1996-11-22-yudkowsky-3178-b",["propositions:yudkowsky-arguments"]],["yk-y19-extropians-1998-09-07-yudkowsky-2171-c",["propositions:yudkowsky-arguments"]],["yk-y19-sl4-2000-10-03-yudkowsky-0010-0125-c",["propositions:yudkowsky-arguments"]],["yk-y19-sl4-2002-06-23-yudkowsky-0206-4208-c",["propositions:yudkowsky-arguments"]],["yk-y19-sl4-2005-07-13-yudkowsky-0507-11483-c",["propositions:yudkowsky-arguments"]],["yk-y19-ai-positive-negative-factor-global-risk-c",["propositions:yudkowsky-arguments"]],["yk-y19-iem-2013-c",["propositions:yudkowsky-arguments"]],["yk-y19-arbital-instrumental-convergence-c",["propositions:yudkowsky-arguments"]],["yk-y19-time-2023-shut-it-all-down-c",["propositions:yudkowsky-arguments"]],["yk-y20-staring-into-the-singularity-b",["propositions:yudkowsky-arguments"]],["yk-y20-extropians-1997-09-25-yudkowsky-4400-b",["propositions:yudkowsky-arguments"]],["yk-y20-extropians-1999-08-04-yudkowsky-1591-c",["propositions:yudkowsky-arguments"]],["yk-y20-sl4-2001-04-19-yudkowsky-0104-1166-c",["propositions:yudkowsky-arguments"]],["yk-y20-creating-friendly-ai-2001-c",["propositions:yudkowsky-arguments"]],["yk-y20-ai-positive-negative-factor-global-risk-b",["propositions:yudkowsky-arguments"]],["yk-y20-lw-2007-the-logical-fallacy-of-generalization-from-c",["propositions:yudkowsky-arguments"]],["yk-y21-extropians-1999-08-04-yudkowsky-1589-c",["propositions:yudkowsky-arguments"]],["yk-y21-sl4-2002-03-09-yudkowsky-0203-3148-c",["propositions:yudkowsky-arguments"]],["yk-y21-sl4-2002-06-23-yudkowsky-0206-4219-c",["propositions:yudkowsky-arguments"]],["yk-y21-lw-2018-challenges-to-christianos-capability-amplification-proposal-c",["propositions:yudkowsky-arguments"]],["yk-y21-agi-ruin-2022-c",["propositions:yudkowsky-arguments"]],["yk-y21-lw-2023-gpts-are-predictors-not-imitators-c",["propositions:yudkowsky-arguments"]],["yk-y21-lw-2025-re-recent-anthropic-safety-research-c",["propositions:yudkowsky-arguments"]],["yk-y22-staring-into-the-singularity-a",["propositions:yudkowsky-arguments"]],["yk-y22-extropians-1996-11-22-yudkowsky-3178-a",["propositions:yudkowsky-arguments"]],["yk-y22-sl4-2000-11-13-yudkowsky-0011-0181-c",["propositions:yudkowsky-arguments"]],["yk-y22-sl4-2001-08-01-yudkowsky-0108-2031-c",["propositions:yudkowsky-arguments"]],["yk-y22-lw-2008-fighting-a-rearguard-action-against-the-b",["propositions:yudkowsky-arguments"]],["yk-y22-lw-2008-the-magnitude-of-his-own-folly-c",["propositions:yudkowsky-arguments"]],["yk-y22-time-2023-shut-it-all-down-c",["propositions:yudkowsky-arguments"]],["yk-y24-extropians-2000-09-29-yudkowsky-6080-a",["propositions:yudkowsky-arguments"]],["yk-y24-sl4-2001-02-26-yudkowsky-0102-0603-b",["propositions:yudkowsky-arguments"]],["yk-y24-sl4-2002-03-10-yudkowsky-0203-3160-b",["propositions:yudkowsky-arguments"]],["yk-y24-sl4-2002-05-03-yudkowsky-0205-3551-a",["propositions:yudkowsky-arguments"]],["yk-y24-ai-positive-negative-factor-global-risk-b",["propositions:yudkowsky-arguments"]],["yk-y24-sl4-2007-11-24-yudkowsky-0711-17248-c",["propositions:yudkowsky-arguments"]],["yk-y24-lw-2021-discussion-with-eliezer-yudkowsky-on-agi-c",["propositions:yudkowsky-arguments"]],["yk-y24-time-2023-shut-it-all-down-c",["propositions:yudkowsky-arguments"]],["yk-y25-sl4-2000-07-20-yudkowsky-0007-0023-a",["propositions:yudkowsky-arguments"]],["yk-y25-creating-friendly-ai-2001-b",["propositions:yudkowsky-arguments"]],["yk-y25-sl4-2003-05-22-yudkowsky-0305-6783-c",["propositions:yudkowsky-arguments"]],["yk-y25-agi-ruin-2022-c",["propositions:yudkowsky-arguments"]],["yk-y26-extropians-1996-12-31-yudkowsky-5142-c",["propositions:yudkowsky-arguments"]],["yk-y26-extropians-1997-09-26-yudkowsky-4420-b",["propositions:yudkowsky-arguments"]],["yk-y26-extropians-2000-09-05-yudkowsky-4292-c",["propositions:yudkowsky-arguments"]],["yk-y26-sl4-2000-09-16-yudkowsky-0009-0086-c",["propositions:yudkowsky-arguments"]],["yk-y26-ai-positive-negative-factor-global-risk-c",["propositions:yudkowsky-arguments"]],["yk-y26-agi-ruin-2022-c",["propositions:yudkowsky-arguments"]],["yk-y26-time-2023-shut-it-all-down-c",["propositions:yudkowsky-arguments"]],["yk-y27-sl4-2000-09-17-yudkowsky-0009-0092-c",["propositions:yudkowsky-arguments"]],["yk-y27-creating-friendly-ai-2001-b",["propositions:yudkowsky-arguments"]],["yk-y27-agi-ruin-2022-c",["propositions:yudkowsky-arguments"]],["yk-y27-time-2023-shut-it-all-down-c",["propositions:yudkowsky-arguments"]],["yk-y27-lw-2026-irretrievability-or-murphys-curse-of-oneshotness-c",["propositions:yudkowsky-arguments"]],["yk-y28-extropians-2000-09-29-yudkowsky-6073-b",["propositions:yudkowsky-arguments"]],["yk-y28-creating-friendly-ai-2001-a",["propositions:yudkowsky-arguments"]],["yk-y28-sl4-2001-08-01-yudkowsky-0108-2031-c",["propositions:yudkowsky-arguments"]],["yk-y28-ai-positive-negative-factor-global-risk-c",["propositions:yudkowsky-arguments"]],["yk-y28-sl4-2006-02-15-yudkowsky-0602-14096-c",["propositions:yudkowsky-arguments"]],["yk-y28-lw-2008-above-average-ai-scientists-b",["propositions:yudkowsky-arguments"]],["yk-y28-lw-2022-six-dimensions-of-operational-adequacy-in-c",["propositions:yudkowsky-arguments"]],["yk-y28-agi-ruin-2022-c",["propositions:yudkowsky-arguments"]],["yk-y29-extropians-2000-03-16-yudkowsky-5105-c",["propositions:yudkowsky-arguments"]],["yk-y29-sl4-2001-02-04-yudkowsky-0102-0539-c",["propositions:yudkowsky-arguments"]],["yk-y29-sl4-2001-02-05-yudkowsky-0102-0542-c",["propositions:yudkowsky-arguments"]],["yk-y29-sl4-2004-05-30-yudkowsky-0405-8843-c",["propositions:yudkowsky-arguments"]],["yk-y29-ai-positive-negative-factor-global-risk-c",["propositions:yudkowsky-arguments"]],["yk-y29-agi-ruin-2022-c",["propositions:yudkowsky-arguments"]],["yk-y29-lw-2026-irretrievability-or-murphys-curse-of-oneshotness-c",["propositions:yudkowsky-arguments"]],["yk-y30-staring-into-the-singularity-b",["propositions:yudkowsky-arguments"]],["yk-y30-extropians-1996-11-22-yudkowsky-3178-a",["propositions:yudkowsky-arguments"]],["yk-y30-sl4-2000-11-13-yudkowsky-0011-0181-c",["propositions:yudkowsky-arguments"]],["yk-y30-sl4-2001-02-07-yudkowsky-0102-0545-c",["propositions:yudkowsky-arguments"]],["yk-y30-sl4-2004-06-01-yudkowsky-0406-8905-c",["propositions:yudkowsky-arguments"]],["yk-y30-ai-positive-negative-factor-global-risk-c",["propositions:yudkowsky-arguments"]],["yk-y30-lw-2008-that-tiny-note-of-discord-b",["propositions:yudkowsky-arguments"]],["yk-y30-lw-2022-miri-announces-new-death-with-dignity-c",["propositions:yudkowsky-arguments"]],["yk-y30-time-2023-shut-it-all-down-c",["propositions:yudkowsky-arguments"]]],"marks":[["staring-into-the-singularity",[{"collectionId":"yudkowsky-arguments","propositionId":"build-superintelligence-fast","claimId":"yk-y22-staring-into-the-singularity-a","text":"Earliest located in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026 for Y22 Build superintelligence as fast as possible"}]],["extropians-1996-12-31-yudkowsky-5142",[{"collectionId":"superintelligence","propositionId":"observed-behaviour-as-evidence","claimId":"ts-b5-yudkowsky-1996-12-31-5142","text":"Earliest located in Propositions about superintelligence, 1863–2022 for B5 Observed behaviour under training or oversight as evidence"},{"collectionId":"yudkowsky-arguments","propositionId":"confinement-keeps-it-safe","claimId":"yk-y06-extropians-1996-12-31-yudkowsky-5142-c","text":"Earliest located text contesting Y06 Confinement without network access keeps it safe in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"},{"collectionId":"yudkowsky-arguments","propositionId":"no-body-no-danger","claimId":"yk-y26-extropians-1996-12-31-yudkowsky-5142-c","text":"Earliest located text contesting Y26 Without a body it cannot do much in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["creating-friendly-ai-2001",[{"collectionId":"superintelligence","propositionId":"recursive-self-improvement","claimId":"ts-a3-cfai-context","text":"Earliest located use of the name “recursive self-improvement” in Propositions about superintelligence, 1863–2022 for A3 Recursive self-improvement"},{"collectionId":"superintelligence","propositionId":"treacherous-turn","claimId":"ts-b1-cfai","text":"Earliest located in Propositions about superintelligence, 1863–2022 for B1 Treacherous turn"},{"collectionId":"risk-debate","propositionId":"treacherous-turn","claimId":"ts-b1-cfai","text":"Earliest located in Arguments about catastrophic risk from advanced AI for B1 Treacherous turn"},{"collectionId":"yudkowsky-arguments","propositionId":"evaluations-catch-danger","claimId":"yk-y12-creating-friendly-ai-2001-c","text":"Earliest located text contesting Y12 Safety evaluations would catch a dangerous system in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"},{"collectionId":"yudkowsky-arguments","propositionId":"developers-know-what-they-are-doing","claimId":"yk-y28-creating-friendly-ai-2001-a","text":"Earliest located in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026 for Y28 The developers know what they are doing"}]],["ai-positive-negative-factor-global-risk",[{"collectionId":"yudkowsky-arguments","propositionId":"agi-decades-away","claimId":"yk-y13-ai-positive-negative-factor-global-risk-c","text":"Earliest located text contesting Y13 AGI is decades away, so the problem can wait in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["yudkowsky-2008-recursive-self-improvement",[{"collectionId":"economics-of-machine-intelligence","propositionId":"ai-improving-ai-accelerates-capability","claimId":"ec-e01-yudkowsky-2008-recursive-self-improvement-a","text":"Earliest located in tier 2 of Economics of machine intelligence for E01 Machine intelligence improving itself accelerates its capability growth"}]],["extropians-2000-03-16-yudkowsky-5105",[{"collectionId":"yudkowsky-arguments","propositionId":"many-ais-balance","claimId":"yk-y29-extropians-2000-03-16-yudkowsky-5105-c","text":"Earliest located text contesting Y29 Many AIs will keep each other in check in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["extropians-1999-01-06-yudkowsky-0160",[{"collectionId":"yudkowsky-arguments","propositionId":"warning-signs-in-time","claimId":"yk-y08-extropians-1999-01-06-yudkowsky-0160-c","text":"Earliest located text contesting Y08 Clear warning signs will come before danger in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["extropians-2000-09-05-yudkowsky-4277",[{"collectionId":"yudkowsky-arguments","propositionId":"fix-problems-as-they-arise","claimId":"yk-y14-extropians-2000-09-05-yudkowsky-4277-c","text":"Earliest located text contesting Y14 Problems can be fixed as they arise in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["extropians-1998-09-07-yudkowsky-2171",[{"collectionId":"yudkowsky-arguments","propositionId":"harm-requires-hostility","claimId":"yk-y19-extropians-1998-09-07-yudkowsky-2171-c","text":"Earliest located text contesting Y19 Without hatred it will not harm us in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["extropians-1998-09-08-yudkowsky-2235",[{"collectionId":"yudkowsky-arguments","propositionId":"slow-takeoff-time-to-adjust","claimId":"yk-y11-extropians-1998-09-08-yudkowsky-2235-c","text":"Earliest located text contesting Y11 A slow takeoff will leave time to adjust in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["extropians-1998-09-15-yudkowsky-2411",[{"collectionId":"yudkowsky-arguments","propositionId":"tool-without-goals","claimId":"yk-y15-extropians-1998-09-15-yudkowsky-2411-c","text":"Earliest located text contesting Y15 A tool that wants nothing in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["extropians-1999-08-01-yudkowsky-1386",[{"collectionId":"yudkowsky-arguments","propositionId":"intelligence-brings-benevolence","claimId":"yk-y02-extropians-1999-08-01-yudkowsky-1386-c","text":"Earliest located text contesting Y02 Sufficient intelligence brings good values in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["extropians-1999-08-04-yudkowsky-1587",[{"collectionId":"yudkowsky-arguments","propositionId":"feedback-instils-values","claimId":"yk-y05-extropians-1999-08-04-yudkowsky-1587-c","text":"Earliest located text contesting Y05 Training on human feedback instils human values in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["extropians-1999-08-04-yudkowsky-1589",[{"collectionId":"yudkowsky-arguments","propositionId":"human-data-humanlike-mind","claimId":"yk-y21-extropians-1999-08-04-yudkowsky-1589-c","text":"Earliest located text contesting Y21 Trained on human data, so humanlike in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["extropians-1999-08-04-yudkowsky-1591",[{"collectionId":"yudkowsky-arguments","propositionId":"risk-is-film-fiction","claimId":"yk-y20-extropians-1999-08-04-yudkowsky-1591-c","text":"Earliest located text contesting Y20 AI risk as film science fiction in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["extropians-1999-08-25-yudkowsky-2783",[{"collectionId":"yudkowsky-arguments","propositionId":"earth-not-worth-taking","claimId":"yk-y18-extropians-1999-08-25-yudkowsky-2783-c","text":"Earliest located text contesting Y18 A superintelligence would leave Earth alone in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["extropians-1998-12-06-yudkowsky-2153",[{"collectionId":"yudkowsky-arguments","propositionId":"short-instruction-suffices","claimId":"yk-y04-extropians-1998-12-06-yudkowsky-2153-c","text":"Earliest located text contesting Y04 A short instruction such as making everyone happy in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"},{"collectionId":"yudkowsky-arguments","propositionId":"pause-or-regulation-suffices","claimId":"yk-y17-extropians-1998-12-06-yudkowsky-2153-c","text":"Earliest located text contesting Y17 A pause or regulation will handle it in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["extropians-1996-11-13-yudkowsky-2941",[{"collectionId":"yudkowsky-arguments","propositionId":"diminishing-returns-to-intelligence","claimId":"yk-y10-extropians-1996-11-13-yudkowsky-2941-c","text":"Earliest located text contesting Y10 Diminishing returns rule out a runaway in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["extropians-1996-11-22-yudkowsky-3178",[{"collectionId":"yudkowsky-arguments","propositionId":"intelligence-brings-benevolence","claimId":"yk-y02-extropians-1996-11-22-yudkowsky-3178-a","text":"Earliest located in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026 for Y02 Sufficient intelligence brings good values"},{"collectionId":"yudkowsky-arguments","propositionId":"default-outcome-good","claimId":"yk-y30-extropians-1996-11-22-yudkowsky-3178-a","text":"Earliest located in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026 for Y30 It will probably turn out fine"}]],["extropians-1996-12-08-yudkowsky-3761",[{"collectionId":"yudkowsky-arguments","propositionId":"switch-off-if-dangerous","claimId":"yk-y01-extropians-1996-12-08-yudkowsky-3761-c","text":"Earliest located text contesting Y01 Switching it off if it becomes dangerous in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"},{"collectionId":"yudkowsky-arguments","propositionId":"give-it-human-values","claimId":"yk-y16-extropians-1996-12-08-yudkowsky-3761-c","text":"Earliest located text contesting Y16 Giving it good human values in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["lw-2022-shah-and-yudkowsky-on-alignment-failures",[{"collectionId":"yudkowsky-arguments","propositionId":"next-word-prediction-is-shallow","claimId":"yk-y09-lw-2022-shah-and-yudkowsky-on-alignment-failures-c","text":"Earliest located text contesting Y09 Next-word prediction is only autocomplete in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["sl4-2000-07-20-yudkowsky-0007-0023",[{"collectionId":"yudkowsky-arguments","propositionId":"interpretability-reads-its-mind","claimId":"yk-y25-sl4-2000-07-20-yudkowsky-0007-0023-a","text":"Earliest located in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026 for Y25 Interpretability will let us read its mind"}]],["sl4-2000-09-17-yudkowsky-0009-0092",[{"collectionId":"yudkowsky-arguments","propositionId":"experiment-and-see","claimId":"yk-y27-sl4-2000-09-17-yudkowsky-0009-0092-c","text":"Earliest located text contesting Y27 Run experiments and see what happens in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["sl4-2000-11-13-yudkowsky-0011-0181",[{"collectionId":"yudkowsky-arguments","propositionId":"build-superintelligence-fast","claimId":"yk-y22-sl4-2000-11-13-yudkowsky-0011-0181-c","text":"Earliest located text contesting Y22 Build superintelligence as fast as possible in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"},{"collectionId":"yudkowsky-arguments","propositionId":"default-outcome-good","claimId":"yk-y30-sl4-2000-11-13-yudkowsky-0011-0181-c","text":"Earliest located text contesting Y30 It will probably turn out fine in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["sl4-2000-12-10-yudkowsky-0012-0355",[{"collectionId":"yudkowsky-arguments","propositionId":"understanding-implies-compliance","claimId":"yk-y03-sl4-2000-12-10-yudkowsky-0012-0355-c","text":"Earliest located text contesting Y03 A superintelligence will do what we mean in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["sl4-2001-08-01-yudkowsky-0108-2031",[{"collectionId":"yudkowsky-arguments","propositionId":"developers-know-what-they-are-doing","claimId":"yk-y28-sl4-2001-08-01-yudkowsky-0108-2031-c","text":"Earliest located text contesting Y28 The developers know what they are doing in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["sl4-2003-05-22-yudkowsky-0305-6783",[{"collectionId":"yudkowsky-arguments","propositionId":"interpretability-reads-its-mind","claimId":"yk-y25-sl4-2003-05-22-yudkowsky-0305-6783-c","text":"Earliest located text contesting Y25 Interpretability will let us read its mind in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["sl4-2005-01-02-yudkowsky-0501-10512",[{"collectionId":"yudkowsky-arguments","propositionId":"ai-solves-alignment","claimId":"yk-y07-sl4-2005-01-02-yudkowsky-0501-10512-c","text":"Earliest located text contesting Y07 AI systems will solve alignment for us in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]],["sl4-2007-11-24-yudkowsky-0711-17248",[{"collectionId":"yudkowsky-arguments","propositionId":"build-it-before-rivals","claimId":"yk-y24-sl4-2007-11-24-yudkowsky-0711-17248-c","text":"Earliest located text contesting Y24 Build it before rivals do in Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026"}]]],"chronicle":{"years":[{"year":1863,"topics":{"ideas":5},"stated":0},{"year":1865,"topics":{"ideas":1},"stated":0},{"year":1872,"topics":{"ideas":7},"stated":0},{"year":1943,"topics":{"ideas":1},"stated":0},{"year":1947,"topics":{"ideas":2},"stated":0},{"year":1950,"topics":{"ideas":7},"stated":0},{"year":1951,"topics":{"ideas":5},"stated":0},{"year":1952,"topics":{"ideas":2},"stated":0},{"year":1955,"topics":{"ideas":31},"stated":0},{"year":1956,"topics":{"ideas":1},"stated":0},{"year":1957,"topics":{"ideas":1},"stated":0},{"year":1958,"topics":{"ideas":3},"stated":0},{"year":1959,"topics":{"ideas":6},"stated":0},{"year":1960,"topics":{"ideas":11},"stated":0},{"year":1961,"topics":{"ideas":10},"stated":0},{"year":1965,"topics":{"ideas":8},"stated":0},{"year":1966,"topics":{"ideas":20},"stated":0},{"year":1968,"topics":{"ideas":5},"stated":0},{"year":1969,"topics":{"ideas":2},"stated":0},{"year":1971,"topics":{"ideas":4},"stated":0},{"year":1972,"topics":{"ideas":1},"stated":0},{"year":1973,"topics":{"ideas":2},"stated":0},{"year":1974,"topics":{"ideas":2},"stated":0},{"year":1975,"topics":{"ideas":1},"stated":0},{"year":1976,"topics":{"ideas":1},"stated":0},{"year":1977,"topics":{"people":2,"ideas":8},"stated":0},{"year":1979,"topics":{"ideas":10},"stated":0},{"year":1980,"topics":{"ideas":2},"stated":0},{"year":1981,"topics":{"people":2},"stated":0},{"year":1982,"topics":{"ideas":4},"stated":0},{"year":1983,"topics":{"ideas":4},"stated":0},{"year":1984,"topics":{"ideas":21},"stated":0},{"year":1985,"topics":{"ideas":9},"stated":0},{"year":1986,"topics":{"ideas":18},"stated":0},{"year":1987,"topics":{"ideas":1},"stated":0},{"year":1988,"topics":{"ideas":6},"stated":0},{"year":1989,"topics":{"ideas":25},"stated":0},{"year":1990,"topics":{"ideas":9},"stated":0},{"year":1991,"topics":{"ideas":14},"stated":0},{"year":1992,"topics":{"ideas":15},"stated":0},{"year":1993,"topics":{"ideas":21,"people":3},"stated":0},{"year":1994,"topics":{"ideas":18},"stated":0},{"year":1995,"topics":{"ideas":10,"people":1},"stated":0},{"year":1996,"topics":{"ideas":71},"stated":0},{"year":1997,"topics":{"ideas":17},"stated":0},{"year":1998,"topics":{"ideas":50},"stated":0},{"year":1999,"topics":{"ideas":31,"people":1,"institutions":1},"stated":0},{"year":2000,"topics":{"ideas":70},"stated":0},{"year":2001,"topics":{"ideas":52},"stated":0},{"year":2002,"topics":{"ideas":30},"stated":0},{"year":2003,"topics":{"ideas":9},"stated":0},{"year":2004,"topics":{"ideas":14,"people":3},"stated":0},{"year":2005,"topics":{"ideas":20,"people":1},"stated":0},{"year":2006,"topics":{"ideas":26,"people":1},"stated":0},{"year":2007,"topics":{"ideas":21},"stated":0},{"year":2008,"topics":{"ideas":65,"people":3,"institutions":1},"stated":0},{"year":2009,"topics":{"ideas":12,"people":1},"stated":0},{"year":2010,"topics":{"people":3,"ideas":13,"institutions":1,"governance":4},"stated":0},{"year":2011,"topics":{"people":3,"ideas":16,"institutions":2},"stated":0},{"year":2012,"topics":{"people":4,"ideas":19,"institutions":2},"stated":0},{"year":2013,"topics":{"ideas":45,"people":4,"institutions":2,"capital":1},"stated":0},{"year":2014,"topics":{"institutions":2,"ideas":57,"people":5},"stated":0},{"year":2015,"topics":{"people":23,"ideas":133,"institutions":3,"capital":1},"stated":0},{"year":2016,"topics":{"ideas":88,"people":2},"stated":0},{"year":2017,"topics":{"people":16,"institutions":2,"ideas":130,"governance":2},"stated":0},{"year":2018,"topics":{"people":9,"ideas":159,"institutions":8,"governance":1},"stated":0},{"year":2019,"topics":{"capital":1,"compute":1,"people":11,"governance":19,"ideas":175,"institutions":4},"stated":0},{"year":2020,"topics":{"people":6,"ideas":98,"institutions":1,"governance":2},"stated":0},{"year":2021,"topics":{"people":27,"ideas":177,"compute":1,"institutions":23,"governance":4,"capital":6},"stated":0},{"year":2022,"topics":{"people":15,"ideas":228,"institutions":6,"governance":3,"compute":2,"capital":1},"stated":0},{"year":2023,"topics":{"capital":14,"compute":7,"institutions":24,"people":54,"governance":179,"ideas":409},"stated":0},{"year":2024,"topics":{"capital":31,"compute":19,"governance":282,"people":103,"ideas":180,"institutions":32},"stated":1},{"year":2025,"topics":{"governance":548,"compute":69,"capital":61,"people":145,"institutions":26,"ideas":180},"stated":7},{"year":2026,"topics":{"governance":444,"capital":31,"compute":32,"institutions":23,"ideas":97,"people":139},"stated":19},{"year":2027,"topics":{},"stated":8},{"year":2028,"topics":{},"stated":6},{"year":2029,"topics":{},"stated":3},{"year":2030,"topics":{},"stated":3},{"year":2032,"topics":{},"stated":2},{"year":2035,"topics":{},"stated":2},{"year":2039,"topics":{},"stated":1},{"year":2042,"topics":{},"stated":1},{"year":2050,"topics":{},"stated":1}],"scheduled":[],"statedBy":[]},"collections":[{"target":{"kind":"propositions","id":"superintelligence"},"title":"Propositions about superintelligence, 1863–2022","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"risk-debate"},"title":"Arguments about catastrophic risk from advanced AI","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"yudkowsky-arguments"},"title":"Eliezer Yudkowsky’s arguments on common claims about AI, 1996–2026","claimIds":["yk-y01-extropians-1996-12-08-yudkowsky-3761-c","yk-y01-extropians-1997-09-26-yudkowsky-4420-c","yk-y01-sl4-2000-09-17-yudkowsky-0009-0092-c","yk-y01-creating-friendly-ai-2001-b","yk-y01-sl4-2002-05-03-yudkowsky-0205-3551-c","yk-y01-sl4-2002-09-19-yudkowsky-0209-5571-b","yk-y01-sl4-2006-06-06-yudkowsky-0606-15174-c","yk-y01-arbital-instrumental-convergence-c","yk-y01-agi-ruin-2022-c","yk-y01-time-2023-shut-it-all-down-c","yk-y02-staring-into-the-singularity-b","yk-y02-extropians-1996-11-22-yudkowsky-3178-a","yk-y02-extropians-1996-11-22-yudkowsky-3184-a","yk-y02-extropians-1999-08-01-yudkowsky-1386-c","yk-y02-sl4-2000-11-13-yudkowsky-0011-0181-c","yk-y02-sl4-2000-12-10-yudkowsky-0012-0355-c","yk-y02-sl4-2001-04-12-yudkowsky-0104-1076-c","yk-y02-coherent-extrapolated-volition-2004-c","yk-y02-lw-2008-no-universally-compelling-arguments-c","yk-y02-lw-2008-that-tiny-note-of-discord-c","yk-y02-arbital-orthogonality-c","yk-y03-sl4-2000-12-10-yudkowsky-0012-0355-c","yk-y03-ai-positive-negative-factor-global-risk-c","yk-y03-lw-2007-the-hidden-complexity-of-wishes-c","yk-y03-lw-2008-ghosts-in-the-machine-b","yk-y03-lw-2026-the-talker-does-not-control-the-c","yk-y04-extropians-1998-12-06-yudkowsky-2153-c","yk-y04-extropians-1999-02-24-yudkowsky-2293-b","yk-y04-extropians-2000-03-14-yudkowsky-4837-b","yk-y04-sl4-2000-10-03-yudkowsky-0010-0125-c","yk-y04-sl4-2003-05-29-yudkowsky-0305-6846-c","yk-y04-coherent-extrapolated-volition-2004-c","yk-y04-lw-2007-the-hidden-complexity-of-wishes-c","yk-y04-lw-2008-magical-categories-c","yk-y04-lw-2008-what-i-think-if-not-why-c","yk-y05-extropians-1999-08-01-yudkowsky-1386-b","yk-y05-extropians-1999-08-04-yudkowsky-1587-c","yk-y05-sl4-2000-10-03-yudkowsky-0010-0125-c","yk-y05-sl4-2000-12-12-yudkowsky-0012-0381-c","yk-y05-coherent-extrapolated-volition-2004-c","yk-y05-sl4-2005-08-09-yudkowsky-0508-11866-b","yk-y05-lw-2007-thou-art-godshatter-b","yk-y05-agi-ruin-2022-c","yk-y05-lw-2023-alexander-and-yudkowsky-on-agi-goals-c","yk-y05-lw-2026-the-talker-does-not-control-the-c","yk-y06-extropians-1996-12-31-yudkowsky-5142-c","yk-y06-extropians-1996-12-31-yudkowsky-5159-c","yk-y06-ai-box-experiment-page-c","yk-y06-agi-ruin-2022-c","yk-y07-sl4-2005-01-02-yudkowsky-0501-10512-c","yk-y07-sl4-2008-02-11-yudkowsky-0802-17621-c","yk-y07-lw-2008-the-rhythm-of-disagreement-c","yk-y07-lw-2021-discussion-with-eliezer-yudkowsky-on-agi-c","yk-y07-lw-2021-ngo-and-yudkowsky-on-alignment-difficulty-c","yk-y07-time-2023-shut-it-all-down-c","yk-y07-lw-2026-cat-belling-problems-c","yk-y08-extropians-1999-01-06-yudkowsky-0160-c","yk-y08-sl4-2000-07-09-yudkowsky-0007-0017-b","yk-y08-sl4-2005-07-13-yudkowsky-0507-11477-c","yk-y08-ai-positive-negative-factor-global-risk-c","yk-y08-sl4-2006-06-06-yudkowsky-0606-15174-c","yk-y08-lw-2013-after-critical-event-w-happens-they-c","yk-y08-lw-2017-theres-no-fire-alarm-for-artificial-c","yk-y08-lw-2022-miri-announces-new-death-with-dignity-c","yk-y08-agi-ruin-2022-c","yk-y09-lw-2022-shah-and-yudkowsky-on-alignment-failures-c","yk-y09-lw-2023-gpts-are-predictors-not-imitators-c","yk-y10-extropians-1996-11-13-yudkowsky-2941-c","yk-y10-extropians-1996-11-16-yudkowsky-3042-c","yk-y10-staring-into-the-singularity-c","yk-y10-extropians-1998-09-18-yudkowsky-2487-c","yk-y10-yudkowsky-2008-recursive-self-improvement-c","yk-y10-iem-2013-c","yk-y11-extropians-1998-09-08-yudkowsky-2235-c","yk-y11-extropians-1999-08-03-yudkowsky-1487-c","yk-y11-sl4-2000-07-09-yudkowsky-0007-0017-b","yk-y11-sl4-2000-08-21-yudkowsky-0008-0046-b","yk-y11-sl4-2000-10-03-yudkowsky-0010-0125-c","yk-y11-sl4-2001-01-24-yudkowsky-0101-0452-c","yk-y11-creating-friendly-ai-2001-c","yk-y11-sl4-2004-12-29-yudkowsky-0412-10488-c","yk-y11-yudkowsky-2008-hard-takeoff-c","yk-y11-lw-2021-yudkowsky-and-christiano-discuss-takeoff-speeds-c","yk-y11-agi-ruin-2022-c","yk-y12-creating-friendly-ai-2001-c","yk-y12-arbital-programmer-deception-c","yk-y12-agi-ruin-2022-c","yk-y12-lw-2026-irretrievability-or-murphys-curse-of-oneshotness-c","yk-y13-sl4-2000-11-18-yudkowsky-0011-0240-b","yk-y13-sl4-2002-07-02-yudkowsky-0207-4667-b","yk-y13-ai-positive-negative-factor-global-risk-c","yk-y13-lw-2017-theres-no-fire-alarm-for-artificial-c","yk-y13-lw-2021-biology-inspired-agi-timelines-the-trick-c","yk-y14-extropians-2000-09-05-yudkowsky-4277-c","yk-y14-sl4-2000-09-17-yudkowsky-0009-0092-c","yk-y14-creating-friendly-ai-2001-b","yk-y14-sl4-2002-09-10-yudkowsky-0209-5398-c","yk-y14-sl4-2005-07-25-yudkowsky-0507-11765-b","yk-y14-time-2023-shut-it-all-down-c","yk-y14-lw-2026-irretrievability-or-murphys-curse-of-oneshotness-c","yk-y15-extropians-1998-09-15-yudkowsky-2411-c","yk-y15-extropians-1999-04-08-yudkowsky-0310-c","yk-y15-sl4-2001-02-25-yudkowsky-0102-0594-c","yk-y15-lw-2008-the-rhythm-of-disagreement-c","yk-y15-lw-2008-dreams-of-friendliness-c","yk-y15-lw-2012-reply-to-holden-on-tool-ai-c","yk-y15-lw-2021-ngo-and-yudkowsky-on-alignment-difficulty-c","yk-y16-extropians-1996-12-08-yudkowsky-3761-c","yk-y16-sl4-2000-10-03-yudkowsky-0010-0125-c","yk-y16-sl4-2000-11-18-yudkowsky-0011-0240-c","yk-y16-sl4-2002-06-27-yudkowsky-0206-4453-c","yk-y16-coherent-extrapolated-volition-2004-c","yk-y16-lw-2009-value-is-fragile-c","yk-y16-agi-ruin-2022-c","yk-y17-extropians-1998-12-06-yudkowsky-2153-c","yk-y17-extropians-1999-10-21-yudkowsky-1086-c","yk-y17-sl4-2001-04-06-yudkowsky-0104-0956-c","yk-y17-sl4-2001-04-08-yudkowsky-0104-1019-c","yk-y17-creating-friendly-ai-2001-c","yk-y17-time-2023-shut-it-all-down-b","yk-y17-time-2023-shut-it-all-down-c","yk-y17-lw-2026-only-law-can-prevent-extinction-b","yk-y17-lw-2026-only-law-can-prevent-extinction-c","yk-y18-extropians-1996-11-22-yudkowsky-3178-b","yk-y18-extropians-1999-08-25-yudkowsky-2783-c","yk-y18-sl4-2000-10-03-yudkowsky-0010-0125-c","yk-y18-sl4-2002-06-23-yudkowsky-0206-4208-c","yk-y18-sl4-2006-06-13-yudkowsky-0606-15271-c","yk-y18-lw-2024-the-sun-is-big-but-superintelligences-c","yk-y19-extropians-1996-11-22-yudkowsky-3178-b","yk-y19-extropians-1998-09-07-yudkowsky-2171-c","yk-y19-sl4-2000-10-03-yudkowsky-0010-0125-c","yk-y19-sl4-2002-06-23-yudkowsky-0206-4208-c","yk-y19-sl4-2005-07-13-yudkowsky-0507-11483-c","yk-y19-ai-positive-negative-factor-global-risk-c","yk-y19-iem-2013-c","yk-y19-arbital-instrumental-convergence-c","yk-y19-time-2023-shut-it-all-down-c","yk-y20-staring-into-the-singularity-b","yk-y20-extropians-1997-09-25-yudkowsky-4400-b","yk-y20-extropians-1999-08-04-yudkowsky-1591-c","yk-y20-sl4-2001-04-19-yudkowsky-0104-1166-c","yk-y20-creating-friendly-ai-2001-c","yk-y20-ai-positive-negative-factor-global-risk-b","yk-y20-lw-2007-the-logical-fallacy-of-generalization-from-c","yk-y21-extropians-1999-08-04-yudkowsky-1589-c","yk-y21-sl4-2002-03-09-yudkowsky-0203-3148-c","yk-y21-sl4-2002-06-23-yudkowsky-0206-4219-c","yk-y21-lw-2018-challenges-to-christianos-capability-amplification-proposal-c","yk-y21-agi-ruin-2022-c","yk-y21-lw-2023-gpts-are-predictors-not-imitators-c","yk-y21-lw-2025-re-recent-anthropic-safety-research-c","yk-y22-staring-into-the-singularity-a","yk-y22-extropians-1996-11-22-yudkowsky-3178-a","yk-y22-sl4-2000-11-13-yudkowsky-0011-0181-c","yk-y22-sl4-2001-08-01-yudkowsky-0108-2031-c","yk-y22-lw-2008-fighting-a-rearguard-action-against-the-b","yk-y22-lw-2008-the-magnitude-of-his-own-folly-c","yk-y22-time-2023-shut-it-all-down-c","yk-y24-extropians-2000-09-29-yudkowsky-6080-a","yk-y24-sl4-2001-02-26-yudkowsky-0102-0603-b","yk-y24-sl4-2002-03-10-yudkowsky-0203-3160-b","yk-y24-sl4-2002-05-03-yudkowsky-0205-3551-a","yk-y24-ai-positive-negative-factor-global-risk-b","yk-y24-sl4-2007-11-24-yudkowsky-0711-17248-c","yk-y24-lw-2021-discussion-with-eliezer-yudkowsky-on-agi-c","yk-y24-time-2023-shut-it-all-down-c","yk-y25-sl4-2000-07-20-yudkowsky-0007-0023-a","yk-y25-creating-friendly-ai-2001-b","yk-y25-sl4-2003-05-22-yudkowsky-0305-6783-c","yk-y25-agi-ruin-2022-c","yk-y26-extropians-1996-12-31-yudkowsky-5142-c","yk-y26-extropians-1997-09-26-yudkowsky-4420-b","yk-y26-extropians-2000-09-05-yudkowsky-4292-c","yk-y26-sl4-2000-09-16-yudkowsky-0009-0086-c","yk-y26-ai-positive-negative-factor-global-risk-c","yk-y26-agi-ruin-2022-c","yk-y26-time-2023-shut-it-all-down-c","yk-y27-sl4-2000-09-17-yudkowsky-0009-0092-c","yk-y27-creating-friendly-ai-2001-b","yk-y27-agi-ruin-2022-c","yk-y27-time-2023-shut-it-all-down-c","yk-y27-lw-2026-irretrievability-or-murphys-curse-of-oneshotness-c","yk-y28-extropians-2000-09-29-yudkowsky-6073-b","yk-y28-creating-friendly-ai-2001-a","yk-y28-sl4-2001-08-01-yudkowsky-0108-2031-c","yk-y28-ai-positive-negative-factor-global-risk-c","yk-y28-sl4-2006-02-15-yudkowsky-0602-14096-c","yk-y28-lw-2008-above-average-ai-scientists-b","yk-y28-lw-2022-six-dimensions-of-operational-adequacy-in-c","yk-y28-agi-ruin-2022-c","yk-y29-extropians-2000-03-16-yudkowsky-5105-c","yk-y29-sl4-2001-02-04-yudkowsky-0102-0539-c","yk-y29-sl4-2001-02-05-yudkowsky-0102-0542-c","yk-y29-sl4-2004-05-30-yudkowsky-0405-8843-c","yk-y29-ai-positive-negative-factor-global-risk-c","yk-y29-agi-ruin-2022-c","yk-y29-lw-2026-irretrievability-or-murphys-curse-of-oneshotness-c","yk-y30-staring-into-the-singularity-b","yk-y30-extropians-1996-11-22-yudkowsky-3178-a","yk-y30-sl4-2000-11-13-yudkowsky-0011-0181-c","yk-y30-sl4-2001-02-07-yudkowsky-0102-0545-c","yk-y30-sl4-2004-06-01-yudkowsky-0406-8905-c","yk-y30-ai-positive-negative-factor-global-risk-c","yk-y30-lw-2008-that-tiny-note-of-discord-b","yk-y30-lw-2022-miri-announces-new-death-with-dignity-c","yk-y30-time-2023-shut-it-all-down-c"],"records":[]},{"target":{"kind":"propositions","id":"machine-succession"},"title":"Machine succession, scale and acceleration: propositions in located texts, 1979–2026","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"economics-of-machine-intelligence"},"title":"Economics of machine intelligence","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"orbital-compute-arguments"},"title":"Arguments about compute in orbit","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"ai-bio-arguments"},"title":"Arguments about AI and biological risk","claimIds":[],"records":[]},{"target":{"kind":"propositions","id":"ai-in-science"},"title":"AI in science","claimIds":[],"records":[]},{"target":{"kind":"documents"},"title":"Frontier safety frameworks","claimIds":[]},{"target":{"kind":"compare","id":"frontier-safety-frameworks"},"title":"Compare frameworks","claimIds":[]},{"target":{"kind":"control","id":"openai"},"title":"OpenAI’s control structure, 2015–2026","claimIds":[]},{"target":{"kind":"control","id":"anthropic"},"title":"Anthropic’s control structure, 2021–2026","claimIds":[]},{"target":{"kind":"commitments"},"title":"External commitments and obligations","claimIds":[]},{"target":{"kind":"question","id":"frameworks-to-obligations"},"title":"Frameworks, commitments and legal obligations, 2023–2026","claimIds":[]},{"target":{"kind":"reading"},"title":"Founding and early leadership","claimIds":[]}]},"version":"15mccro"}