{"entities":[{"id":"eliezer-yudkowsky","name":"Eliezer Yudkowsky","type":"person"},{"id":"sl4-2002-03-08-yudkowsky","shortName":"Yudkowsky, SL4 message of 2002-03-08","type":"document"},{"id":"ai-box-experiment-page","shortName":"Yudkowsky, The AI-Box Experiment (2002)","type":"document"},{"id":"sl4-2002-03-09-yudkowsky-0203-3148","shortName":"Yudkowsky, SL4 message of 2002-03-09 (3148)","type":"document"},{"id":"sl4-2002-03-10-yudkowsky-0203-3160","shortName":"Yudkowsky, SL4 message of 2002-03-10 (3160)","type":"document"},{"id":"sl4-2002-05-03-yudkowsky-0205-3551","shortName":"Yudkowsky, SL4 message of 2002-05-03 (3551)","type":"document"},{"id":"sl4-2002-06-23-yudkowsky-0206-4208","shortName":"Yudkowsky, SL4 message of 2002-06-23 (4208)","type":"document"},{"id":"sl4-2002-06-23-yudkowsky-0206-4219","shortName":"Yudkowsky, SL4 message of 2002-06-23 (4219)","type":"document"},{"id":"sl4-2002-06-27-yudkowsky-0206-4453","shortName":"Yudkowsky, SL4 message of 2002-06-27 (4453)","type":"document"},{"id":"sl4-2002-07-02-yudkowsky-0207-4667","shortName":"Yudkowsky, SL4 message of 2002-07-02 (4667)","type":"document"},{"id":"sl4-2002-09-10-yudkowsky-0209-5398","shortName":"Yudkowsky, SL4 message of 2002-09-10 (5398)","type":"document"},{"id":"sl4-2002-09-19-yudkowsky-0209-5571","shortName":"Yudkowsky, SL4 message of 2002-09-19 (5571)","type":"document"}],"claims":[{"id":"ts-b2-yudkowsky-2002-aibox-context","subject":"sl4-2002-03-08-yudkowsky","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"strategic-concealment"},"topic":"ideas","statement":"Yudkowsky’s SL4 message of 2002-03-08 proposes a role-play in which the author plays a mind in a box and tries to persuade the other person to let it out.","time":{"kind":"event","date":"2002-03-08","basis":"publication","note":"Sent header: Fri, 08 Mar 2002 07:13:21 -0500 (sender-local date)."},"derivation":"explicit","resolution":"supported","attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"A role-play of persuasion between two people, not a claim about AI systems."}}},{"id":"yk-y01-sl4-2002-05-03-yudkowsky-0205-3551-c","subject":"sl4-2002-05-03-yudkowsky-0205-3551","predicate":"contests_proposition","object":{"kind":"entity","entityId":"switch-off-if-dangerous"},"topic":"ideas","statement":"The SL4 message “Re: supergoal stability” (2002-05-03) contests Y01. Replying to Wei Dai in a thread on supergoal stability, he says a non-Friendly mind will keep whatever goals it had when it first became able to protect its goal system from modification, and that only the CFAI semantics would keep goals as the programmers understood them.","time":{"kind":"event","date":"2002-05-03","basis":"publication","note":"Archive author index lists the message as \"Fri May 03 2002 - 16:38:46 MDT\" under \"Eliezer S. Yudkowsky\"."},"derivation":"explicit","resolution":"supported","attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that a capable goal-directed system will not act to prevent its own shutdown or modification, and a shutdown or correction mechanism can simply be added to it, even when read with its message or dialogue turn."}}}},{"id":"yk-y01-sl4-2002-09-19-yudkowsky-0209-5571-b","subject":"sl4-2002-09-19-yudkowsky-0209-5571","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"switch-off-if-dangerous"},"topic":"ideas","statement":"The SL4 message “Re: QUES: CFAI [#2]” (2002-09-19) bears on Y01. Answering a series of questions about Creating Friendly AI, he gives the first of four 'most likely reasons FAI might fail': takeoff is easier than expected, and the mechanisms meant to detect and halt an unexpected ascent fail to notice it.","time":{"kind":"event","date":"2002-09-19","basis":"publication","note":"Archive author index lists the message as \"Thu Sep 19 2002 - 13:58:03 MDT\" under \"Eliezer S. Yudkowsky\"."},"derivation":"explicit","resolution":"supported","attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"A conditional failure scenario in which detection of a fast ascent fails; it does not predict that dangerous capability will go unrecognised."}}},{"id":"yk-y06-ai-box-experiment-page-c","subject":"ai-box-experiment-page","predicate":"contests_proposition","object":{"kind":"entity","entityId":"confinement-keeps-it-safe"},"topic":"ideas","statement":"“The AI-Box Experiment” (2002) contests Y06. Explains the rule that the AI party wins only if the gatekeeper voluntarily lets it out, and names the claim the experiment was meant to test. The page describes two 2002 role-play tests in which Yudkowsky played the AI.","time":{"kind":"event","date":"2002","basis":"publication","note":"Page states \"This document is ©2002\"; the current page is a 2020 re-hosting (posted September 4, 2020, updated July 2, 2021) and carries a CC BY-ND 3.0 licence and a MIRI footer, so it was revised after 2002. The 2002 wording was not compared. Precision: year (stated copyright)."},"derivation":"explicit","resolution":"supported","attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","attested":{"date":"2021-07-02","note":"The page states ©2002, but the copy retrieved is a 2020 re-hosting (posted 2020-09-04, updated 2021-07-02) with a later licence and footer; the 2002 wording was not compared."},"engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a smarter-than-human mind confined to a channel with its human operators cannot get out through those operators."}}}},{"id":"yk-y13-sl4-2002-07-02-yudkowsky-0207-4667-b","subject":"sl4-2002-07-02-yudkowsky-0207-4667","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"agi-decades-away"},"topic":"ideas","statement":"The SL4 message “Re: Seed AI (was: How hard a Singularity?)” (2002-07-02) bears on Y13. Replying to Samantha Atkins, who questions his earlier warning against waiting until an AI can read humanity's moral literature, he says Friendliness work started at that stage would come too late and must be present from the beginning.","time":{"kind":"event","date":"2002-07-02","basis":"publication","note":"Archive author index lists the message as \"Tue Jul 02 2002 - 23:01:42 MDT\" under \"Eliezer S. Yudkowsky\"."},"derivation":"explicit","resolution":"supported","attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"About the stage of an AI's development at which Friendliness work must start, not about calendar timelines."}}},{"id":"yk-y14-sl4-2002-09-10-yudkowsky-0209-5398-c","subject":"sl4-2002-09-10-yudkowsky-0209-5398","predicate":"contests_proposition","object":{"kind":"entity","entityId":"fix-problems-as-they-arise"},"topic":"ideas","statement":"The SL4 message “Friendly AI communications issues (was: Consciousness)” (2002-09-10) contests Y14. Writing about Friendly AI problems, he declines to rely on the historical pattern that technological problems get solved as they arise and says he wants to know a solution exists in advance.","time":{"kind":"event","date":"2002-09-10","basis":"publication","note":"Archive author index lists the message as \"Tue Sep 10 2002 - 07:46:27 MDT\" under \"Eliezer S. Yudkowsky\"."},"derivation":"explicit","resolution":"supported","attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that we get to learn from failures with powerful AI and try again, as with other technologies."}}}},{"id":"yk-y16-sl4-2002-06-27-yudkowsky-0206-4453-c","subject":"sl4-2002-06-27-yudkowsky-0206-4453","predicate":"contests_proposition","object":{"kind":"entity","entityId":"give-it-human-values"},"topic":"ideas","statement":"The SL4 message “Re: Military Friendly AI” (2002-06-27) contests Y16. In a thread on military Friendly AI, replying to Ben Goertzel, he argues that present-day researchers probably hold undiscovered moral errors, so an AI's morality must be able to outgrow its programmers' rather than be fixed.","time":{"kind":"event","date":"2002-06-27","basis":"publication","note":"Archive author index lists the message as \"Thu Jun 27 2002 - 11:53:11 MDT\" under \"Eliezer S. Yudkowsky\"."},"derivation":"explicit","resolution":"supported","attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that present-day or programmers' values are settled enough to fix permanently in an AI."}}}},{"id":"yk-y18-sl4-2002-06-23-yudkowsky-0206-4208-c","subject":"sl4-2002-06-23-yudkowsky-0206-4208","predicate":"contests_proposition","object":{"kind":"entity","entityId":"earth-not-worth-taking"},"topic":"ideas","statement":"The SL4 message “Re: Threats to the Singularity.” (2002-06-23) contests Y18. Replying to Ben Goertzel's low-confidence guess that superintelligent Powers might neither help nor attack humans, he says deliberate extermination is unlikely but indifference would probably amount to extermination.","time":{"kind":"event","date":"2002-06-23","basis":"publication","note":"Archive author index lists the message as \"Sun Jun 23 2002 - 08:16:52 MDT\" under \"Eliezer S. Yudkowsky\"."},"derivation":"explicit","resolution":"supported","attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that a superintelligence would leave us a small share, because sparing Earth would cost it very little."}}}},{"id":"yk-y19-sl4-2002-06-23-yudkowsky-0206-4208-c","subject":"sl4-2002-06-23-yudkowsky-0206-4208","predicate":"contests_proposition","object":{"kind":"entity","entityId":"harm-requires-hostility"},"topic":"ideas","statement":"The SL4 message “Re: Threats to the Singularity.” (2002-06-23) contests Y19. Replying to Ben Goertzel, he separates deliberate extermination, which he thinks unlikely, from indifference, which he says would very probably have the same result.","time":{"kind":"event","date":"2002-06-23","basis":"publication","note":"Archive author index lists the message as \"Sun Jun 23 2002 - 08:16:52 MDT\" under \"Eliezer S. Yudkowsky\"."},"derivation":"explicit","resolution":"supported","attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"contrary-only","rationale":"Under the message-level rule, the passage states the contrary conclusion without giving a reason against the premise that an AI would harm humans only out of hostility, even when read with its message or dialogue turn."}}}},{"id":"yk-y21-sl4-2002-03-09-yudkowsky-0203-3148-c","subject":"sl4-2002-03-09-yudkowsky-0203-3148","predicate":"contests_proposition","object":{"kind":"entity","entityId":"human-data-humanlike-mind"},"topic":"ideas","statement":"The SL4 message “Re: Novamente goal system” (2002-03-09) contests Y21. Replying to Ben Goertzel on the Novamente goal system, he lists as an anthropomorphic intuition the expectation that Novamente will be 'socialized' by interacting with humans, and argues that human socialization depends on evolved adaptations an AI would lack.","time":{"kind":"event","date":"2002-03-09","basis":"publication","note":"Archive author index lists the message as \"Sat Mar 09 2002 - 12:56:21 MST\" under \"Eliezer S. Yudkowsky\"."},"derivation":"explicit","resolution":"supported","attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that training on human text or behaviour yields human-like thought and motives."}}}},{"id":"yk-y21-sl4-2002-06-23-yudkowsky-0206-4219-c","subject":"sl4-2002-06-23-yudkowsky-0206-4219","predicate":"contests_proposition","object":{"kind":"entity","entityId":"human-data-humanlike-mind"},"topic":"ideas","statement":"The SL4 message “Re: Seed AI (was: How hard a Singularity?)” (2002-06-23) contests Y21. Replying to Ben Goertzel's view that an AI will absorb human thought-patterns through interaction, he argues that human socialization and empathy rest on evolved, hardwired mechanisms and that an AGI will not acquire them merely by exposure to humans.","time":{"kind":"event","date":"2002-06-23","basis":"publication","note":"Archive author index lists the message as \"Sun Jun 23 2002 - 11:56:42 MDT\" under \"Eliezer S. Yudkowsky\"."},"derivation":"explicit","resolution":"supported","attributes":{"thought":{"roles":["contesting"],"register":"nonfiction","engagement":{"kind":"engages-premise","rationale":"Under the message-level rule, the passage, read with the paragraph, message or dialogue turn it belongs to, gives a reason against the premise that training on human text or behaviour yields human-like thought and motives."}}}},{"id":"yk-y24-sl4-2002-03-10-yudkowsky-0203-3160-b","subject":"sl4-2002-03-10-yudkowsky-0203-3160","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"build-it-before-rivals"},"topic":"ideas","statement":"The SL4 message “Webmind Inc. as social process” (2002-03-10) bears on Y24. Writing to Ben Goertzel about convincing him of Friendly AI, he says SIAI may simply have to finish its own seed AI first, but that he would not mind another project winning if it understood Friendliness.","time":{"kind":"event","date":"2002-03-10","basis":"publication","note":"Archive author index lists the message as \"Sun Mar 10 2002 - 18:33:00 MST\" under \"Eliezer S. Yudkowsky\"."},"derivation":"explicit","resolution":"supported","attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"Being first matters here only through whether the builder understands Friendliness; it does not claim that the builder controls the result."}}},{"id":"yk-y24-sl4-2002-05-03-yudkowsky-0205-3551-a","subject":"sl4-2002-05-03-yudkowsky-0205-3551","predicate":"bears_on_proposition","object":{"kind":"entity","entityId":"build-it-before-rivals"},"topic":"ideas","statement":"The SL4 message “Re: supergoal stability” (2002-05-03) bears on Y24. Replying to Wei Dai on goal-system stability, he refuses to design Friendliness that would serve a programmer's propaganda and says the only thing to do about a project that tries this is to try to beat it to the punch.","time":{"kind":"event","date":"2002-05-03","basis":"publication","note":"Archive author index lists the message as \"Fri May 03 2002 - 16:38:46 MDT\" under \"Eliezer S. Yudkowsky\"."},"derivation":"explicit","resolution":"supported","attributes":{"thought":{"roles":["context"],"register":"nonfiction","missing":"Says the only thing to do about a project designing Friendliness for propaganda is to try to beat it to the punch; it does not state that the builder controls the outcome or that racing to be first improves it."}}},{"id":"ap-eliezer-yudkowsky-ai-box-experiment-page","subject":"eliezer-yudkowsky","predicate":"authored","object":{"kind":"entity","entityId":"ai-box-experiment-page"},"topic":"ideas","statement":"“The AI-Box Experiment” (2002) credits Eliezer S. Yudkowsky as its author.","time":{"kind":"event","date":"2002","basis":"publication","note":"Date the source record gives for this text; authorship is credited in the inspected expression."},"derivation":"explicit","resolution":"supported"},{"id":"ap-eliezer-yudkowsky-sl4-2002-03-08-yudkowsky","subject":"eliezer-yudkowsky","predicate":"authored","object":{"kind":"entity","entityId":"sl4-2002-03-08-yudkowsky"},"topic":"ideas","statement":"The mailing-list message “The \"AI Box\" experiment” (2002-03-08) is sent under the name Eliezer S. Yudkowsky.","time":{"kind":"event","date":"2002-03-08","basis":"publication","note":"Date the source record gives for this text; authorship is credited in the inspected expression."},"derivation":"explicit","resolution":"supported"},{"id":"ap-eliezer-yudkowsky-sl4-2002-03-09-yudkowsky-0203-3148","subject":"eliezer-yudkowsky","predicate":"authored","object":{"kind":"entity","entityId":"sl4-2002-03-09-yudkowsky-0203-3148"},"topic":"ideas","statement":"The mailing-list message “Re: Novamente goal system” (2002-03-09) is sent under the name Eliezer S. Yudkowsky.","time":{"kind":"event","date":"2002-03-09","basis":"publication","note":"Date the source record gives for this text; authorship is credited in the inspected expression."},"derivation":"explicit","resolution":"supported"},{"id":"ap-eliezer-yudkowsky-sl4-2002-03-10-yudkowsky-0203-3160","subject":"eliezer-yudkowsky","predicate":"authored","object":{"kind":"entity","entityId":"sl4-2002-03-10-yudkowsky-0203-3160"},"topic":"ideas","statement":"The mailing-list message “Webmind Inc. as social process” (2002-03-10) is sent under the name Eliezer S. Yudkowsky.","time":{"kind":"event","date":"2002-03-10","basis":"publication","note":"Date the source record gives for this text; authorship is credited in the inspected expression."},"derivation":"explicit","resolution":"supported"},{"id":"ap-eliezer-yudkowsky-sl4-2002-05-03-yudkowsky-0205-3551","subject":"eliezer-yudkowsky","predicate":"authored","object":{"kind":"entity","entityId":"sl4-2002-05-03-yudkowsky-0205-3551"},"topic":"ideas","statement":"The mailing-list message “Re: supergoal stability” (2002-05-03) is sent under the name Eliezer S. Yudkowsky.","time":{"kind":"event","date":"2002-05-03","basis":"publication","note":"Date the source record gives for this text; authorship is credited in the inspected expression."},"derivation":"explicit","resolution":"supported"},{"id":"ap-eliezer-yudkowsky-sl4-2002-06-23-yudkowsky-0206-4208","subject":"eliezer-yudkowsky","predicate":"authored","object":{"kind":"entity","entityId":"sl4-2002-06-23-yudkowsky-0206-4208"},"topic":"ideas","statement":"The mailing-list message “Re: Threats to the Singularity.” (2002-06-23) is sent under the name Eliezer S. Yudkowsky.","time":{"kind":"event","date":"2002-06-23","basis":"publication","note":"Date the source record gives for this text; authorship is credited in the inspected expression."},"derivation":"explicit","resolution":"supported"},{"id":"ap-eliezer-yudkowsky-sl4-2002-06-23-yudkowsky-0206-4219","subject":"eliezer-yudkowsky","predicate":"authored","object":{"kind":"entity","entityId":"sl4-2002-06-23-yudkowsky-0206-4219"},"topic":"ideas","statement":"The SL4 message “Re: Seed AI (was: How hard a Singularity?)” of 2002-06-23, archive message 4219, is sent under the name Eliezer S. Yudkowsky.","time":{"kind":"event","date":"2002-06-23","basis":"publication","note":"Date the source record gives for this text; authorship is credited in the inspected expression."},"derivation":"explicit","resolution":"supported"},{"id":"ap-eliezer-yudkowsky-sl4-2002-06-27-yudkowsky-0206-4453","subject":"eliezer-yudkowsky","predicate":"authored","object":{"kind":"entity","entityId":"sl4-2002-06-27-yudkowsky-0206-4453"},"topic":"ideas","statement":"The mailing-list message “Re: Military Friendly AI” (2002-06-27) is sent under the name Eliezer S. Yudkowsky.","time":{"kind":"event","date":"2002-06-27","basis":"publication","note":"Date the source record gives for this text; authorship is credited in the inspected expression."},"derivation":"explicit","resolution":"supported"},{"id":"ap-eliezer-yudkowsky-sl4-2002-07-02-yudkowsky-0207-4667","subject":"eliezer-yudkowsky","predicate":"authored","object":{"kind":"entity","entityId":"sl4-2002-07-02-yudkowsky-0207-4667"},"topic":"ideas","statement":"The SL4 message “Re: Seed AI (was: How hard a Singularity?)” of 2002-07-02, archive message 4667, is sent under the name Eliezer S. Yudkowsky.","time":{"kind":"event","date":"2002-07-02","basis":"publication","note":"Date the source record gives for this text; authorship is credited in the inspected expression."},"derivation":"explicit","resolution":"supported"},{"id":"ap-eliezer-yudkowsky-sl4-2002-09-10-yudkowsky-0209-5398","subject":"eliezer-yudkowsky","predicate":"authored","object":{"kind":"entity","entityId":"sl4-2002-09-10-yudkowsky-0209-5398"},"topic":"ideas","statement":"The mailing-list message “Friendly AI communications issues (was: Consciousness)” (2002-09-10) is sent under the name Eliezer S. Yudkowsky.","time":{"kind":"event","date":"2002-09-10","basis":"publication","note":"Date the source record gives for this text; authorship is credited in the inspected expression."},"derivation":"explicit","resolution":"supported"},{"id":"ap-eliezer-yudkowsky-sl4-2002-09-19-yudkowsky-0209-5571","subject":"eliezer-yudkowsky","predicate":"authored","object":{"kind":"entity","entityId":"sl4-2002-09-19-yudkowsky-0209-5571"},"topic":"ideas","statement":"The mailing-list message “Re: QUES: CFAI [#2]” (2002-09-19) is sent under the name Eliezer S. Yudkowsky.","time":{"kind":"event","date":"2002-09-19","basis":"publication","note":"Date the source record gives for this text; authorship is credited in the inspected expression."},"derivation":"explicit","resolution":"supported"}],"sources":[],"evidence":[],"at":{"entities":[19,312,379,466,467,468,469,470,471,472,473,474],"claims":[916,989,990,1031,1076,1081,1095,1109,1117,1133,1134,1147,1148,2405,2406,2407,2408,2409,2410,2411,2412,2413,2414,2415],"sources":[],"evidence":[]},"version":"1lg47t7"}