Claim and evidence Close ×
RAND’s red-team study states that it found no statistically significant difference in the viability of the plans produced by its red cells with or without large language model assistance, for the two models and the twelve cells of its main comparison. Copy claim link
Copy claim reference
Machine-checked, not reviewed by an editor Stated in source Supported
Supporting evidence: primary · 1 source; Context: primary · 1 source
Subject version Research report RR-A2977-2, published 25 January 2024 · Published 2024-01-25 · Effective date not established Date 2024-01-25 · publication Evaluated system Two unnamed large language models, referred to as LLM A and LLM B, as available in summer 2023 What the report measured Whether access to a large language model meaningfully increased the feasibility of planning a biological attack, relative to internet access alone By what kind of method A red-team study: teams worked under an internet-only control condition or with model access, and their outputs were scored on a nine-point scale by subject matter experts through an adjudication process The result, in the report’s words No statistically significant difference in the viability of the plans produced with or without model assistance, in the combined analysis and in the per-model comparisons The caveats the report gives The report states that participants might have lacked the expertise to emulate actual attackers, that the study design could have impeded realistic planning, that it did not measure the distance between the capability frontier and the knowledge needed, and that it would widen the test in future work. Scope and limits The result is the study's own, for the two unnamed models as they existed in summer 2023, its participants, its scoring and its vignettes. The report itself lists reasons other than model capability that could account for the absence of a difference, and states that it did not measure the distance between the existing capability frontier and the knowledge that would be needed. A null result here is not evidence that no uplift exists, and nothing is generalised to later models. The main comparison uses twelve red cells; fifteen is the total including the exploratory cells, and the sample framing was corrected at admission (cross-family check, item 14). The study's vignettes and the content of the plans are technical material and are excluded; only the comparative outcome and the report's own limits are recorded.
The evidence 03 primary source Supports the claim
The Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study (RR-A2977-2) ↗ RAND Corporation · Publication date unknown
Copy source reference
Cite the original source Cite the original document, not Selekoa. Check these details against it.
Copy APA
Download BibTeX Download RIS Download CSL-JSON
Source excerpt “We conclude that biological weapon attack planning lies beyond the existing capability frontier of LLMs.” Location in source PDF p. 2, 'We conclude that…' passage in the introductory column
How it was read Retrieved directly from the publisher
Machine-checked · 2026-10-30
Source details and check notes Authors RAND Corporation Version Retrieved for the candidate pack on 2026-10-28 (SHA-256 e09a4103027b4f3c9bd3f8d7e1b7f427fea66f4d636f67c5a254a5f88af9438e). Published PDF of RR-A2977-2, 24 pages, © 2024 Retrieved again at admission on 2026-10-30 (SHA-256 e09a4103027b4f3c9bd3f8d7e1b7f427fea66f4d636f67c5a254a5f88af9438e, the same bytes). Both passes of the candidate pack retrieved this file; the bytes are identical, and one source row carries it. Canonical address Open the canonical source ↗ Accessed 2026-10-28 Language English (en) Attribution RAND Corporation, “The Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study (RR-A2977-2)”. Reuse No open redistribution licence asserted. Bibliographic metadata, editorial paraphrases and short attributed quotations (at most 50 words each). Recheck terms before wider redistribution. Check notes Candidate pack passage, AI and biosecurity (research/ai-bio). The quotation (15 words) matched the text retrieved for the candidate pack on 2026-10-28 and again at admission on 2026-10-30, in a fresh retrieval whose SHA-256 equals the pack’s. Matching ignored differences of whitespace, case, quotation marks, dashes and Unicode compatibility forms, so the source’s typography can differ. The report's stated conclusion, printed alongside the finding of no statistically significant difference. A cross-family machine check (2026-10-30, item 14) re-read the record’s sources and found partial support; its corrections were applied at admission. No human editorial review. primary source Supports the claim
The Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study (RR-A2977-2) ↗ RAND Corporation · Publication date unknown
Copy source reference
Cite the original source Cite the original document, not Selekoa. Check these details against it.
Copy APA
Download BibTeX Download RIS Download CSL-JSON
Source excerpt “there was no significant difference in the viability of OPLANs created with or without the aid of LLMs” Location in source PDF p. 8, analysis of the combined results and of the two models
How it was read Retrieved directly from the publisher
Machine-checked · 2026-10-30
Source details and check notes Authors RAND Corporation Version Retrieved for the candidate pack on 2026-10-28 (SHA-256 e09a4103027b4f3c9bd3f8d7e1b7f427fea66f4d636f67c5a254a5f88af9438e). Published PDF of RR-A2977-2, 24 pages, © 2024 Retrieved again at admission on 2026-10-30 (SHA-256 e09a4103027b4f3c9bd3f8d7e1b7f427fea66f4d636f67c5a254a5f88af9438e, the same bytes). Both passes of the candidate pack retrieved this file; the bytes are identical, and one source row carries it. Canonical address Open the canonical source ↗ Accessed 2026-10-28 Language English (en) Attribution RAND Corporation, “The Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study (RR-A2977-2)”. Reuse No open redistribution licence asserted. Bibliographic metadata, editorial paraphrases and short attributed quotations (at most 50 words each). Recheck terms before wider redistribution. Check notes Candidate pack passage, AI and biosecurity (research/ai-bio). The quotation (18 words) matched the text retrieved for the candidate pack on 2026-10-28 and again at admission on 2026-10-30, in a fresh retrieval whose SHA-256 equals the pack’s. Matching ignored differences of whitespace, case, quotation marks, dashes and Unicode compatibility forms, so the source’s typography can differ. 'OPLAN' is the report's term for the operational plans its cells produced. A cross-family machine check (2026-10-30, item 14) re-read the record’s sources and found partial support; its corrections were applied at admission. No human editorial review. primary source Context only
The Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study (RR-A2977-2) ↗ RAND Corporation · Publication date unknown
Copy source reference
Cite the original source Cite the original document, not Selekoa. Check these details against it.
Copy APA
Download BibTeX Download RIS Download CSL-JSON
Source excerpt “Several factors beyond the capabilities or limitations of LLMs could account for this lack of statistically significant difference between the LLM and internet-only cells.” Location in source PDF p. 8, paragraph following the difference-in-means analysis
How it was read Retrieved directly from the publisher
Machine-checked · 2026-10-30
Source details and check notes Authors RAND Corporation Version Retrieved for the candidate pack on 2026-10-28 (SHA-256 e09a4103027b4f3c9bd3f8d7e1b7f427fea66f4d636f67c5a254a5f88af9438e). Published PDF of RR-A2977-2, 24 pages, © 2024 Retrieved again at admission on 2026-10-30 (SHA-256 e09a4103027b4f3c9bd3f8d7e1b7f427fea66f4d636f67c5a254a5f88af9438e, the same bytes). Both passes of the candidate pack retrieved this file; the bytes are identical, and one source row carries it. Canonical address Open the canonical source ↗ Accessed 2026-10-28 Language English (en) Attribution RAND Corporation, “The Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study (RR-A2977-2)”. Reuse No open redistribution licence asserted. Bibliographic metadata, editorial paraphrases and short attributed quotations (at most 50 words each). Recheck terms before wider redistribution. Check notes Candidate pack passage, AI and biosecurity (research/ai-bio). The quotation (24 words) matched the text retrieved for the candidate pack on 2026-10-28 and again at admission on 2026-10-30, in a fresh retrieval whose SHA-256 equals the pack’s. Matching ignored differences of whitespace, case, quotation marks, dashes and Unicode compatibility forms, so the source’s typography can differ. The report then lists those factors. A cross-family machine check (2026-10-30, item 14) re-read the record’s sources and found partial support; its corrections were applied at admission. No human editorial review. Status and meaning Editorial status. Not reviewed by an editor . An exploratory record that no editor has accepted. This status does not say whether a person has read any linked passage. Each evidence record above shows how it was checked.
Relation. A report states the result of an evaluation it ran or commissioned, in its own words, with what was measured, by what kind of method, and the caveats the report itself gives (schema 2.22). The record is the report’s statement, not a verified finding: it says nothing about another system, a later version or real-world risk, and a null or limiting result is not evidence that no effect exists.
Date. The result as published on 25 January 2024; the report describes the models as those existing as of summer 2023.
Record ID ab-ev-rand-redteam-no-difference