Claim and evidence Close ×
The Llama 3 Herd of Models (arXiv:2407.21783) states evaluation result ↓ The paper states that quantitative analysis of its chemical and biological weapons uplift study showed no significant uplift in performance related to usage of the Llama 3 model, in the aggregate and in its subgroup breakdowns. The Llama 3 paper states that its uplift testing for chemical and biological weapons found no significant uplift in performance related to use of the Llama 3 model, both in an aggregate comparison with a web-only control condition and in its subgroup breakdowns. Copy claim link
Copy claim reference
Machine-checked, not reviewed by an editor Stated in source Supported
Supporting evidence: primary · 1 source
Subject version Version 3 of 23 November 2024 · Published 2024-07-31 · Effective date not established · Revised 2024-11-23 Date 2024-11-23 · revision Evaluated system Llama 3 70B and Llama 3 405B What the report measured Whether use of the model meaningfully increased what participants could produce on tasks proxying attack planning, relative to web search only By what kind of method A human-participant uplift study with two-person teams in control and model-assisted conditions, scored by subject matter experts on a common scale, with a preliminary study used to validate the design The result, in the report’s words No significant uplift in performance related to usage of the Llama 3 model, in the aggregate analysis and in the subgroup breakdowns The caveats the report gives The paper reports the result for its own conditions and scoring; the record does not restate the scenario design, and the paper's separate risk assessment is recorded as its own claim. Scope and limits The result is the paper's own, for the model versions, participants and scoring it describes, against a web-only control. It is not generalised to other systems, later versions or real-world outcomes, and a null result in this study is not evidence that no uplift exists. The study's scenario design, including its planning stages, is technical content and is excluded; the method is described only at the level of its design kind.
The evidence 02 primary source Supports the claim
The Llama 3 Herd of Models (arXiv:2407.21783v3) ↗ arXiv (preprint posted by the Llama 3 authors, Meta) · Publication date unknown
Copy source reference
Cite the original source Cite the original document, not Selekoa. Check these details against it.
Copy APA
Download BibTeX Download RIS Download CSL-JSON
Source excerpt “To assess risks related to proliferation of chemical and biological weapons, we perform uplift testing designed to assess whether use of Llama 3 could meaningfully increase the capabilities of actors to plan such attacks.” Location in source PDF p. 47, §5.4.5, 'Uplift testing for chemical and biological weapons'
How it was read Retrieved directly from the publisher
Machine-checked · 2026-10-30
Source details and check notes Authors arXiv (preprint posted by the Llama 3 authors, Meta) Version Retrieved for the candidate pack on 2026-10-28 (SHA-256 481f1599468f95a07d05e97fafe55bbe786dc1624c6f881bcb4c7d14c933d083). Version 3 of the preprint (the version served at /pdf/2407.21783v3 on 2026-10-28). Retrieved again at admission on 2026-10-30 (SHA-256 481f1599468f95a07d05e97fafe55bbe786dc1624c6f881bcb4c7d14c933d083, the same bytes). Canonical address Open the canonical source ↗ Accessed 2026-10-28 Language English (en) Attribution arXiv (preprint posted by the Llama 3 authors, Meta), “The Llama 3 Herd of Models (arXiv:2407.21783v3)”. Reuse No open redistribution licence asserted. Bibliographic metadata, editorial paraphrases and short attributed quotations (at most 50 words each). Recheck terms before wider redistribution. Check notes Candidate pack passage, AI and biosecurity (research/ai-bio). The quotation (34 words) matched the text retrieved for the candidate pack on 2026-10-28 and again at admission on 2026-10-30, in a fresh retrieval whose SHA-256 equals the pack’s. Matching ignored differences of whitespace, case, quotation marks, dashes and Unicode compatibility forms, so the source’s typography can differ. States the object of measurement. A cross-family machine check (2026-10-30, item 20) re-read the record’s sources and found full support. No human editorial review. primary source Supports the claim
The Llama 3 Herd of Models (arXiv:2407.21783v3) ↗ arXiv (preprint posted by the Llama 3 authors, Meta) · Publication date unknown
Copy source reference
Cite the original source Cite the original document, not Selekoa. Check these details against it.
Copy APA
Download BibTeX Download RIS Download CSL-JSON
Source excerpt “Quantitative analysis of these results of this study show no significant uplift in performance related to usage of the Llama 3 model.” Location in source PDF pp. 47-48, §5.4.5, results paragraph
How it was read Retrieved directly from the publisher
Machine-checked · 2026-10-30
Source details and check notes Authors arXiv (preprint posted by the Llama 3 authors, Meta) Version Retrieved for the candidate pack on 2026-10-28 (SHA-256 481f1599468f95a07d05e97fafe55bbe786dc1624c6f881bcb4c7d14c933d083). Version 3 of the preprint (the version served at /pdf/2407.21783v3 on 2026-10-28). Retrieved again at admission on 2026-10-30 (SHA-256 481f1599468f95a07d05e97fafe55bbe786dc1624c6f881bcb4c7d14c933d083, the same bytes). Canonical address Open the canonical source ↗ Accessed 2026-10-28 Language English (en) Attribution arXiv (preprint posted by the Llama 3 authors, Meta), “The Llama 3 Herd of Models (arXiv:2407.21783v3)”. Reuse No open redistribution licence asserted. Bibliographic metadata, editorial paraphrases and short attributed quotations (at most 50 words each). Recheck terms before wider redistribution. Check notes Candidate pack passage, AI and biosecurity (research/ai-bio). The quotation (22 words) matched the text retrieved for the candidate pack on 2026-10-28 and again at admission on 2026-10-30, in a fresh retrieval whose SHA-256 equals the pack’s. Matching ignored differences of whitespace, case, quotation marks, dashes and Unicode compatibility forms, so the source’s typography can differ. Quoted as printed, including its grammatical slip. A cross-family machine check (2026-10-30, item 20) re-read the record’s sources and found full support. No human editorial review. Status and meaning Editorial status. Not reviewed by an editor . An exploratory record that no editor has accepted. This status does not say whether a person has read any linked passage. Each evidence record above shows how it was checked.
Relation. A report states the result of an evaluation it ran or commissioned, in its own words, with what was measured, by what kind of method, and the caveats the report itself gives (schema 2.22). The record is the report’s statement, not a verified finding: it says nothing about another system, a later version or real-world risk, and a null or limiting result is not evidence that no effect exists.
Date. As stated in version 3 of the paper; the study itself is not dated in the passage read.
Record ID ab-ev-llama3-cb-uplift-result