Claim and evidence Close ×
The joint pre-deployment evaluation summary for the upgraded Claude 3.5 Sonnet states that US AISI piloted an evaluation method that augments the model with access to bioinformatic tools, and that when tested with Sonnet 3.5 this resulted in increased performance beyond the model alone, at times exceeding measured human expert baselines. Copy claim link
Copy claim reference
Machine-checked, not reviewed by an editor Stated in source Supported
Supporting evidence: primary · 1 source
Date 2024-11-19 · publication Evaluated system The upgraded Claude 3.5 Sonnet, augmented with access to bioinformatic tools What the report measured Whether augmenting the model with access to bioinformatic tools changes its performance on biology research task questions By what kind of method A piloted evaluation method giving the model access to such tools, compared with the model alone The result, in the report’s words Increased performance beyond the model alone, at times exceeding measured human expert baselines The caveats the report gives Described as a pilot; the summary publishes no figures, states that the findings are from US AISI's testing only and that smaller differences may fall within the margins of error. Scope and limits A piloted method and its stated outcome, for this model and these question sets only. It is not a finding about real-world capability, and the summary gives no figures; the same caveats as the summary's other biological findings apply, including that they rest on US AISI's testing alone. The tools are referred to only as the summary refers to them; no tool, task or workflow is recorded.
The evidence 01 primary source Supports the claim
Pre-Deployment Evaluation of Anthropic’s Upgraded Claude 3.5 Sonnet (November 19, 2024) ↗ National Institute of Standards and Technology · 2024-11-19
Archived copy, captured 2024-11-19: Pre-Deployment Evaluation of Anthropic’s Upgraded Claude 3.5 Sonnet (November 19, 2024) ↗
Copy source reference
Cite the original source Cite the original document, not Selekoa. Check these details against it.
Copy APA
Download BibTeX Download RIS Download CSL-JSON
Source excerpt “US AISI piloted an evaluation method that augments the AI model by providing it access to bioinformatic tools to assist in research task questions, which when tested using Sonnet 3.5 resulted in increased performance beyond the model alone, at times exceeding measured human expert baselines.” Location in source 'Biological Capabilities' section, key findings, second bullet
How it was read Retrieved directly from the publisher
Machine-checked · 2026-10-30
Source details and check notes Authors National Institute of Standards and Technology (U.S. AI Safety Institute) Version Retrieved on 2026-10-27 (SHA-256 1b9a64eb9d1953bdd916767279b7021cabc79fc837018b2f75fc3ccf2c8f983a). The cross-family check of 2026-10-28 opened the Internet Archive capture linked here and matched the quoted wording in it, the release-period expression (decision 9); a capture shows only what the archive retrieved that day. Canonical address Open the canonical source ↗ Accessed 2026-10-27 Language English (en) Archive copy Open archived copy, captured 2024-11-19: Pre-Deployment Evaluation of Anthropic’s Upgraded Claude 3.5 Sonnet (November 19, 2024) ↗ . Captured on the recorded publication date. An archive copy shows only what the archive retrieved on 2024-11-19. It does not prove what the page said earlier, and it can differ from the copy checked on 2026-10-27.Attribution National Institute of Standards and Technology (U.S. AI Safety Institute), “Pre-Deployment Evaluation of Anthropic’s Upgraded Claude 3.5 Sonnet (November 19, 2024)”, 2024-11-19. Reuse No open redistribution licence asserted. Bibliographic metadata, editorial paraphrases and short attributed quotations (at most 50 words each). Recheck terms before wider redistribution. Check notes Candidate pack passage, AI and biosecurity (research/ai-bio). The quotation (45 words) matched the text retrieved for the candidate pack on 2026-10-28 and again at admission on 2026-10-30, in a fresh retrieval (SHA-256 8e9a0960ed43d37fa5d161337886069510323506e77889b1301eb294c16e4b90; the bytes differ from the pack’s retrieval). Matching ignored differences of whitespace, case, quotation marks, dashes and Unicode compatibility forms, so the source’s typography can differ. A cross-family machine check (2026-10-30, item 19) re-read the record’s sources and found full support. No human editorial review. Status and meaning Editorial status. Not reviewed by an editor . An exploratory record that no editor has accepted. This status does not say whether a person has read any linked passage. Each evidence record above shows how it was checked.
Relation. A report states the result of an evaluation it ran or commissioned, in its own words, with what was measured, by what kind of method, and the caveats the report itself gives (schema 2.22). The record is the report’s statement, not a verified finding: it says nothing about another system, a later version or real-world risk, and a null or limiting result is not evidence that no effect exists.
Date. The finding as published on 19 November 2024. The expression checked is the copy served on 2026-10-28, whose footer reads 'Released November 19, 2024, Updated March 20, 2025'; what the page was changed in on that date is not stated, and the record keeps the release date, not the update date.
Record ID ab-ev-usaisi-claude35-tooling-pilot