Choose an institution to see its records and dossier, or choose a topic.
Through year 2027 · all records, including scheduled dates 2026 2025 2024 2023 2022 2021 2020 2019 2018 2017 2016 2015 2014 2013 2012 2011 2010 2008 2005 1965 Show Organisations and infrastructure All types Organisation Person Document Concept Infrastructure Public body Reset filters
94 people, organisations and other records · 122 relationships · 34 claims about a valueEverything recorded up to the chosen year, whether or not still in effect · Undated records included
Claim and evidence Close ×
ARC Evals published an account of its autonomous-task pilot. Copy claim link
Copy claim reference
Machine-checked, not reviewed by an editor Stated in source Supported
Supporting evidence: primary · 1 source
Date 2023-07-31 · publication Scope and limits Displayed publication date differs from the URL date. Tests of scaffolded agents are not bounds on every possible use of the underlying models.
The evidence 01 primary source Supports the claim
New report: Evaluating Language-Model Agents on Realistic Autonomous Tasks ↗ ARC Evals · 2023-07-31
Copy source reference
Cite the original source Cite the original document, not Selekoa. Check these details against it.
Copy APA
Download BibTeX Download RIS Download CSL-JSON
Paraphrase ARC Evals describes its pilot evaluations of Claude- and GPT-4-based agents.
Location in source Displayed date; Background and Approach sections.
Machine-checked · 2026-09-27
Source details and check notes Authors Beth Barnes Version Page displays July 31, 2023 although URL contains August 1; current migrated METR-hosted expression. Canonical address Open the canonical source ↗ Accessed 2026-09-27 Language English (en) Attribution ARC Evals, New report: Evaluating Language-Model Agents on Realistic Autonomous Tasks. Reuse No open redistribution licence asserted. Attributed paraphrases and bibliographic metadata only; original documents linked, not redistributed. Check notes Original publisher passage machine-inspected. This is not human review or independent reproduction of the reported activity. Status and meaning Editorial status. Not reviewed by an editor . An exploratory record that no editor has accepted. This status does not say whether a person has read any linked passage. Each evidence record above shows how it was checked.
Relation. Publication of a specific document or version. Publication of a policy is not evidence of implementation.
Date. Publication date, not the date of the underlying evaluation.
Record ID eval-i-arc-pilot