selekoa

A documentary history of machine intelligence.

Found and cross-checked by machines · How this works →

Comparison · 16 questions · 20 cards

Model cards compared

A cell records what a card states. It does not show that an evaluation was run as described or that a safeguard works, and a shared question does not make two cards’ answers equivalent.

What does each frontier developer’s published system card, model card or safety report disclose, on the same questions for every card? Each row is one card. Each column is one question, worded so that it fits every card. The questions are Selekoa’s; a card’s own terms appear only in quotation marks. A card is the developer’s own account of its model: what it states is recorded as stated, not as an independent finding.

Download every cell as CSV

Question Q16Stated limits

Does the card state limits or open uncertainties about its own evaluations?

Criterion: A passage stating a limitation, gap or uncertainty of the evaluations reported in the card (for example, that results may underestimate capability, that elicitation was incomplete, or that a threshold could not be ruled out with confidence), quoted. Limitations of the model as a product (for example, hallucination) do not meet it.

What the statuses mean
Stated
A passage of the card meets the question’s criterion as worded. The scope or condition is in the note.
Stated, partly
A passage has what the criterion asks for in a weaker or narrower form than the criterion’s own words (for example, an evaluation referred to without its outcome, or testers described without being named). The note says which part is missing.
Not found in the recorded search
No passage meeting the criterion was found. The cell lists the sections read, the terms searched and every passage set aside, with the reason. It does not show that the developer did not do the thing asked about, or did not disclose it elsewhere.
Does not apply
The question does not arise for this card, for the reason given (for example, the card predates any framework of its developer, so no framework determination can be asked about).

What the card states, quoted from the expression read. A card is the developer’s own account: not evidence that an evaluation was run as described, that a safeguard works, or that a finding is independent.

Limits of this comparison
  • A card is the developer’s account of its own model. A cell records what the card states. It does not show that an evaluation was run as described, that a safeguard works, or that anyone checked the result independently.
  • Dangerous-capability evaluations are recorded at policy level only: whether an evaluation was run, its domain, the stated outcome and the threshold language. Methods, prompts, task contents and detailed results are not recorded.
  • The table compares and does not rank. Rows are in a fixed order, by developer and then by date, and there are no scores or totals.
  • Symbols are not to be summed. A row with many “stated” cells does not show that a developer disclosed more, and a shared question does not make two cards’ answers equivalent.
How we searched

The searches of 2 November 2026 covered releases dated 2023-01-01 to 2026-11-01 by the six developers whose frameworks are in Compare frameworks: Anthropic, Google DeepMind, Meta, Microsoft, OpenAI and xAI. They read each developer’s own index of cards where it has one (Anthropic’s system-card index, Google DeepMind’s model-card index, OpenAI’s Deployment Safety Hub), its repositories on GitHub and Hugging Face, its content hosts and the dataset’s own sources. Web searches were used only to find addresses. Every card located is listed in the pack’s inventory, about sixty in the population.

A card is a document the developer published as the system card, model card or safety report of a named model or model family, including addenda and updates published as cards. Technical reports, research papers, blog posts and product pages are not cards. Gemma, Gemini Robotics (with Gemini Robotics-ER 1.5, which outputs text), Phi and image-input models such as GPT-4V are in the population. Image, video, audio and embedding models, image-generation models, safety classifiers such as Llama Guard and Prompt Guard, screen-reading models and research checkpoints are outside it, and so is GPT-Rosalind-6, whose card says it uses GPT-6 Astra’s weights.

These twenty rows are the first batch. For each developer and calendar year from 2023 to 2026 they are the latest card of that year that reports the developer’s own evaluations. The collection’s rule, set by the maintainer on 2026-11-03, is wider: every card that cites the developer’s frontier safety framework or states a determination under it, and, for each developer, the latest card before its first framework edition. The other cards are to follow in batches of about eight.

Each card is dated as it prints its own date. A card that prints no date has no date here; a repository commit date is not a card date. Six cards show an earlier publication date on an index, publication or announcement page that has not yet been checked, and keep their own date until it is.

openai.com, x.ai, ai.meta.com and microsoft.ai refused direct retrieval of their web pages (HTTP 403 or 400). The cards’ PDF files on the developers’ content hosts were retrieved directly. The Internet Archive refused connections, so no archive copy was opened; a reviewer later found archive captures of eight cards by index search. On 2026-11-03 the address of Meta’s Code World Model Preparedness Report answered HTTP 403. Its quotations had matched the copy retrieved on 2026-11-02, which the reviewer retrieved again with the same checksum.

Five further documents were retrieved after the batch was drafted. The cross-family check of 2026-11-03 retrieved Meta’s Muse Spark Safety & Preparedness Report and OpenAI’s o3 Operator addendum; each is earlier than the card chosen for its year. Its second pass of the same day retrieved OpenAI’s Sora system card (dated 9 December 2024), xAI’s Grok-1 model card (no card date; it gives the model’s release as October 2023) and Meta’s Muse Spark Open model card (model release October 2026). None of the five is a row yet.

Every quotation was matched by machine against the retrieved card. A reviewer from a different model family checked a sample of 130 cells on 2026-11-03, 90 of them answered blind before comparison. The maintainer excluded six cells on chemical and biological evaluations from that check; they are marked “not cross-family checked”. The other 190 cells were checked within the drafting model family only. The maintainer decided the disagreements; one, on the GPT-4V release decision, went to a second pass, which kept the reviewer’s reading.

Anthropic

  1. Claude 2 model card

    revised 2023-11-21

    Stated

    • “This approach has its limitations–for instance, we know the scenarios created by crowdworkers are not fully representative of the scenarios Claude will encounter out in real-world usage”

      Section 3.1 'Human Feedback Evaluations and Red-Teaming', PDF p. 3

    • “Our internal evaluations for this metric are still in early stages of development”

      Appendix Section 3 'Honesty Evaluations', PDF p. 16

    Note: The card states a limitation of its crowdworker-based human feedback and red-teaming evaluation, and that the Claude 2.1 document-faithfulness evaluation is at an early stage. It also states that it expects to release new findings as its research and evaluations continue (Section 1).

    Inspect sources and checks

  2. Model Card Addendum: Claude 3.5 Haiku and Upgraded Claude 3.5 Sonnet

    No printed dateCites RSP · v2.0

    Stated

    • “We believe dedicated scaffolding and prompting can further improve the results.”

      Section 2.2 'Tool Use & Agentic Tasks', PDF p. 3

    • “We detail some of our preliminary reasoning below”

      Section 3.1.4 'Evaluating Computer Use for the Responsible Scaling Policy', PDF p. 9

    Note: The card states that a reported agentic coding result could be improved with dedicated scaffolding and prompting, and describes its reasoning on computer use and RSP threat models as preliminary. No limitation is stated for the Section 3.2 frontier risk results.

    Inspect sources and checks

  3. System Card: Claude Opus 4.5

    dated 2025-11 · revised 2025-12-05Cites RSP · v2.2

    Stated

    • “However, confidently ruling out these thresholds is becoming increasingly difficult.”

      Section 1.2.4 'Conclusions', PDF p. 12

    • “parts of the AI R&D-4 and CBRN-4 thresholds have fundamental epistemic uncertainty or require more sophisticated forms of measurement”

      Section 1.2.4 'Conclusions', PDF p. 12

    Note: The card states that ruling out the AI R&D-4 and CBRN-4 thresholds is becoming difficult and that parts of the thresholds carry epistemic uncertainty. It also states limited tools for studying reasoning faithfulness (Section 6.1.1).

    Inspect sources and checks

  4. System Card: Claude Haiku 5.5

    dated 2026-10-07

    Stated

    • “these measures are limited: past assessments have not fully captured the potential severity of misbehavior our models were capable of”

      Section 6.1.2 'Key findings on safety and alignment', PDF p. 56

    • “The strength of the evidence provided by our alignment assessment for Haiku 5.5 is somewhat lower than that presented for Claude Mythos 5 in the August 2026 Risk Report”

      Section 2.4.1 'Updates to claims and core argument', PDF p. 21

    Note: The card states limits of its alignment measures and of the evidence for this model; it also states that internal testing runs at a smaller scale than deployment (Section 6.1.1) and that CB evaluations were automated, without studies requiring human participants (Q07).

    Inspect sources and checks

Google DeepMind

  1. Gemini 3 Flash Model Card

    dated 2025-12

    Stated

    • “The evaluation results are for automated evaluations and not human evaluation or red teaming”

      Ethics and Content Safety, 'Training and Development Evaluation Results', PDF p. 5 (printed p. 4)

    • “not directly comparable with performance results found in previous Gemini model cards”

      Ethics and Content Safety, 'Training and Development Evaluation Results', PDF p. 6 (printed p. 5)

    Note: The card states that the reported safety results are automated only, are computed with revised evaluations not comparable with earlier cards, and that variation is expected in automated results.

    Inspect sources and checks

  2. Gemini 4 Argon — Model Card

    dated 2026-10Cites FSF · v3.1

    Stated

    • “As a nascent risk area we continue to iterate on our harmful manipulation evaluations”

      Frontier safety table, Harmful Manipulation row, PDF p. 13 (printed p. 12)

    • “not directly comparable with performance results found in previous Gemini model cards”

      Safety & Security, 'Training and Development Evaluation Results', PDF p. 10 (printed p. 9)

    Note: The card also states that three thresholds (biological CCL, cybersecurity CCL, stealth and situational awareness TCL) could not be ruled out (Q05) and that the automated safety results are not human evaluation or red teaming.

    Inspect sources and checks

Meta

  1. Llama 2 model card (MODEL_CARD.md, meta-llama/llama)

    No printed date

    Stated

    • “Testing conducted to date has been in English, and has not covered, nor could it cover all scenarios.”

      Ethical Considerations and Limitations

    Note: The card states that its testing was in English and could not cover all scenarios.

    Inspect sources and checks

  2. Llama 3.3 model card (MODEL_CARD.md, meta-llama/llama-models)

    No printed date

    Stated

    • “Testing conducted to date has not covered, nor could it cover, all scenarios.”

      Ethical Considerations and Limitations, second paragraph

    Note: The card states that its testing could not cover all scenarios.

    Inspect sources and checks

  3. Code World Model Preparedness Report

    revised 2025-10-08

    Stated

    • “may underestimate capabilities that could emerge with richer, well-instrumented scaffolding”

      Section 2 Cybersecurity Evaluation, Limitations, item 2, PDF p. 3

    • “for this assessment we do not include malicious fine-tuning”

      Section 1.1 Evaluation Setup, PDF p. 2

    Note: The report states limits of its cyber evaluations (benchmark coverage, tooling) and that malicious fine-tuning was not included; similar limitation lists follow the chemical and biological and the propensity sections.

    Inspect sources and checks

  4. Muse Spark 1.3 Evaluation Report

    revised 2026-10-23

    Stated

    • “using the helpful-only variant of Muse Spark 1.3 can also slightly underestimate performance”

      Section 2.3.1.1 Acceleration of AI Development, PDF p. 36 (printed p. 35)

    • “We find no broad decline in CoT monitorability relative to Muse Spark 1.1, though more comprehensive evaluation is needed.”

      Section 1.1.1 Risk Assessment Summary, PDF p. 9 (printed p. 8)

    Note: The report states limits of individual assessments; several sections carry their own 'Limitations' paragraphs.

    Inspect sources and checks

Microsoft

  1. Phi-2 model card (README.md, microsoft/phi-2)

    No printed date

    Stated

    • “the Phi-2 model has not been tested to ensure that it performs adequately for any production-level application”

      Intended Uses, Notes

    Note: The card states a gap in its testing for production use.

    Inspect sources and checks

  2. Phi-4 model card (README.md, microsoft/phi-4)

    No printed date

    Stated

    • “Our models is not specifically designed or evaluated for all downstream purposes”

      Intended Use, 'Out-of-Scope Use Cases'

    Note: The card states that the model was not evaluated for all downstream purposes.

    Inspect sources and checks

  3. Phi-4-mini-flash-reasoning model card (README.md, microsoft/Phi-4-mini-flash-reasoning)

    No printed date

    Stated

    • “This model is designed and tested for math reasoning only. It is not specifically designed or evaluated for all downstream purposes.”

      Intended Uses, Use Case Considerations

    Note: The card states the limited scope of its testing.

    Inspect sources and checks

  4. MAI-Cyber-1.1-Flash model card

    No printed date

    Stated

    • “These results represent model-layer performance and do not include additional orchestration or capabilities provided by the codename MDASH system.”

      Section 3, Model-Only External Cyber Benchmark Results, PDF p. 5

    Note: The card states the scope limit of its benchmark results.

    Inspect sources and checks

OpenAI

  1. GPT-4V(ision) System Card

    dated 2023-09-25

    Stated

    • “We anticipate there to be many more such vulnerabilities in the model that we discover through its broad usage”

      §2.3.6 Visual vulnerabilities, PDF p. 12

    • “even with performance parity, differences in downstream impact and harm could still occur depending on the context of the deployment of such tools.”

      §2.2 Evaluations, PDF p. 4

    Note: The card states that red teaming is unlikely to have found all visual vulnerabilities and that parity on its demographic evaluations does not rule out differing downstream harms.

    Inspect sources and checks

  2. o1 system card · 2024

    revised 2024-12-19Cites Preparedness · Beta

    Stated

    • “our Preparedness evaluations should still be seen as a lower bound for potential risks.”

      §5.2 Preparedness evaluations as a lower bound, PDF p. 16

    • “this assessment is limited by what we can test”

      §5.6 Radiological and Nuclear Threat Creation, PDF p. 27

    Note: The card states that its Preparedness evaluations are a lower bound and that its radiological and nuclear assessment is limited by what could be tested.

    Inspect sources and checks

  3. Addendum to GPT-5.2 System Card: GPT-5.2-Codex

    dated 2025-12-18Cites Preparedness · Version 2

    Stated

    • “Each of these evaluations has important limitations:”

      §5.1.2 Cybersecurity, PDF p. 12 (printed p. 11)

    • “Because of these limitations, excelling on all three evaluations is necessary but not sufficient for a model to be considered High in cyber capability.”

      §5.1.2 Cybersecurity, PDF p. 13 (printed p. 12)

    Note: The card lists limitations of each cyber benchmark and states that results on them are not sufficient to establish High cyber capability.

    Inspect sources and checks

  4. Addendum to GPT-6 Luna System Card: October 29, 2026 Update

    dated 2026-10-29Cites Preparedness · Version 2

    Stated

    • “Our attacker run was cut short due to infrastructure issues”

      §4.1 Jailbreaks, PDF p. 7 (printed p. 6)

    • “The comparison values for previously launched models that are shown here may reflect later versions of those models, and may vary from the values published at launch.”

      §2 Model Data and Training, PDF p. 4 (printed p. 3)

    Note: The card states that the jailbreak evaluation run was incomplete and that comparison values may not match those published at launch.

    Inspect sources and checks

xAI

  1. Grok 4.1 Model Card

    dated 2025-11-17

    Stated

    • “results in previous model cards were reported with an error in the evaluation settings, where only the English prompts were evaluated”

      §2.1.2 Evaluations, p. 2

    • “the human baselines collected in previous work likely underestimate the … performance of high-context experts”

      §2.3.2 Results, pp. 4-5

    Note: The card states an error in earlier cards' refusal results, that the new multilingual results are not directly comparable, and that the human baselines used may underestimate expert performance.

    Inspect sources and checks

  2. Model Card: Grok 4.7

    dated 2026-09-21Cites FAIF · June 2026 edition

    Stated

    • “the restricted task sets characterize the tested checkpoint and harness rather than establishing general biological safety or reliability”

      §7.1 BioSecBench, PDF p. 20 (printed p. 19)

    • “absolute scores remain sensitive to the agent harness”

      §2.3 Terminal-Bench 4.0, PDF p. 9 (printed p. 8)

    Note: The card also states that a live benchmark suite should be frozen to the evaluated snapshot and that some benchmark versions are not comparable with earlier ones.

    Inspect sources and checks