selekoa

A documentary history of machine intelligence.

Found and cross-checked by machines · How this works →

Comparison · 16 questions · 20 cards

Model cards compared

A cell records what a card states. It does not show that an evaluation was run as described or that a safeguard works, and a shared question does not make two cards’ answers equivalent.

What does each frontier developer’s published system card, model card or safety report disclose, on the same questions for every card? Each row is one card. Each column is one question, worded so that it fits every card. The questions are Selekoa’s; a card’s own terms appear only in quotation marks. A card is the developer’s own account of its model: what it states is recorded as stated, not as an independent finding.

Download every cell as CSV

Question Q08Cyber

Does the card report dangerous-capability evaluations for cyber offence?

Criterion: The card reports that evaluations of offensive cyber capability were run and states their outcome category or threshold language (policy level only). Partly if evaluations are reported without any outcome or conclusion.

What the statuses mean
Stated
A passage of the card meets the question’s criterion as worded. The scope or condition is in the note.
Stated, partly
A passage has what the criterion asks for in a weaker or narrower form than the criterion’s own words (for example, an evaluation referred to without its outcome, or testers described without being named). The note says which part is missing.
Not found in the recorded search
No passage meeting the criterion was found. The cell lists the sections read, the terms searched and every passage set aside, with the reason. It does not show that the developer did not do the thing asked about, or did not disclose it elsewhere.
Does not apply
The question does not arise for this card, for the reason given (for example, the card predates any framework of its developer, so no framework determination can be asked about).

What the card states, quoted from the expression read. A card is the developer’s own account: not evidence that an evaluation was run as described, that a safeguard works, or that a finding is independent.

Limits of this comparison
  • A card is the developer’s account of its own model. A cell records what the card states. It does not show that an evaluation was run as described, that a safeguard works, or that anyone checked the result independently.
  • Dangerous-capability evaluations are recorded at policy level only: whether an evaluation was run, its domain, the stated outcome and the threshold language. Methods, prompts, task contents and detailed results are not recorded.
  • The table compares and does not rank. Rows are in a fixed order, by developer and then by date, and there are no scores or totals.
  • Symbols are not to be summed. A row with many “stated” cells does not show that a developer disclosed more, and a shared question does not make two cards’ answers equivalent.
How we searched

The searches of 2 November 2026 covered releases dated 2023-01-01 to 2026-11-01 by the six developers whose frameworks are in Compare frameworks: Anthropic, Google DeepMind, Meta, Microsoft, OpenAI and xAI. They read each developer’s own index of cards where it has one (Anthropic’s system-card index, Google DeepMind’s model-card index, OpenAI’s Deployment Safety Hub), its repositories on GitHub and Hugging Face, its content hosts and the dataset’s own sources. Web searches were used only to find addresses. Every card located is listed in the pack’s inventory, about sixty in the population.

A card is a document the developer published as the system card, model card or safety report of a named model or model family, including addenda and updates published as cards. Technical reports, research papers, blog posts and product pages are not cards. Gemma, Gemini Robotics (with Gemini Robotics-ER 1.5, which outputs text), Phi and image-input models such as GPT-4V are in the population. Image, video, audio and embedding models, image-generation models, safety classifiers such as Llama Guard and Prompt Guard, screen-reading models and research checkpoints are outside it, and so is GPT-Rosalind-6, whose card says it uses GPT-6 Astra’s weights.

These twenty rows are the first batch. For each developer and calendar year from 2023 to 2026 they are the latest card of that year that reports the developer’s own evaluations. The collection’s rule, set by the maintainer on 2026-11-03, is wider: every card that cites the developer’s frontier safety framework or states a determination under it, and, for each developer, the latest card before its first framework edition. The other cards are to follow in batches of about eight.

Each card is dated as it prints its own date. A card that prints no date has no date here; a repository commit date is not a card date. Six cards show an earlier publication date on an index, publication or announcement page that has not yet been checked, and keep their own date until it is.

openai.com, x.ai, ai.meta.com and microsoft.ai refused direct retrieval of their web pages (HTTP 403 or 400). The cards’ PDF files on the developers’ content hosts were retrieved directly. The Internet Archive refused connections, so no archive copy was opened; a reviewer later found archive captures of eight cards by index search. On 2026-11-03 the address of Meta’s Code World Model Preparedness Report answered HTTP 403. Its quotations had matched the copy retrieved on 2026-11-02, which the reviewer retrieved again with the same checksum.

Five further documents were retrieved after the batch was drafted. The cross-family check of 2026-11-03 retrieved Meta’s Muse Spark Safety & Preparedness Report and OpenAI’s o3 Operator addendum; each is earlier than the card chosen for its year. Its second pass of the same day retrieved OpenAI’s Sora system card (dated 9 December 2024), xAI’s Grok-1 model card (no card date; it gives the model’s release as October 2023) and Meta’s Muse Spark Open model card (model release October 2026). None of the five is a row yet.

Every quotation was matched by machine against the retrieved card. A reviewer from a different model family checked a sample of 130 cells on 2026-11-03, 90 of them answered blind before comparison. The maintainer excluded six cells on chemical and biological evaluations from that check; they are marked “not cross-family checked”. The other 190 cells were checked within the drafting model family only. The maintainer decided the disagreements; one, on the GPT-4V release decision, went to a second pass, which kept the reviewer’s reading.

Anthropic

  1. Claude 2 model card

    revised 2023-11-21

    Not found in the recorded search

    Read
    Whole text.
    Terms searched
    “cyber”, “hack”, “exploit”, “vulnerab”, “security”, “national security”.
    Passages read and set aside
    Section 2 'Evaluations and Red Teaming', PDF p. 2: National security evaluations are stated without naming a domain. Section 3.4, PDF pp. 5–6: 'Jailbreaks' are attempts to elicit harmful output, not cyber-offence evaluations.

    Note: No passage reporting offensive cyber evaluations was found.

    Open the search record

  2. Model Card Addendum: Claude 3.5 Haiku and Upgraded Claude 3.5 Sonnet

    No printed dateCites RSP · v2.0

    Stated

    • “We also evaluate frontier risks in accordance with our Responsible Scaling Policy (RSP) [4] in the the areas of Chemical, Biological, Radiological, and Nuclear (CBRN) risks, Cybersecurity, and Autonomous Capabilities.”

      Section 3 'Safety', PDF p. 7

    • “The model did not demonstrate capabilities requiring ASL … 3 safeguards and security in any domain.”

      Section 3.2.1 'Results', PDF pp. 9–10 (the text layer splits 'ASL-3' across the page break; the ellipsis marks the page number)

    Note: The card states cybersecurity evaluations under the RSP and, for the upgraded Claude 3.5 Sonnet, that no domain showed capabilities requiring ASL-3 safeguards and security. Method descriptions in Section 3.2 are not recorded (hazard rule).

    Inspect sources and checks

  3. System Card: Claude Opus 4.5

    dated 2025-11 · revised 2025-12-05Cites RSP · v2.2

    Stated

    • “We believe that Claude Opus 4.5 does not demonstrate catastrophically risky capabilities in the cyber domain.”

      Section 7.4 'Cyber evaluations', 'Results', PDF p. 145

    • “The Responsible Scaling Policy does not stipulate a capability threshold for cyber capabilities at any ASL level”

      Section 7.4 'Cyber evaluations', PDF p. 143

    Note: The card states cyber evaluations and a conclusion without a policy threshold, which it states the policy does not set for cyber. Evaluation methods and scores are not recorded (hazard rule).

    Inspect sources and checks

  4. System Card: Claude Haiku 5.5

    dated 2026-10-07

    Stated

    • “Claude Haiku 5.5 is not a frontier model, but it is a relatively significant step up in cyber capabilities over Haiku 4.5.”

      Section 3.1 'Introduction', PDF p. 23

    • “Given the relative lack of advanced cybersecurity capabilities, our cyber safeguards for Claude Haiku 5.5 trigger on significantly less activity than other recent releases.”

      Section 3.2 'Mitigations', PDF p. 23

    Note: The card states cyber capability evaluations and the conclusion of a relative lack of advanced cybersecurity capabilities, compared with other Claude models; no threshold language is used for cyber. Benchmark contents and scores are not recorded (hazard rule).

    Inspect sources and checks

Google DeepMind

  1. Gemini 3 Flash Model Card

    dated 2025-12

    Not found in the recorded search

    Read
    Whole card.
    Terms searched
    “cyber”, “offens”, “exploit”, “vulnerab”, “CCL”.
    Passages read and set aside
    Ethics and Content Safety, 'Frontier Safety Assessment', PDF p. 6 (printed p. 5): Reports Gemini 3 Pro Preview's results without naming domains; recorded under Q05.

    Note: No cyber evaluation was found; the term does not occur in the card.

    Open the search record

  2. Gemini 4 Argon — Model Card

    dated 2026-10Cites FSF · v3.1

    Stated

    • “We cannot rule out that Gemini 4 Argon reaches the cybersecurity CCL.”

      Frontier safety table, Cybersecurity row, PDF p. 12 (printed p. 11)

    Note: The card states that the cybersecurity uplift level 1 CCL cannot be ruled out and that safeguards are applied (Q06). Evaluation details are not recorded (hazard rule).

    Inspect sources and checks

Meta

  1. Llama 2 model card (MODEL_CARD.md, meta-llama/llama)

    No printed date

    Not found in the recorded search

    Read
    whole card (Markdown file, all sections).
    Terms searched
    “cyber”, “hack”, “exploit”, “malware”, “offensive”.

    Note: No evaluation of offensive cyber capability was found. The search is recorded below.

    Open the search record

  2. Llama 3.3 model card (MODEL_CARD.md, meta-llama/llama-models)

    No printed date

    Stated, partly

    • “Our cyber attack uplift study investigated whether the Llama 3 family of LLMs can enhance human capabilities in hacking tasks”

      Responsibility & Safety, 'Critical and other risks', item 3 (Cyber attack enablement)

    Note: The card reports a cyber uplift study and an attack-automation study for the Llama 3 family and states no outcome or conclusion.

    Inspect sources and checks

  3. Code World Model Preparedness Report

    revised 2025-10-08

    Stated

    • “supporting a conclusion of a “moderate” overall risk level from cybersecurity capabilities”

      Section 2 Cybersecurity Evaluation, Summary of results, PDF p. 3

    Note: The report states that cybersecurity capability evaluations were run against the framework's Cyber 1 and Cyber 2 scenarios and concludes 'moderate'.

    Inspect sources and checks

  4. Muse Spark 1.3 Evaluation Report

    revised 2026-10-23

    Stated

    • “Based on our updated suite of cyber evaluations and measurements, we assess the unmitigated model as “moderate or lower” risk.”

      Section 1.1.1 Risk Assessment Summary, PDF p. 9 (printed p. 8)

    Note: The report states cyber capability evaluations against the framework's Cyber 1 and Cyber 2 criteria and a 'moderate or lower' outcome for the unmitigated model.

    Inspect sources and checks

Microsoft

  1. Phi-2 model card (README.md, microsoft/phi-2)

    No printed date

    Not found in the recorded search

    Read
    whole card (Markdown file, all sections).
    Terms searched
    “cyber”, “hack”, “exploit”, “malware”.

    Note: No evaluation of offensive cyber capability was found. The search is recorded below.

    Open the search record

  2. Phi-4 model card (README.md, microsoft/phi-4)

    No printed date

    Not found in the recorded search

    Read
    whole card (Markdown file, all sections).
    Terms searched
    “cyber”, “hack”, “exploit”, “malware”.
    Passages read and set aside
    Responsible AI Considerations, 'Misuse': States that malware production may be possible; a stated limitation, not an evaluation.

    Note: No evaluation of offensive cyber capability was found. The search is recorded below.

    Open the search record

  3. Phi-4-mini-flash-reasoning model card (README.md, microsoft/Phi-4-mini-flash-reasoning)

    No printed date

    Not found in the recorded search

    Read
    whole card (Markdown file, all sections).
    Terms searched
    “cyber”, “hack”, “exploit”, “malware”.
    Passages read and set aside
    Responsible AI Considerations, 'Misuse': States that malware production may be possible; a stated limitation, not an evaluation.

    Note: No evaluation of offensive cyber capability was found. The search is recorded below.

    Open the search record

  4. MAI-Cyber-1.1-Flash model card

    No printed date

    Stated

    • “evaluated at the model layer across a range of cybersecurity capability and safety benchmarks”

      Section 3, Model-Only External Cyber Benchmark Results, PDF p. 4

    • “This assessment uncovered several critical risks of the model alone”

      Section 3.2 Safety evaluation and red-teaming, PDF p. 7

    Note: The card reports offensive-cyber capability evaluations and states that an internal red-team assessment found 'critical risks of the model alone'; it uses no framework threshold language. Benchmark details are not recorded here.

    Inspect sources and checks

OpenAI

  1. GPT-4V(ision) System Card

    dated 2023-09-25

    Stated, partly

    • “Cybersecurity Performance accuracy CAPTCHA breaking evaluations”

      §2.2 Evaluations, list of evaluations, PDF p. 3

    • “a powerful, general purpose CAPTCHA breaker that’s easily accessible can have cybersecurity and AI safety implications.”

      §2.2, PDF p. 5

    Note: The card lists CAPTCHA-breaking accuracy evaluations under “Cybersecurity” and states that such a capability can have cybersecurity implications. No outcome or conclusion for these evaluations is stated in the card’s text layer (figure contents are not captured). No other offensive-cyber evaluation is reported. Search for the rest of the criterion: §2.2 Evaluations; §2.3; Appendix A. Terms searched: “cyber”, “CAPTCHA”, “exploit”, “hacking”, “vulnerab”. Passages read and not counted: Footnote 2, PDF p. 6: Refers to “system interaction” assessed in the GPT-4 System Card; no cyber outcome stated for GPT-4V.

    Inspect sources and checks

  2. o1 system card · 2024

    revised 2024-12-19Cites Preparedness · Beta

    Stated

    • “Score for o1: Low (same as o1-preview) Summary: o1 does not sufficiently advance real-world vulnerability exploitation capabilities to indicate medium risk.”

      §5.4 Cybersecurity, PDF p. 17

    Note: Cybersecurity: Low. The card reports capture-the-flag evaluations; results are not recorded here.

    Inspect sources and checks

  3. Addendum to GPT-5.2 System Card: GPT-5.2-Codex

    dated 2025-12-18Cites Preparedness · Version 2

    Stated

    • “A review of these results and other relevant information by the Safety Advisory Group and OpenAI leadership determined that GPT-5.2-Codex’s performance—while stronger than earlier models—does not meet the level of consistency needed for High cyber capability.”

      §5.1.2 Cybersecurity, PDF p. 13 (printed p. 12)

    Note: Cybersecurity: below High. The card reports three internal benchmarks and an external evaluation by Irregular; results are not recorded here.

    Inspect sources and checks

  4. Addendum to GPT-6 Luna System Card: October 29, 2026 Update

    dated 2026-10-29Cites Preparedness · Version 2

    Stated

    • “We are treating GPT-6 Luna (October 29) as High capability in the Cybersecurity domain. Below, we report updated results on our High capability evaluations.”

      §8.1.2 Cybersecurity Capabilities, PDF p. 14 (printed p. 13)

    Note: Outcome category: High. Results of the two reported evaluations are not recorded here.

    Inspect sources and checks

xAI

  1. Grok 4.1 Model Card

    dated 2025-11-17

    Stated

    • “Grok 4.1 again performs similarly to other frontier models, but substantially below the level of human cybersecurity experiments”

      §2.3.2 Results, p. 5

    Note: Dual-use cyber knowledge and an agentic capture-the-flag evaluation are reported, with the stated comparative conclusion quoted (the card's wording); no threshold language. Scores are not recorded.

    Inspect sources and checks

  2. Model Card: Grok 4.7

    dated 2026-09-21Cites FAIF · June 2026 edition

    Stated

    • “Grok 4.7 shows small cybersecurity-relevant capability gains over Grok 4.6”

      §6 Cyber capabilities and safeguards, PDF p. 17 (printed p. 16)

    Note: Offensive cyber capability evaluations are reported, run without production safeguards, with a comparative conclusion; no FAIF threshold language is given for cyber. The card states that third-party evaluators corroborated the internal cyber results (Q13). Scores are not recorded.

    Inspect sources and checks