Document · Research paper

MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark

Work-level record. Its arXiv versions and any conference or journal version are listed as sources, each with its own date. A citation to the work does not identify which version the citing author read.

Issued by

No issuing record yet. A missing record is a gap in coverage.

Authors credited

No authorship claim recorded yet. The source records list Neural Information Processing Systems, arXiv, Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue and Wenhu Chen.

Dates

No date claim recorded yet. The source records give the publication date as 2024 and 2024-06-03.

Exact source

MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark Neural Information Processing Systems · published 2024 · Advances in Neural Information Processing Systems 37, Datasets and Benchmarks Track. Located by the cross-family machine check on 2026-10-30 (item A043; Proceedings heading), at the precision of its source; not read again at admission.
MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark arXiv · published 2024-06-03 · arXiv abstract page, retrieved for the candidate pack on 2026-10-28 (SHA-256 1a27b66c661db5cdcd1448e98ab515354377e8de82a275875213a9af50c70953). Submission history as arXiv records it (UTC): v1 3 June 2024; v2 4 June 2024; v3 5 June 2024; v4 23 June 2024; v5 7 October 2024; v6 6 November 2024. The page serves the latest version; each version is a separate expression, and only the one a record names was read.
MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark arXiv · published 2024-06-03 · arXiv:2406.01574v1, submitted 3 June 2024 (UTC). Retrieved for the candidate pack on 2026-10-28 (SHA-256 9bf7b08bf1c220cb6c1ed0dcafb7d10848a21178137e13dd5e13981d8ecd15e3). The author list is the byline as printed on the first page.

Appears in

No collection holds a record about this document yet.

Versions

No other versions of this document are recorded. The evidence applies to the source version listed above. Other versions, such as a preprint, working paper or journal publication, are not identified or substituted.

Documentary relations

Each relation is listed separately. Citing, crediting and reporting the use of a method are different acts, and none of them shows agreement or influence by itself.

Cited by · 6

Recorded documents that explicitly reference this one. A citation to a work does not identify which version was read.

Not recorded for this document: cites · explicitly credits · reports using a method from · method reported as used by · applies parts of · parts applied by · discusses · reports evaluation of. A missing record does not show that the relation does not exist.

What the document says it adds

The document’s own description of its contribution, quoted from the version each row names. A claim to be the first is the document’s claim; Selekoa does not assess it.

In its own wordsThe paper introduces MMLU-Pro, a dataset that extends the MMLU benchmark with more reasoning-focused questions and expands the answer choices from four to ten.“This paper introduces MMLU-Pro, an enhanced dataset designed to extend the mostly knowledge-driven MMLU benchmark by integrating more challenging, reasoning-focused questions and expanding the choice set from four to ten options.” arXiv v1, PDF p. 1, abstract2024-06-03 · publication · Stated in source · Supported Inspect claim & evidence ↗