ctxdexSynthetic model data for learning

Benchmark definition

MMLU

ActiveEstablishedCommon useMMLU

A 57-task multiple-choice benchmark covering broad academic and professional knowledge.

Applicability does not mean evaluated; benchmark result/model score data is outside this UI contract.

Compare with others

About

Description
MMLU evaluates text models across 57 subjects spanning mathematics, history, computer science, law and many other academic and professional areas.
What it measures
Breadth and depth of academic/professional knowledge together with problem solving across many subjects.
What it does not measure
It does not directly test tool use, interactive environments, long-horizon agent behavior, or non-text modalities.
Why it matters
It became a standard broad-coverage benchmark for comparing general language-model knowledge.
Comparison guidance
Not published
Intended difficulty
Not published

Not recorded for this benchmark: motivation, gap addressed.

Classification

Categories
KnowledgeGeneral reasoning
Domains
General academic
Evaluation style
Automatic reference matchStatic Q&A

Structure

1 component. Applicability can differ between them.

Components

  • 57 subject tasksCategoryThe benchmark is partitioned into 57 academic/professional subjects; full enumeration remains to be imported from the official dataset.

Tasks

Multiple-choice academic/professional question answering

Metrics

What a result on this benchmark would be reported as. ctxdex does not carry results.

MetricRoleDirectionUnit
Accuracy
PrimaryHigher is betterPercent

Methodology

Original few-shot MMLU evaluationThis definition

Multiple-choice evaluation aggregated across 57 tasks; commonly run few-shot, including the original paper's few-shot setting.

Evaluation environment

What the benchmark's harness provides and allows. These are properties of the environment, not model requirements, and applicability does not test for them.

Interaction

None. Static prompts and answers.

Evaluation style
Automatic reference match, Static Q&A
Environment summary
Not published

Model requirements

Required conditions decide applicability. Relevant ones describe what the benchmark exercises and decide nothing.

Required · decides applicability

Whole benchmark
Text (input)Text (output)

Relevant · informational

Complex reasoningGeneral purpose modelsReasoning models

CTXDEX models

30 of the 58 active, preview, experimental models meet this benchmark’s recorded requirements, at some scope. Applicable means the requirements are met, not that the model was evaluated.

View all 30 applicable models →
Applicable · full
30
Applicable · components
0
Applicable · scenarios
0
Conditionally applicable
0
Not recorded
28

Counted over 58 models in the default lifecycle scope (active, preview, experimental).

Every required condition, for the whole benchmark, is recorded as supported. Each line is one recorded condition and why it counts.

  • Ctxdex Chintana 1 Large
    Ctxdex Efficient Reasoning
    Full benchmark

    Every one of the 2 required conditions is recorded as supported.

    • MODALITY:INPUT:TEXTthe catalogue records SUPPORTED for TEXT (input)
    • MODALITY:OUTPUT:TEXTthe catalogue records SUPPORTED for TEXT (output)
  • Ctxdex Chintana 1 Mini
    Ctxdex Efficient Reasoning
    Full benchmark

    Every one of the 2 required conditions is recorded as supported.

    • MODALITY:INPUT:TEXTthe catalogue records SUPPORTED for TEXT (input)
    • MODALITY:OUTPUT:TEXTthe catalogue records SUPPORTED for TEXT (output)
Preview holds 12 of 30. View all →

Applicability/relevance ordering only; never a performance ranking.

Scale

  • 57
    Task
    Total · Exact

Data and access

Dataset
Open
Evaluation code
Open source
Unknown
Leaderboard, Peer review, Maintenance

Coverage

Language
English (primary)
Coverage summary
Not published

Limitations

None recorded

Lineage

Family
FamilyMMLU
Related
Basis for BenchmarkMMLU-Pro
Other definitions in family
MMLU-Pro Active
Changes

None recorded

Sources

Freshness

Content version
1.0
Last reviewed
2026-09-22
Refresh model
Static
Time sensitivity
Low
Benchmark data
c40cccac878af5d0

13 contract fields are empty for every benchmark in this release and are not shown: applicability groups, conditional requirements, feature requirements, model relevance domains, required capabilities, openness summary, reference artifacts, interaction protocols, tool protocols, benchmark last major refresh at, short name, target expertise level, target population.