Benchmark definition

OlympiadBench

ActiveEstablishedCommon useOLYMPIADBENCH

A bilingual multimodal benchmark of 8,476 olympiad-level mathematics and physics problems.

Applicability does not mean evaluated; benchmark result/model score data is outside this UI contract.

Compare with others

About

Description: OlympiadBench combines text-only and multimodal math/physics problems from international and Chinese olympiads and Chinese college entrance exams, with detailed expert annotations.
What it measures: Advanced mathematical and physical reasoning, including multimodal scientific problem solving.
Comparison guidance: Not published
Intended difficulty: Not published

Not recorded for this benchmark: what it does not measure, why it matters, motivation, gap addressed.

Classification

Categories: MathematicsScience reasoningMultimodal
Domains: MathematicsPhysics
Evaluation style: Automatic metric

Structure

2 components. Applicability can differ between them.

Components

Text-onlyCategoryOfficial dataset categories marked TO.
MultimodalCategoryOfficial dataset categories marked MM.

Tasks

None recorded

Metrics

What a result on this benchmark would be reported as. ctxdex does not carry results.

Metric	Role	Direction	Unit
Accuracy	Primary	Higher is better	Percent

Methodology

None recorded

Evaluation environment

What the benchmark's harness provides and allows. These are properties of the environment, not model requirements, and applicability does not test for them.

Interaction: None. Static prompts and answers.
Evaluation style: Automatic metric
Environment summary: Not published

Model requirements

Required conditions decide applicability. Relevant ones describe what the benchmark exercises and decide nothing.

Required · decides applicability

Whole benchmark

Text (output)Complex reasoning

Component: Multimodal

Image (input)Text (input)Text (output)

Component: Text-only

Text (input)Text (output)

Relevant · informational

Multimodal understandingExplicit reasoningGeneral purpose modelsReasoning models

CTXDEX models

13 of the 58 active, preview, experimental models meet this benchmark’s recorded requirements, at some scope. Applicable means the requirements are met, not that the model was evaluated.

View all 13 applicable models →

Applicable · full: 7
Applicable · components: 6
Applicable · scenarios: 0
Conditionally applicable: 0
Not recorded: 45

Counted over 58 models in the default lifecycle scope (active, preview, experimental).

Every required condition, for the whole benchmark, is recorded as supported. Each line is one recorded condition and why it counts.

Ctxdex Efficient 3
Ctxdex Efficient
Full benchmark
Every one of the 7 required conditions is recorded as supported.
- MODALITY:OUTPUT:TEXTthe catalogue records SUPPORTED for TEXT (output) — benchmark, Text-only, Multimodal
- MODEL_TASK:COMPLEX_REASONINGthe catalogue records SUPPORTED for COMPLEX_REASONING
- MODALITY:INPUT:TEXTthe catalogue records SUPPORTED for TEXT (input) — Text-only, Multimodal
- MODALITY:INPUT:IMAGEthe catalogue records SUPPORTED for IMAGE (input) — Multimodal
Ctxdex Enterprise 4
Ctxdex Enterprise
Full benchmark
Every one of the 7 required conditions is recorded as supported.
- MODALITY:OUTPUT:TEXTthe catalogue records SUPPORTED for TEXT (output) — benchmark, Text-only, Multimodal
- MODEL_TASK:COMPLEX_REASONINGthe catalogue records SUPPORTED for COMPLEX_REASONING
- MODALITY:INPUT:TEXTthe catalogue records SUPPORTED for TEXT (input) — Text-only, Multimodal
- MODALITY:INPUT:IMAGEthe catalogue records SUPPORTED for IMAGE (input) — Multimodal

The whole benchmark is not resolved for these models, but one or more components are. The benchmark state is not a summary of its components, so both are shown.

Ctxdex Chintana 1 Large
Ctxdex Efficient Reasoning
Components only
The full benchmark is not resolved for this subject, and 1 component(s) are.
Applies to Text-only. Not recorded for Multimodal.
- MODALITY:OUTPUT:TEXTthe catalogue records SUPPORTED for TEXT (output) — benchmark, Text-only, Multimodal
- MODEL_TASK:COMPLEX_REASONINGthe catalogue records SUPPORTED for COMPLEX_REASONING
- MODALITY:INPUT:TEXTthe catalogue records SUPPORTED for TEXT (input) — Text-only, Multimodal
Ctxdex Chintana 1 Mini
Ctxdex Efficient Reasoning
Components only
The full benchmark is not resolved for this subject, and 1 component(s) are.
Applies to Text-only. Not recorded for Multimodal.
- MODALITY:OUTPUT:TEXTthe catalogue records SUPPORTED for TEXT (output) — benchmark, Text-only, Multimodal
- MODEL_TASK:COMPLEX_REASONINGthe catalogue records SUPPORTED for COMPLEX_REASONING
- MODALITY:INPUT:TEXTthe catalogue records SUPPORTED for TEXT (input) — Text-only, Multimodal

Preview holds 5 of 6. View all →

Applicability/relevance ordering only; never a performance ranking.

Scale

8,476
Problem
Total · Exact

Data and access

Dataset: Open
Evaluation code: Open source
Unknown: Leaderboard, Peer review, Maintenance

Coverage

Languages, regions: None recorded
Coverage summary: Not published

Limitations

None recorded

Lineage

Family: None recorded
Related: None recorded
Changes: None recorded

Sources

OlympiadBench paper ↗Research paper · Original creator
Official OlympiadBench repository ↗Official repository · Benchmark official

Freshness

Content version: 1.0
Last reviewed: 2026-09-22
Refresh model: Static
Time sensitivity: Low
Benchmark data: c40cccac878af5d0

13 contract fields are empty for every benchmark in this release and are not shown: applicability groups, conditional requirements, feature requirements, model relevance domains, required capabilities, openness summary, reference artifacts, interaction protocols, tool protocols, benchmark last major refresh at, short name, target expertise level, target population.