Benchmark definition
GPQA Diamond
The 198-question highest-quality/hard GPQA subset widely used for frontier-model reasoning evaluation.
Applicability does not mean evaluated; benchmark result/model score data is outside this UI contract.
About
- Description
- GPQA Diamond contains 198 expert-written biology, physics, and chemistry questions selected from GPQA for stronger expert agreement and difficulty to knowledgeable non-experts.
- What it measures
- Graduate-level scientific reasoning and knowledge on the strictest GPQA subset.
- What it does not measure
- The same scope limitations as GPQA apply; it is a small, science-only, static multiple-choice set.
- Comparison guidance
- Not published
- Intended difficulty
- Not published
Not recorded for this benchmark: why it matters, motivation, gap addressed.
Classification
- Categories
- KnowledgeScience reasoningGeneral reasoning
- Domains
- BiologyPhysicsChemistry
- Evaluation style
- Automatic reference matchStatic Q&A
Structure
One undivided item set. No components or scenarios recorded.
Tasks
Metrics
What a result on this benchmark would be reported as. ctxdex does not carry results.
| Metric | Role | Direction | Unit |
|---|---|---|---|
Accuracy | Primary | Higher is better | Percent |
Methodology
None recorded
Evaluation environment
What the benchmark's harness provides and allows. These are properties of the environment, not model requirements, and applicability does not test for them.
- Interaction
None. Static prompts and answers.
- Evaluation style
- Automatic reference match, Static Q&A
- Environment summary
- Not published
Model requirements
Required conditions decide applicability. Relevant ones describe what the benchmark exercises and decide nothing.
Required · decides applicability
Relevant · informational
CTXDEX models
13 of the 58 active, preview, experimental models meet this benchmark’s recorded requirements, at some scope. Applicable means the requirements are met, not that the model was evaluated.
View all 13 applicable models →- Applicable · full
- 13
- Applicable · components
- 0
- Applicable · scenarios
- 0
- Conditionally applicable
- 0
- Not recorded
- 45
Counted over 58 models in the default lifecycle scope (active, preview, experimental).
Every required condition, for the whole benchmark, is recorded as supported. Each line is one recorded condition and why it counts.
Every one of the 3 required conditions is recorded as supported.
MODALITY:INPUT:TEXTthe catalogue records SUPPORTED for TEXT (input)MODALITY:OUTPUT:TEXTthe catalogue records SUPPORTED for TEXT (output)MODEL_TASK:COMPLEX_REASONINGthe catalogue records SUPPORTED for COMPLEX_REASONING
Every one of the 3 required conditions is recorded as supported.
MODALITY:INPUT:TEXTthe catalogue records SUPPORTED for TEXT (input)MODALITY:OUTPUT:TEXTthe catalogue records SUPPORTED for TEXT (output)MODEL_TASK:COMPLEX_REASONINGthe catalogue records SUPPORTED for COMPLEX_REASONING
Applicability/relevance ordering only; never a performance ranking.
Scale
- 198QuestionTotal · Exact
Data and access
- Dataset
- Open
- Evaluation code
- Open source
- Unknown
- Leaderboard, Peer review, Maintenance
Coverage
- Language
- English (primary)
- Coverage summary
- Not published
Limitations
None recorded
Lineage
Sources
- Official GPQA repository ↗Official repository · Benchmark official
- GPQA dataset hub ↗Dataset · Benchmark official
Freshness
- Content version
- 1.0
- Last reviewed
- 2026-09-22
- Refresh model
- Static
- Time sensitivity
- Low
- Benchmark data
- c40cccac878af5d0
13 contract fields are empty for every benchmark in this release and are not shown: applicability groups, conditional requirements, feature requirements, model relevance domains, required capabilities, openness summary, reference artifacts, interaction protocols, tool protocols, benchmark last major refresh at, short name, target expertise level, target population.