ctxdexSynthetic model data for learning

Benchmark definition

ARC-AGI-2

2
ActiveEstablishedCommon useARC_AGI_2

The second ARC-AGI benchmark generation, with calibrated harder evaluation tasks designed for modern reasoning systems.

Applicability does not mean evaluated; benchmark result/model score data is outside this UI contract.

Compare with others

About

Description
ARC-AGI-2 retains the grid-transformation format while expanding training data and introducing calibrated public, semi-private, and private evaluation sets designed to challenge frontier reasoning systems.
What it measures
Abstract reasoning, compositional rule application, symbolic interpretation, and efficient adaptation from minimal examples.
Comparison guidance
Not published
Intended difficulty
Not published

Not recorded for this benchmark: what it does not measure, why it matters, motivation, gap addressed.

Classification

Categories
Abstract reasoningGeneral reasoning
Domains
General academic
Evaluation style
Automatic reference match

Structure

One undivided item set. No components or scenarios recorded.

Tasks

None recorded

Metrics

What a result on this benchmark would be reported as. ctxdex does not carry results.

MetricRoleDirectionUnit
Pass@2
PrimaryHigher is betterPercent

Methodology

CTXDEX text-serialization applicability profileThis definition

For text-model applicability, an evaluator-provided adapter serializes ARC integer grids into text/JSON and parses the model output back into the native grid structure.

Evaluation environment

What the benchmark's harness provides and allows. These are properties of the environment, not model requirements, and applicability does not test for them.

Interaction

None. Static prompts and answers.

Evaluation style
Automatic reference match
Environment summary
Not published

Model requirements

Required conditions decide applicability. Relevant ones describe what the benchmark exercises and decide nothing.

Required · decides applicability

Whole benchmark
Text (input)Text (output)Complex reasoningPlain text (input)Plain text (output)

Relevant · informational

General purpose modelsReasoning models

CTXDEX models

0 of the 58 active, preview, experimental models meet this benchmark’s recorded requirements, at some scope. Applicable means the requirements are met, not that the model was evaluated.

View the 58 not recorded →
Applicable · full
0
Applicable · components
0
Applicable · scenarios
0
Conditionally applicable
0
Not recorded
58

Counted over 58 models in the default lifecycle scope (active, preview, experimental).

Applicability/relevance ordering only; never a performance ranking.

Scale

  • 1,000
    Task
    Public train · Exact
  • 120
    Task
    Public eval · Exact
  • 120
    Task
    Semi private eval · Exact
  • 120
    Task
    Private eval · Exact

Data and access

Dataset
Open
Evaluation code
Open source
Unknown
Leaderboard, Peer review, Maintenance

Coverage

Languages, regions

None recorded

Coverage summary
Not published

Limitations

None recorded

Lineage

Family
FamilyARC-AGI
Related
Successor of BenchmarkARC-AGI-1
Other definitions in family
ARC-AGI-1 1 Active
Changes

None recorded

Sources

Freshness

Content version
1.0
Last reviewed
2026-09-22
Refresh model
Static
Time sensitivity
Low
Benchmark data
c40cccac878af5d0

13 contract fields are empty for every benchmark in this release and are not shown: applicability groups, conditional requirements, feature requirements, model relevance domains, required capabilities, openness summary, reference artifacts, interaction protocols, tool protocols, benchmark last major refresh at, short name, target expertise level, target population.