Benchmark definition
MathArena ArXivLean
03/2026MathArena ArXivLean 03/2026, containing 41 problems.
Applicability does not mean evaluated; benchmark result/model score data is outside this UI contract.
About
- Description
- Research-level mathematical statements from recent arXiv papers formalized as Lean theorem-proving tasks.
- What it measures
- Research-level mathematical statements from recent arXiv papers formalized as Lean theorem-proving tasks.
- Why it matters
- The release is separately identified by MathArena and must not be merged with other monthly competition sets.
- Comparison guidance
- Not published
- Intended difficulty
- Not published
Not recorded for this benchmark: what it does not measure, motivation, gap addressed.
Classification
- Categories
- Formal reasoningResearch mathematicsRolling evaluation
- Domains
- Research mathematics
- Evaluation style
- AgenticExecutable test based
Structure
One undivided item set. No components or scenarios recorded.
Tasks
None recorded
Metrics
What a result on this benchmark would be reported as. ctxdex does not carry results.
| Metric | Role | Direction | Unit |
|---|---|---|---|
Formal Proof Acceptance | Primary | Higher is better | Percent |
Methodology
Release-specific MathArena methodology; preserve this monthly identity rather than applying one timeless family methodology.
Evaluation environment
What the benchmark's harness provides and allows. These are properties of the environment, not model requirements, and applicability does not test for them.
- Interaction
None. Static prompts and answers.
- Evaluation style
- Agentic, Executable test based
- Environment summary
- Not published
Model requirements
Required conditions decide applicability. Relevant ones describe what the benchmark exercises and decide nothing.
Required · decides applicability
Relevant · informational
CTXDEX models
0 of the 58 active, preview, experimental models meet this benchmark’s recorded requirements, at some scope. Applicable means the requirements are met, not that the model was evaluated.
View the 58 not recorded →- Applicable · full
- 0
- Applicable · components
- 0
- Applicable · scenarios
- 0
- Conditionally applicable
- 0
- Not recorded
- 58
Counted over 58 models in the default lifecycle scope (active, preview, experimental).
Applicability/relevance ordering only; never a performance ranking.
Scale
- 41ProblemTotal · Exact
Data and access
- Dataset
- Open
- Evaluation code
- Open source
- Peer review
- Technical report
- Unknown
- Leaderboard, Maintenance
Coverage
- Languages, regions
None recorded
- Coverage summary
- Not published
Limitations
None recorded
Lineage
- Family
- FamilyMathArena ArXivLean
- Related
- Other definitions in family
- MathArena ArXivLean 06/2026 Active
- Changes
None recorded
Sources
- MathArena competitions index ↗Official website · Benchmark official
Freshness
- Content version
- 1.0
- Last reviewed
- 2026-09-22
- Refresh model
- Periodic
- Time sensitivity
- High
- Benchmark data
- c40cccac878af5d0
13 contract fields are empty for every benchmark in this release and are not shown: applicability groups, conditional requirements, feature requirements, model relevance domains, required capabilities, openness summary, reference artifacts, interaction protocols, tool protocols, benchmark last major refresh at, short name, target expertise level, target population.