Benchmark definition
LiveBench
A rolling benchmark refreshed with objective, verifiable questions across six categories to reduce contamination and avoid LLM-as-judge scoring.
Applicability does not mean evaluated; benchmark result/model score data is outside this UI contract.
About
- Description
- LiveBench releases new questions and harder tasks over time. The maintained repository currently describes 18 tasks across six categories: math, reasoning, coding, data analysis, language, and instruction following.
- What it measures
- A broad, periodically refreshed set of objectively gradable LLM capabilities.
- What it does not measure
- A generic family record is not itself an immutable score-comparison target; exact release/version definitions are required for results.
- Comparison guidance
- Not published
- Intended difficulty
- Not published
Not recorded for this benchmark: why it matters, motivation, gap addressed.
Classification
- Categories
- Rolling evaluationGeneral reasoning
- Domains
- General academic
- Evaluation style
- Automatic reference match
Structure
One undivided item set. No components or scenarios recorded.
Tasks
None recorded
Metrics
What a result on this benchmark would be reported as. ctxdex does not carry results.
| Metric | Role | Direction | Unit |
|---|---|---|---|
Accuracy | Primary | Higher is better | Percent |
Methodology
None recorded
Evaluation environment
What the benchmark's harness provides and allows. These are properties of the environment, not model requirements, and applicability does not test for them.
- Interaction
None. Static prompts and answers.
- Evaluation style
- Automatic reference match
- Environment summary
- Not published
Model requirements
Required conditions decide applicability. Relevant ones describe what the benchmark exercises and decide nothing.
Required · decides applicability
No required conditions recorded. Every model is Not recorded for this benchmark.
Relevant · informational
CTXDEX models
This benchmark records no required conditions. Without requirements, no model can be shown to meet them, so every model is Not recorded. This is correct, not missing data.
Scale
None recorded
Data and access
- Dataset
- Open
- Evaluation code
- Open source
- Unknown
- Leaderboard, Peer review, Maintenance
Coverage
- Languages, regions
None recorded
- Coverage summary
- Not published
Limitations
None recorded
Lineage
- Family
- FamilyLiveBench
- Related
None recorded
- Other definitions in family
- LiveBench 2024-06-24 SupersededLiveBench 2024-07-26 SupersededLiveBench 2024-08-31 SupersededLiveBench 2024-11-25 SupersededLiveBench 2025-04-02 SupersededLiveBench 2025-04-25 SupersededLiveBench 2025-05-30 SupersededLiveBench 2025-11-25 SupersededLiveBench 2025-12-23 SupersededLiveBench 2026-01-08 SupersededLiveBench 2026-06-25 Active
- Changes
None recorded
Sources
- Official LiveBench repository ↗Official repository · Benchmark official
- LiveBench changelog ↗Documentation · Benchmark official
Freshness
- Content version
- 1.0
- Last reviewed
- 2026-09-22
- Refresh model
- Periodic
- Time sensitivity
- High
- Benchmark data
- c40cccac878af5d0
13 contract fields are empty for every benchmark in this release and are not shown: applicability groups, conditional requirements, feature requirements, model relevance domains, required capabilities, openness summary, reference artifacts, interaction protocols, tool protocols, benchmark last major refresh at, short name, target expertise level, target population.