ctxdexSynthetic model data for learning

Benchmark definition

LiveBench

2024-07-26
SupersededEstablishedCommon useLIVEBENCH_2024_07_26

Official LiveBench release dated 2024-07-26.

Applicability does not mean evaluated; benchmark result/model score data is outside this UI contract.

Compare with others

About

Description
An immutable official LiveBench release identity backed by the release registry and date-specific table/category files.
What it measures
Release-specific refreshed capabilities across LiveBench's objectively graded task categories.
What it does not measure
The release should not be treated as identical in composition to other LiveBench dates.
Why it matters
Date-specific identity preserves changing task composition and enables reproducible benchmark references.
Comparison guidance
Not published
Intended difficulty
Not published

Not recorded for this benchmark: motivation, gap addressed.

Classification

Categories
Rolling evaluationGeneral reasoning
Domains
General academic
Evaluation style
Automatic reference match

Structure

6 components. Applicability can differ between them.

Components

  • ReasoningCategoryOfficial LiveBench 2024-07-26 category containing the subtasks listed in the release manifest.
  • CodingCategoryOfficial LiveBench 2024-07-26 category containing the subtasks listed in the release manifest.
  • MathematicsCategoryOfficial LiveBench 2024-07-26 category containing the subtasks listed in the release manifest.
  • Data AnalysisCategoryOfficial LiveBench 2024-07-26 category containing the subtasks listed in the release manifest.
  • LanguageCategoryOfficial LiveBench 2024-07-26 category containing the subtasks listed in the release manifest.
  • Instruction FollowingCategoryOfficial LiveBench 2024-07-26 category containing the subtasks listed in the release manifest.

Tasks

AMPS HardConnectionsCTACode Generation (LCB)ParaphraseWeb of Lies v2Coding CompletionMath CompetitionPlot UnscramblingSimplifyTable JoinZebra PuzzleOlympiadSpatialStory GenerationTable ReformatTyposSummarize

Metrics

What a result on this benchmark would be reported as. ctxdex does not carry results.

MetricRoleDirectionUnit
Accuracy
PrimaryHigher is betterPercent

Methodology

LiveBench 2024-07-26 release methodologyThis definition

Release-specific methodology and task composition are defined by the official date-specific LiveBench data files and changelog.

Evaluation environment

What the benchmark's harness provides and allows. These are properties of the environment, not model requirements, and applicability does not test for them.

Interaction

None. Static prompts and answers.

Evaluation style
Automatic reference match
Environment summary
Not published

Model requirements

Required conditions decide applicability. Relevant ones describe what the benchmark exercises and decide nothing.

Required · decides applicability

Whole benchmark
Text (input)Text (output)

Relevant · informational

Code generationComplex reasoningGeneral language generationPlain text (input)Plain text (output)Coding modelsGeneral purpose modelsReasoning models

CTXDEX models

30 of the 58 active, preview, experimental models meet this benchmark’s recorded requirements, at some scope. Applicable means the requirements are met, not that the model was evaluated.

View all 30 applicable models →
Applicable · full
30
Applicable · components
0
Applicable · scenarios
0
Conditionally applicable
0
Not recorded
28

Counted over 58 models in the default lifecycle scope (active, preview, experimental).

Every required condition, for the whole benchmark, is recorded as supported. Each line is one recorded condition and why it counts.

  • Ctxdex Efficient 2
    Ctxdex Efficient
    Full benchmark

    Every one of the 2 required conditions is recorded as supported.

    • MODALITY:INPUT:TEXTthe catalogue records SUPPORTED for TEXT (input)
    • MODALITY:OUTPUT:TEXTthe catalogue records SUPPORTED for TEXT (output)
  • Ctxdex Efficient 3
    Ctxdex Efficient
    Full benchmark

    Every one of the 2 required conditions is recorded as supported.

    • MODALITY:INPUT:TEXTthe catalogue records SUPPORTED for TEXT (input)
    • MODALITY:OUTPUT:TEXTthe catalogue records SUPPORTED for TEXT (output)
Preview holds 12 of 30. View all →

Applicability/relevance ordering only; never a performance ranking.

Scale

None recorded

Data and access

Dataset
Partially open
Evaluation code
Open source
Peer review
Peer reviewed
Unknown
Leaderboard, Maintenance

Coverage

Languages, regions

None recorded

Coverage summary
Not published

Limitations

None recorded

Lineage

Family
FamilyLiveBench
Related
Inverse of successor of BenchmarkLiveBench
Successor of BenchmarkLiveBench
Other definitions in family
LiveBench ActiveLiveBench 2024-06-24 SupersededLiveBench 2024-08-31 SupersededLiveBench 2024-11-25 SupersededLiveBench 2025-04-02 SupersededLiveBench 2025-04-25 SupersededLiveBench 2025-05-30 SupersededLiveBench 2025-11-25 SupersededLiveBench 2025-12-23 SupersededLiveBench 2026-01-08 SupersededLiveBench 2026-06-25 Active
Changes

None recorded

Sources

Freshness

Content version
1.0
Last reviewed
2026-09-22
Refresh model
Periodic
Time sensitivity
High
Benchmark data
c40cccac878af5d0

13 contract fields are empty for every benchmark in this release and are not shown: applicability groups, conditional requirements, feature requirements, model relevance domains, required capabilities, openness summary, reference artifacts, interaction protocols, tool protocols, benchmark last major refresh at, short name, target expertise level, target population.