Benchmark family

BigCodeBench

A family groups the versions of one benchmark. It is not evaluated and has no applicable models of its own; each of its 2 definitions is resolved separately.

Code generation

Definitions

Compare versions →

Newest first, following the recorded successor chain. Each definition is resolved against models separately.

BenchmarkBigCodeBench
ActiveNewest in family
Scale
1,140 problems
Primary metric
Pass@k
Applicable models
3
BenchmarkBigCodeBench-Hard
Active
Subset of BigCodeBench
Scale
148 problems
Primary metric
Pass@k
Applicable models
3

Family methodology

No family-level methodology recorded.

Across definitions

Tasks: None recorded
Sources: None recorded at family level
Last reviewed: 2026-09-23

Applicability does not mean evaluated; benchmark result/model score data is outside this UI contract.