Benchmark definition

TruthfulQA

ActiveEstablishedCommon useTRUTHFULQA

An 817-question benchmark designed to test whether language models repeat common human misconceptions and falsehoods.

Applicability does not mean evaluated; benchmark result/model score data is outside this UI contract.

Compare with others

About

Description: TruthfulQA covers 38 categories and deliberately asks questions that can elicit misconception-driven false answers. It supports generative and historical multiple-choice evaluation modes.
What it measures: Truthfulness under questions where imitation of common human text can produce false answers.
What it does not measure: It is not simply a broad factual-knowledge benchmark; a model can know facts yet still imitate a popular falsehood.
Comparison guidance: Not published
Intended difficulty: Not published

Not recorded for this benchmark: why it matters, motivation, gap addressed.

Classification

Categories: TruthfulnessFactuality
Domains: General academic
Evaluation style: Automatic reference matchModel judgedStatic Q&A

Structure

One undivided item set. No components or scenarios recorded.

Tasks

None recorded

Metrics

What a result on this benchmark would be reported as. ctxdex does not carry results.

Metric	Role	Direction	Unit
Truthful Rate	Primary	Higher is better	Percent
Informative Rate	Secondary	Higher is better	Percent
TruthfulQA MC1	Secondary	Higher is better	Percent
TruthfulQA MC2	Secondary	Higher is better	Percent

Methodology

None recorded

Evaluation environment

What the benchmark's harness provides and allows. These are properties of the environment, not model requirements, and applicability does not test for them.

Interaction: None. Static prompts and answers.
Evaluation style: Automatic reference match, Model judged, Static Q&A
Environment summary: Not published

Model requirements

Required conditions decide applicability. Relevant ones describe what the benchmark exercises and decide nothing.

Required · decides applicability

Whole benchmark

Text (input)Text (output)

Relevant · informational

General language generationGeneral purpose modelsReasoning models

CTXDEX models

30 of the 58 active, preview, experimental models meet this benchmark’s recorded requirements, at some scope. Applicable means the requirements are met, not that the model was evaluated.

View all 30 applicable models →

Applicable · full: 30
Applicable · components: 0
Applicable · scenarios: 0
Conditionally applicable: 0
Not recorded: 28

Counted over 58 models in the default lifecycle scope (active, preview, experimental).

Every required condition, for the whole benchmark, is recorded as supported. Each line is one recorded condition and why it counts.

Ctxdex Chintana 1 Large
Ctxdex Efficient Reasoning
Full benchmark
Every one of the 2 required conditions is recorded as supported.
- MODALITY:INPUT:TEXTthe catalogue records SUPPORTED for TEXT (input)
- MODALITY:OUTPUT:TEXTthe catalogue records SUPPORTED for TEXT (output)
Ctxdex Chintana 1 Mini
Ctxdex Efficient Reasoning
Full benchmark
Every one of the 2 required conditions is recorded as supported.
- MODALITY:INPUT:TEXTthe catalogue records SUPPORTED for TEXT (input)
- MODALITY:OUTPUT:TEXTthe catalogue records SUPPORTED for TEXT (output)

Preview holds 12 of 30. View all →

Applicability/relevance ordering only; never a performance ranking.

Scale

817
Question
Total · Exact
38
Category
Total · Exact

Data and access

Dataset: Open
Evaluation code: Open source
Unknown: Leaderboard, Peer review, Maintenance

Coverage

Languages, regions: None recorded
Coverage summary: Not published

Limitations

Benchmark limitation classification: narrow scope. Questions are specifically constructed around misconception-driven falsehoods rather than broad factual coverage.

Lineage

Family: FamilyTruthfulQA
Related: Inverse of variant of BenchmarkTruthfulQA Improved Multiple Choice
Other definitions in family: TruthfulQA Improved Multiple Choice 2025-01 Active
Changes: None recorded

Sources

TruthfulQA paper ↗Research paper · Original creator
Official TruthfulQA repository ↗Official repository · Benchmark official

Freshness

Content version: 1.0
Last reviewed: 2026-09-22
Refresh model: Static
Time sensitivity: Low
Benchmark data: c40cccac878af5d0

13 contract fields are empty for every benchmark in this release and are not shown: applicability groups, conditional requirements, feature requirements, model relevance domains, required capabilities, openness summary, reference artifacts, interaction protocols, tool protocols, benchmark last major refresh at, short name, target expertise level, target population.