Benchmark definition

StableToolBench

ActiveSTABLETOOLBENCH

Applicability does not mean evaluated; benchmark result/model score data is outside this UI contract.

Compare with others

About

Description: StableToolBench
What it measures: StableToolBench
Comparison guidance: Not published
Intended difficulty: Not published

Not recorded for this benchmark: what it does not measure, why it matters, motivation, gap addressed.

Classification

Categories: Tool UseApi Use
Domains: None recorded
Evaluation style: None recorded

Structure

One undivided item set. No components or scenarios recorded.

Tasks

Open Domain Tool Selection And InvocationMulti Tool CompositionTool Retrieval

Metrics

What a result on this benchmark would be reported as. ctxdex does not carry results.

Metric	Role	Direction	Unit
Task Success Rate Canonical Batch-2 metric code TASK_SUCCESS_RATE; benchmark-specific formula/parameters remain on benchmark_metric and methodology records.	Primary	Higher is better	Percent
Win Rate Canonical Batch-2 metric code WIN_RATE; benchmark-specific formula/parameters remain on benchmark_metric and methodology records.	Secondary	Higher is better	Percent

Methodology

None recorded

Evaluation environment

What the benchmark's harness provides and allows. These are properties of the environment, not model requirements, and applicability does not test for them.

Interaction: None. Static prompts and answers.
Evaluation style: Not recorded
Environment summary: Not published

Model requirements

Required conditions decide applicability. Relevant ones describe what the benchmark exercises and decide nothing.

Required · decides applicability

No required conditions recorded. Every model is Not recorded for this benchmark.

Relevant · informational

Tool calling

CTXDEX models

Not recorded58 of 58

This benchmark records no required conditions. Without requirements, no model can be shown to meet them, so every model is Not recorded. This is correct, not missing data.

Scale

153
G1 category
Total · Exact
163
G1 instruction
Total · Exact
158
Tool
Total · Exact
124
G2 category
Total · Exact
106
G2 instruction
Total · Exact
61
G3 instruction
Total · Exact
>7000
API
Total · Exact
765
Case
Total · Exact

Data and access

UnknownDataset, evaluation code, leaderboard, peer review and maintenance status are all unrecorded for this benchmark.

Coverage

Languages, regions: None recorded
Coverage summary: Not published

Limitations

None recorded

Lineage

Family: FamilyToolBench
Related: Derived from BenchmarkToolBench
Other definitions in family: ToolBench Active
Changes: None recorded

Sources

https://github.com/THUNLP-MT/StableToolBench ↗Official repository · Benchmark official
https://arxiv.org/abs/2403.07714 ↗Research paper · Original creator

Freshness

Content version: B2-1.0
Last reviewed: 2026-09-23
Refresh model: Unknown
Time sensitivity: Unknown
Benchmark data: c40cccac878af5d0

13 contract fields are empty for every benchmark in this release and are not shown: applicability groups, conditional requirements, feature requirements, model relevance domains, required capabilities, openness summary, reference artifacts, interaction protocols, tool protocols, benchmark last major refresh at, short name, target expertise level, target population.