Benchmark definition

SWE-PolyVision v1.0

v1.0

ActiveSWE_POLYVISION_V1_0

Applicability does not mean evaluated; benchmark result/model score data is outside this UI contract.

Compare with others

About

Description: SWE-PolyVision v1.0
What it measures: SWE-PolyVision v1.0
Comparison guidance: Not published
Intended difficulty: Not published

Not recorded for this benchmark: what it does not measure, why it matters, motivation, gap addressed.

Classification

Categories: Software Engineering
Domains: None recorded
Evaluation style: None recorded

Structure

3 scenarios. Applicability can differ between them.

Scenarios

Text Only
Native Vision
Tool Mediated Vision

Tasks

Issue ResolutionCross Image Repository Abductive Reasoning

Metrics

What a result on this benchmark would be reported as. ctxdex does not carry results.

Metric	Role	Direction	Unit
Resolved Rate Metric code used by canonical Batch-2 benchmark definitions; exact benchmark-specific semantics remain in benchmark_metric parameters/methodology.	Primary	Higher is better	Percent

Methodology

None recorded

Evaluation environment

What the benchmark's harness provides and allows. These are properties of the environment, not model requirements, and applicability does not test for them.

Interaction: None. Static prompts and answers.
Evaluation style: Not recorded
Environment summary: Not published

Model requirements

Required conditions decide applicability. Relevant ones describe what the benchmark exercises and decide nothing.

Required · decides applicability

Whole benchmark

Agentic software engineering

Relevant · informational

Code editingComplex reasoning

CTXDEX models

6 of the 58 active, preview, experimental models meet this benchmark’s recorded requirements, at some scope. Applicable means the requirements are met, not that the model was evaluated.

View all 6 applicable models →

Applicable · full: 6
Applicable · components: 0
Applicable · scenarios: 0
Conditionally applicable: 0
Not recorded: 52

Counted over 58 models in the default lifecycle scope (active, preview, experimental).

Every required condition, for the whole benchmark, is recorded as supported. Each line is one recorded condition and why it counts.

Ctxdex Code 2
Ctxdex Code
Full benchmark
Every one of the 1 required conditions is recorded as supported.
- MODEL_TASK:AGENTIC_SOFTWARE_ENGINEERINGthe catalogue records SUPPORTED for AGENTIC_SOFTWARE_ENGINEERING
Ctxdex Efficient 3
Ctxdex Efficient
Full benchmark
Every one of the 1 required conditions is recorded as supported.
- MODEL_TASK:AGENTIC_SOFTWARE_ENGINEERINGthe catalogue records SUPPORTED for AGENTIC_SOFTWARE_ENGINEERING

Applicability/relevance ordering only; never a performance ranking.

Scale

92
Task
Total · Exact
405
Current visual inputs
Total · Exact
36
Organizations
Total · Exact
48
Public subset
Total · Exact

Data and access

UnknownDataset, evaluation code, leaderboard, peer review and maintenance status are all unrecorded for this benchmark.

Coverage

Languages, regions: None recorded
Coverage summary: Not published

Limitations

None recorded

Lineage

Family: FamilySwe Polyvision
Related: Related to BenchmarkSWE-bench Multimodal
Changes: None recorded

Sources

None recorded

Freshness

Content version: B2-1.0
Last reviewed: 2026-09-23
Refresh model: Unknown
Time sensitivity: Unknown
Benchmark data: c40cccac878af5d0

13 contract fields are empty for every benchmark in this release and are not shown: applicability groups, conditional requirements, feature requirements, model relevance domains, required capabilities, openness summary, reference artifacts, interaction protocols, tool protocols, benchmark last major refresh at, short name, target expertise level, target population.