Reasoning view
Applies to Scores, Reasoning variants, Head-to-head, Per game heatmap, and Economics (when available).
Rank by
Diagnostic orderingConservative score is the public leaderboard score after public-score adjustments; other buttons show diagnostic views when the selected models have that data.
Economics
Coming soonStandard-price equivalent versus score will appear when the economics segment is available.
Generation cost evidence, output-token counts, API latency, and scores include only economics-eligible games once that segment is available. Cost means equal-weight unique model/game/reasoning cells; hover text gives the median, observed-run count, and evidence basis. Provider-account billing remains separate below. Output tokens and API latency are generated-code/repair response telemetry; latency excludes local validation and compile probe time. “All” combines public reasoning variants; XHigh / Medium / None show that variant only.
Actual spend
Provider account reconciliation
Account-billed totals stay separate from model economics. Exact benchmark attribution includes only generation attempts carrying billed evidence and an exact provider/time match.
| Provider | Period | Account billed | Exact benchmark attribution | Coverage | Evidence records |
|---|---|---|---|---|---|
| OpenAI | 2026-06-18 to 2026-07-11 | $87.39 | $0.00 | 0.0% | 875 usage / 162 cost |
Provider-account totals may include non-benchmark activity. Unattributed account billing is never distributed across models or benchmark attempts.
Scores
Overall min-max and mean; small dots are each game (averaged across reasoning variants where scores exist)Overall
Reasoning variantsHead-to-head
Combined view across XHigh, Medium, and None.Off-diagonal cells use win rate for the row model against the column opponent (W-L-D in the tooltip). Use Reasoning view to show one reasoning variant.
Per game
Strength heatmap (per-game scores averaged across reasoning variants)Per-game comparison
Top selected models within each gameEach panel ranks selected models within one game for the active reasoning view.