Compare two entries
Select two models to inspect their evidence together.
Rankings
Selected models, ordered by the Rank by choice aboveScore is the public leaderboard score; the other Rank by choices show diagnostic views when the selected models have that data.
Scores
Overall min-max and mean; small dots are each game (averaged across reasoning variants where scores exist); game dots show working-program strength, without failed generation attemptsEconomics
Coming soonGeneration cost versus score will appear when the economics segment is available.
Generation cost evidence and scores include only economics-eligible games once that segment is available. Cost means equal-weight unique model/game/reasoning cells; hover text gives the median, observed-run count, and evidence basis. Output-token and API-latency views are under More charts. “All settings” combines public reasoning variants; the three bracket controls select their model-relative provider settings.
Overall
Reasoning variantsPer game
Working-program strength heatmap (per-game scores of working programs averaged across reasoning variants; failed generation attempts count only in the headline score)Head-to-head
Combined view across tested provider settings.Off-diagonal cells use win rate for the row model against the column opponent (W-L-D in the tooltip). Use Reasoning view to show one reasoning variant.
More charts
Less-used views open on demand, so the page stays short.Per-game comparison Top selected models within each game
Each panel ranks selected models within one game for the active reasoning view, by working-program strength; failed generation attempts count only in the headline score.
Cycle detector Non-transitive matchups between selected models
What it finds: Leaderboard averages can hide matchup loops. The detector searches the currently selected models and reasoning view for cases where A outperforms B, B outperforms C, and C outperforms A.
How to read it: In A → B → C → A, each arrow points from the model with the higher pair score to the model it outscored. Hover over a result for each edge’s win-loss-draw record and pair score.
Each arrow requires at least four matches and a pair score above 50%, with a draw counting as half a win. Results are ordered by the cycle’s weakest edge: weakest edge +x pp is the smallest of the three margins above 50%, while n is the total match count across all three edges. The detector shows up to ten cycles. These are descriptive matchup patterns, not tests of statistical significance. “No cycles” means none met these rules in the current selection and reasoning view; it does not prove that the matchups are transitive.
Output tokens and API latency How much each model wrote, and how fast
These views will appear when the economics segment is available.
Output tokens and API latency are generated-code and repair response telemetry on economics-eligible games; latency excludes local validation and compile probe time.
Static rankings — current page release
This table remains available without chart scripts. It describes this page's release; restoring a requested comparison and its filters requires JavaScript. Open benchmark rankings for the detailed tables.
Current comparison tables
Overall leaderboard scores
All officially ranked Overall models from the same data as the interactive charts. Partial rows keep their official rank and show that their evidence is incomplete.
| Model | Evidence status | Rank | Mean | Min | Max |
|---|---|---|---|---|---|
 GPT-6 Astra | Complete evidence | 1 | 89.4 | 72.2 | 98.9 |
 Claude Opus 5.5 | Complete evidence | 2 | 88.6 | 52.7 | 99.3 |
| Claude Fable 5.1 | Complete evidence | 3 | 88.4 | 65.7 | 97.9 |
| GPT-6.1 Sol | Complete evidence | 4 | 85.8 | 51.0 | 98.3 |
| Claude Opus 5 | Complete evidence | 5 | 78.3 | 44.3 | 95.8 |
| GPT-6 Sol | Complete evidence | 6 | 74.5 | 10.0 | 94.8 |
| GPT-5.6 Sol | Complete evidence | 7 | 68.5 | 39.4 | 88.9 |
| Claude Fable 5 | Complete evidence | 8 | 67.2 | 21.4 | 92.8 |
 Muse Wonder | Complete evidence | 9 | 66.2 | 16.8 | 92.9 |
 GLM-5.3 | Complete evidence | 10 | 65.3 | 0.0 | 93.2 |
 Grok 4.7 | Complete evidence | 11 | 65.1 | 0.0 | 93.5 |
| Claude Sonnet 5.5 | Complete evidence | 12 | 61.6 | 14.1 | 98.8 |
| Grok 4.6 | Complete evidence | 13 | 60.8 | 13.7 | 85.7 |
 Gemini 3.8 Flash | Complete evidence | 14 | 60.5 | 34.4 | 86.6 |
| Muse Spark 1.3 | Complete evidence | 15 | 59.4 | 26.5 | 84.2 |
| Grok 4.5 | Complete evidence | 16 | 57.9 | 16.5 | 89.8 |
| GLM-5.3 FlashX | Complete evidence | 17 | 57.5 | 19.1 | 84.8 |
| Claude Haiku 5.5 | Coverage incomplete: some games cannot be scored: their only generation attempts failed because of the test harness or the provider and cannot currently be repeated | 18 | 57.1 | 16.9 | 91.1 |
| GLM-5.3 Flash | Complete evidence | 19 | 57.0 | 18.4 | 81.0 |
| GPT-5.4 | Complete evidence | 20 | 56.2 | 0.0 | 98.0 |
 Qwen3.8 Max 0902 | Complete evidence | 21 | 55.0 | 26.5 | 92.7 |
| Qwen3.8 2.4T A95B | Complete evidence | 22 | 54.3 | 32.1 | 91.0 |
 Hy4 Preview | Complete evidence | 23 | 53.6 | 0.0 | 90.3 |
 DeepSeek V4.1 Flash | Complete evidence | 24 | 53.4 | 22.9 | 83.0 |
| Muse Spark 1.2 | Complete evidence | 25 | 53.2 | 19.8 | 85.0 |
 Ember-1 | Complete evidence | 26 | 52.8 | 22.0 | 85.1 |
 Ox Alpha | Complete evidence | 27 | 52.2 | 21.7 | 74.9 |
| Gemini 3.6 Flash | Complete evidence | 28 | 51.7 | 19.1 | 84.2 |
| Claude Opus 4.8 | Complete evidence | 29 | 51.7 | 0.0 | 84.1 |
| Gemini 3.5 Flash | Complete evidence | 30 | 51.0 | 12.1 | 90.6 |
| GPT-5.5 | Complete evidence | 31 | 50.3 | 0.0 | 87.7 |
| Qwen3.8 Max | Complete evidence | 32 | 49.7 | 19.1 | 90.7 |
| GPT-6 Luna | Complete evidence | 33 | 48.8 | 29.6 | 86.3 |
 Kimi K3 | Complete evidence | 34 | 48.7 | 0.0 | 93.6 |
| Claude Opus 4.5 | Complete evidence | 35 | 48.6 | 33.0 | 77.4 |
| Claude Sonnet 5 | Complete evidence | 36 | 47.8 | 23.7 | 81.2 |
| GPT-5.6 Terra | Complete evidence | 37 | 47.3 | 0.0 | 80.1 |
| Gemini 3.1 Pro Preview | Complete evidence | 38 | 46.5 | 14.2 | 85.2 |
| DeepSeek V4 Pro | Complete evidence | 39 | 44.5 | 0.0 | 84.8 |
| GLM-5.2 | Complete evidence | 40 | 44.4 | 10.1 | 79.2 |
| DeepSeek V4 Pro 0813 | Complete evidence | 41 | 44.2 | 0.0 | 79.9 |
| GPT-5 | Coverage incomplete: one or more documented settings are untested or ineligible | 42 | 43.0 | 0.0 | 75.1 |
| GPT-5.6 Luna | Complete evidence | 43 | 41.8 | 11.5 | 82.3 |
| o3 | Complete evidence | 44 | 36.5 | 0.0 | 58.9 |
 Seed 2.0 Code | Complete evidence | 45 | 36.0 | 0.0 | 87.5 |
| Qwen3.8 27B | Coverage incomplete: generation quality is below the publication threshold | 46 | 34.3 | 0.0 | 77.2 |
| DeepSeek V4 Flash | Complete evidence | 47 | 34.1 | 0.0 | 81.9 |
| Gemini 3.5 Flash Lite | Complete evidence | 48 | 33.5 | 10.3 | 62.0 |
| Space Bunny Alpha | Complete evidence | 49 | 33.3 | 0.0 | 74.6 |
 Inkling Small | Complete evidence | 50 | 32.6 | 4.6 | 55.2 |
| Muse Glimmer 30B | Complete evidence | 51 | 30.5 | 0.0 | 48.7 |
| GPT-OSS 120B | Complete evidence | 52 | 30.2 | 0.0 | 59.2 |
 Nemotron 3 Ultra 550B A55B | Coverage incomplete: one or more documented settings are untested or ineligible; some games cannot be scored: their only generation attempts failed because of the test harness or the provider and cannot currently be repeated | 53 | 27.5 | 0.0 | 60.7 |
| Inkling | Complete evidence | 54 | 26.4 | 0.0 | 48.0 |
 Step 3.7 Flash | Complete evidence | 55 | 24.8 | 0.0 | 51.3 |
 Mercury 2.5 Preview | Complete evidence | 56 | 21.9 | 0.0 | 45.3 |
| Gemini 3.1 Flash Lite | Coverage incomplete: one or more documented settings are untested or ineligible | 57 | 16.2 | 0.0 | 44.3 |
 Solar Mini 4 | Coverage incomplete: generation quality is below the publication threshold; some games cannot be scored: their only generation attempts failed because of the test harness or the provider and cannot currently be repeated | 58 | 8.2 | 0.0 | 46.9 |
 Granite 4.2 8B | Coverage incomplete: every generation attempt of a reasoning setting failed, so that setting scores 0; generation quality is below the publication threshold | 59 | 1.6 | 0.0 | 38.9 |