All models as a static table

Chart controls

Models

This list holds the models with an official Overall rank, which needs all three reasoning settings. Partial results keep their rank; the icon explains what evidence is incomplete. Top 2, or any two models, opens a side-by-side report.

Who leads among the selected models?

How to read this chart

Selected models by the leaderboard score; the number is each model's official rank in this view

The public leaderboard score of each selected model.

What does a score cost to generate?

How to read this chart

Generation cost per player against headline score; the line marks the best value

Cost is the generation cost per player program: the equal-weight mean over each model's unique game and reasoning-setting cells, on economics-eligible games only. Each tooltip names its evidence basis (billed, provider-reported, or an estimate from list prices, which is not an invoice) with the median and run count. The score is the headline score on the same games: each failed generation attempt counts as 0, and its cost is included. Every model on this page is a point; the selected models are named. The best-value line runs through the points with complete cost evidence that no other such point matches or beats, that is, none costs the same or less and scores as high, in Per reasoning setting over every model and setting; hollow markers have incomplete cost evidence (known charges only), never join the line and are left out of that comparison. The cost axis is logarithmic, so a cost of $0 sits in its own lane left of the axis break: filled when no charge was recorded on complete evidence, hollow when the evidence is incomplete and the true cost unknown. “All settings” combines the public reasoning settings; Baseline, Balanced and Intensive each select the provider setting assigned to that role for each model, and Per reasoning setting always shows all three. Output-token and API-latency views are in the next chart.

How much does more reasoning help each model?

How to read this chart

Each row shows one model's three settings (headline score, 0–100; each failed generation attempt counts as 0); Gain is Intensive minus Baseline.

Baseline, Balanced and Intensive are each model's own lowest, middle and highest tested provider settings, so the literal setting differs between models; hover a marker or open the numbers to see it. A heavier setting can score lower than a lighter one.

How consistent is each model from game to game?

How to read this chart

All settings: each dot is one game (averaged across settings); the ring is the model's headline score, where each failed generation attempt counts as 0

How much does each model write, and how fast?

How to read this chart

Both measures cover the code-generation and repair responses on economics-eligible games, summed per run. Output tokens are the provider-reported generated tokens, reasoning tokens included where the provider reports them, so they are more than the visible code. API latency is the time our client waited for each provider call, which can include queueing and retries within the call; it excludes local validation and compile checks, and batch requests have none. A run counts in each view only with a recorded value, so the two views can cover different runs. Output tokens/sec divides the summed output tokens by the summed latency of the same responses, those that record both from a single request. These axes are logarithmic.

Where is each model strong or weak?

How to read this chart

Headline score per game, averaged across reasoning settings; each failed generation attempt counts as 0

Values use one colour, faint for low and strong for high. Difference from a model shows each value minus the chosen model's value in the same game, setting and score type, on a scale centred on 0: above 0 (blue) the row model scores higher, below 0 (orange) lower; it compares two published values and ranks nothing. A cell without a value says why: fail, every counted attempt failed (the headline score counts it as 0); slow, rejected by the speed check and not counted; excl, lost to a provider, API or test-machine fault and not counted; wait, a working program not rated yet; –, not tested; and in difference mode n/a, the chosen model has no value for that game.

Who beats whom?

How to read this chart

Combined view across tested provider settings.

Each cell is the row model's score against the column opponent: wins plus half the draws, as a share of their matches (W-L-D in the tooltip), so the two cells of a pair add up to 100. Use Reasoning view to show one reasoning setting.

Program reliability How often each model writes a player program that works
How to read this chart

How each model's generation attempts ended, all reasoning settings pooled

Each bar counts one model's generation attempts in the active reasoning view. Each failed attempt counts as 0 in the headline score, so a model with failures has a headline score below its working-program strength. A program rejected only for being too slow does not play and does not count; attempts that failed for reasons outside the model's code, such as provider, API or test-machine faults, are left out. Working gives the working programs out of the counted attempts.

Working code vs strong code Does code that works also play well?
How to read this chart

Each point is one model on this page; selected models are named

Program success rate is working programs divided by counted attempts, from the same counts as Program reliability: an attempt counts when it produced a working program or failed in the model's own code; a program rejected only for being too slow, and an attempt lost to a provider, API or test-machine fault, does not count. Working-program strength is the score of the programs that worked, with failed attempts left out (the leaderboard's column of the same name); All settings pools the three settings' attempts and averages the strengths of the settings that have one, as the Overall score averages its setting scores. Marker area shows how many counted attempts a rate rests on, so 100% from eight attempts reads as weaker evidence than 100% from 24. A model without a rated working program has no strength: it is listed under the chart, not drawn.

Game spread Which games separate the selected models most
How to read this chart

Each row is one game; dots are the selected models' working-program strength in the active reasoning view (failed generation attempts left out). Games are ordered by the standard deviation of those scores, shown at the right: a wide spread means the game separates stronger from weaker programs, a narrow one that the selected models play it about equally well.

Cycle detector Non-transitive matchups between selected models
How to read this chart

What it finds: Leaderboard averages can hide matchup loops. The detector searches the currently selected models and reasoning view for cases where A outperforms B, B outperforms C, and C outperforms A.

How to read it: In A → B → C → A, each arrow points from the model with the higher pair score to the model it outscored. Each result lists its three edges with their pair score and win-loss-draw record.

Each arrow requires at least four matches and a pair score above 50%, with a draw counting as half a win. Results are ordered by the cycle’s weakest edge: weakest edge +x pp is the smallest of the three margins above 50%, while n is the total match count across all three edges. The detector shows up to ten cycles. These are descriptive matchup patterns, not tests of statistical significance. “No cycles” means none met these rules in the current selection and reasoning view; it does not prove that the matchups are transitive.

Static rankings — current page release

This table remains available without chart scripts. It describes this page's release; restoring a requested comparison and its filters requires JavaScript. Open benchmark rankings for the detailed tables.

Overall leaderboard scores

All officially ranked Overall models from the same data as the interactive charts. Partial rows keep their official rank and show that their evidence is incomplete.

Rank Model Evidence status Mean Min Max
1Claude Fable 5.2Complete evidence95.079.799.8
2GPT-6.2 AstraComplete evidence93.682.799.5
3GPT-6.2 Astra ProComplete evidence92.265.999.6
4GPT-6.2 SolComplete evidence91.566.898.6
5Gemini 4 NeonComplete evidence91.374.499.8
6GPT-6 Astra ProComplete evidence90.271.199.2
7Gemini 4 CarbonComplete evidence90.163.099.2
8GPT-6.2 Sol ProComplete evidence89.620.998.3
9GPT-6 AstraComplete evidence88.772.698.9
10Claude Opus 5.5Complete evidence87.650.499.5
11Claude Fable 5.1Complete evidence86.963.998.0
12GPT-6.1 SolComplete evidence84.950.198.0
13GPT-6.1 Sol ProComplete evidence82.829.796.7
14GPT-6.1 Terra ProComplete evidence82.333.295.5
15Gemini 4 ArgonComplete evidence81.457.496.6
16GPT-6.1 TerraComplete evidence81.160.596.7
17Kimi K3.1 ProComplete evidence78.314.795.6
18Gemini 4 HydrogenComplete evidence78.238.799.2
19Claude Opus 5Complete evidence76.746.594.9
20GPT-6 SolComplete evidence73.09.793.3
21Qwen4 MaxComplete evidence68.40.094.6
22GLM-5.3Complete evidence67.438.993.2
23GPT-5.6 SolComplete evidence66.838.988.9
24Claude Fable 5Complete evidence65.720.793.8
25Muse WonderComplete evidence64.817.991.8
26Grok 4.7Complete evidence63.70.092.5
27Nex N2.6Complete evidence61.926.589.3
28DeepSeek V4.1 Flash 1017Complete evidence61.332.380.5
29Claude Sonnet 5.5Complete evidence60.414.299.7
30Grok 4.6Complete evidence59.412.283.5
31Gemini 3.8 FlashComplete evidence58.939.186.8
32Particle-1Complete evidence58.024.491.0
33Muse Spark 1.3Complete evidence57.725.181.5
34GLM-5.3 FlashX LegalComplete evidence57.10.092.9
35Grok 4.5Complete evidence56.715.889.0
36GLM-5.3 FlashXComplete evidence56.217.081.5
37Claude Haiku 5.5Complete evidence56.015.291.1
38Muse Spark OpenComplete evidence55.131.389.3
39GLM-5.3 FlashComplete evidence54.815.779.1
40Qwen3.8 Max 0902Complete evidence54.225.590.4
41GPT-5.4Complete evidence53.40.096.3
42DeepSeek V4.1 FlashComplete evidence52.923.780.1
43Qwen3.8 2.4T A95BComplete evidence52.831.587.2
44Muse EmberComplete evidence51.80.085.7
45Muse Spark 1.2Complete evidence51.518.384.1
46Gemini 3.6 FlashComplete evidence50.617.881.6
47Ox AlphaComplete evidence50.518.072.4
48Hy4 PreviewComplete evidence50.40.085.2
49Ember-1Complete evidence50.210.086.8
50Claude Opus 4.8Complete evidence49.60.084.2
51Qwen3.8 MaxComplete evidence49.621.488.6
52Gemini 3.5 FlashComplete evidence49.511.589.9
53GPT-5.5Complete evidence49.40.084.9
54Mistral Large 4 (Oct 29)Complete evidence48.30.084.2
55Kimi K3Complete evidence47.90.091.8
56GPT-6 LunaComplete evidence47.615.682.6
57Claude Opus 4.5Complete evidence47.332.674.3
58Claude Sonnet 5Complete evidence46.622.577.9
59GPT-5.6 TerraComplete evidence45.70.077.0
60Mercury 3Complete evidence45.30.076.1
61Gemini 3.1 Pro PreviewComplete evidence44.413.786.6
62Inkling 2 SmallComplete evidence43.10.072.9
63GLM-5.2Complete evidence43.110.774.7
64Beam 1.0 501B A23BComplete evidence42.80.079.8
65DeepSeek V4 Pro 0813Complete evidence41.80.076.4
66GPT-5Coverage incomplete: one or more documented settings are untested or ineligible41.30.072.5
67DeepSeek V4 ProComplete evidence41.10.080.3
68GPT-5.6 LunaComplete evidence40.27.181.3
69Minimax M3.1 FlashCoverage incomplete: the provider capability record is unavailable36.80.070.5
70Seed 2.0 CodeComplete evidence35.10.084.6
71DeepSeek V4 FlashComplete evidence34.10.079.2
72Hy3Complete evidence34.10.064.9
73o3Complete evidence34.10.056.5
74Qwen3.8 27BCoverage incomplete: generation quality is below the publication threshold33.10.075.4
75Space Bunny AlphaComplete evidence32.40.072.2
76Nemotron 3 Ultra 550B A55BCoverage incomplete: one or more documented settings are untested or ineligible31.70.086.5
77Gemini 3.5 Flash LiteComplete evidence31.010.460.2
78Inkling SmallComplete evidence29.25.354.8
79Muse Glimmer 30BComplete evidence28.70.048.5
80GPT-OSS 120BComplete evidence28.50.056.6
81InklingComplete evidence25.60.047.9
82Step 3.7 FlashComplete evidence23.20.046.9
83Mercury 2.5 PreviewComplete evidence20.90.045.3
84Hy4Coverage incomplete: every generation attempt of a reasoning setting reached the model's own output-token limit before finishing a program, so that setting scores 016.20.079.1
85Gemini 3.1 Flash LiteCoverage incomplete: one or more documented settings are untested or ineligible15.10.041.9
86Solar Mini 4Coverage incomplete: generation quality is below the publication threshold9.10.044.5
87Granite 4.2 8BCoverage incomplete: every generation attempt of a reasoning setting failed, so that setting scores 0; generation quality is below the publication threshold1.60.037.7