Models

Model set

Every model with an official Overall rank is shown by default. Partial results keep their rank; the icon explains what evidence is incomplete.

Reasoning view

Rank by

Reasoning view applies to Scores, Reasoning variants, Head-to-head, Per game heatmap, Economics (when available), and Cycle detector.

Compare two entries

Select two models to inspect their evidence together.

Rankings

Selected models, ordered by the Rank by choice above

Score is the public leaderboard score; the other Rank by choices show diagnostic views when the selected models have that data.

Scores

Overall min-max and mean; small dots are each game (averaged across reasoning variants where scores exist); game dots show working-program strength, without failed generation attempts

Economics

Coming soon

Generation cost versus score will appear when the economics segment is available.

Overall

Reasoning variants

Per game

Working-program strength heatmap (per-game scores of working programs averaged across reasoning variants; failed generation attempts count only in the headline score)

Head-to-head

Combined view across tested provider settings.

Off-diagonal cells use win rate for the row model against the column opponent (W-L-D in the tooltip). Use Reasoning view to show one reasoning variant.

More charts

Less-used views open on demand, so the page stays short.

Per-game comparison Top selected models within each game

Each panel ranks selected models within one game for the active reasoning view, by working-program strength; failed generation attempts count only in the headline score.

Cycle detector Non-transitive matchups between selected models

What it finds: Leaderboard averages can hide matchup loops. The detector searches the currently selected models and reasoning view for cases where A outperforms B, B outperforms C, and C outperforms A.

How to read it: In A → B → C → A, each arrow points from the model with the higher pair score to the model it outscored. Hover over a result for each edge’s win-loss-draw record and pair score.

Each arrow requires at least four matches and a pair score above 50%, with a draw counting as half a win. Results are ordered by the cycle’s weakest edge: weakest edge +x pp is the smallest of the three margins above 50%, while n is the total match count across all three edges. The detector shows up to ten cycles. These are descriptive matchup patterns, not tests of statistical significance. “No cycles” means none met these rules in the current selection and reasoning view; it does not prove that the matchups are transitive.

Output tokens and API latency How much each model wrote, and how fast

These views will appear when the economics segment is available.

Static rankings — current page release

This table remains available without chart scripts. It describes this page's release; restoring a requested comparison and its filters requires JavaScript. Open benchmark rankings for the detailed tables.

Current comparison tables

Overall leaderboard scores

All officially ranked Overall models from the same data as the interactive charts. Partial rows keep their official rank and show that their evidence is incomplete.

Model	Evidence status	Rank	Mean	Min	Max
GPT-6 Astra	Complete evidence	1	89.7	72.3	98.9
Claude Opus 5.5	Complete evidence	2	88.7	52.9	99.3
Claude Fable 5.1	Complete evidence	3	88.5	63.8	98.2
GPT-6.1 Sol	Complete evidence	4	85.8	51.6	98.1
Gemini 4 Argon	Complete evidence	5	82.9	59.9	97.3
Claude Opus 5	Complete evidence	6	78.2	44.3	96.0
GPT-6 Sol	Complete evidence	7	74.6	10.4	94.8
GPT-5.6 Sol	Complete evidence	8	68.4	41.0	89.1
Claude Fable 5	Complete evidence	9	67.0	20.6	92.5
Muse Wonder	Complete evidence	10	66.2	16.9	92.9
GLM-5.3	Complete evidence	11	65.5	0.0	92.7
Grok 4.7	Complete evidence	12	65.0	0.0	93.5
DeepSeek V4.1 Flash 1017	Complete evidence	13	63.3	32.0	85.4
Claude Sonnet 5.5	Complete evidence	14	61.7	14.1	98.9
Grok 4.6	Complete evidence	15	60.9	13.7	85.2
Gemini 3.8 Flash	Complete evidence	16	60.6	35.5	87.8
Muse Spark 1.3	Complete evidence	17	59.7	26.6	84.4
Muse Spark Open	Coverage incomplete: some games cannot be scored: their only generation attempts failed because of the test harness or the provider and cannot currently be repeated	18	58.3	32.7	90.5
Grok 4.5	Complete evidence	19	58.1	16.1	90.2
GLM-5.3 FlashX	Complete evidence	20	57.5	18.8	85.0
GLM-5.3 Flash	Complete evidence	21	57.2	18.2	81.2
Claude Haiku 5.5	Coverage incomplete: some games cannot be scored: their only generation attempts failed because of the test harness or the provider and cannot currently be repeated	22	57.1	16.6	91.3
GPT-5.4	Complete evidence	23	55.4	0.0	97.8
Qwen3.8 Max 0902	Complete evidence	24	55.1	26.4	92.4
Qwen3.8 2.4T A95B	Complete evidence	25	54.5	31.9	90.9
Hy4 Preview	Complete evidence	26	53.6	0.0	89.9
Muse Spark 1.2	Complete evidence	27	53.5	19.8	85.6
DeepSeek V4.1 Flash	Complete evidence	28	53.5	24.0	82.5
Ember-1	Complete evidence	29	52.9	22.0	86.0
Gemini 3.6 Flash	Complete evidence	30	52.4	18.8	83.8
Ox Alpha	Complete evidence	31	52.3	21.6	73.9
Claude Opus 4.8	Complete evidence	32	51.8	0.0	83.0
Gemini 3.5 Flash	Complete evidence	33	51.1	11.9	90.3
GPT-5.5	Complete evidence	34	50.5	0.0	87.4
Qwen3.8 Max	Complete evidence	35	50.0	20.8	90.5
GPT-6 Luna	Complete evidence	36	48.9	28.1	86.2
Kimi K3	Complete evidence	37	48.7	0.0	93.4
Claude Opus 4.5	Complete evidence	38	48.6	32.1	77.3
Claude Sonnet 5	Complete evidence	39	48.0	23.0	81.2
Mercury 3	Coverage incomplete: some games cannot be scored: their only generation attempts failed because of the test harness or the provider and cannot currently be repeated	40	47.9	0.0	79.6
GPT-5.6 Terra	Complete evidence	41	47.0	0.0	79.9
Gemini 3.1 Pro Preview	Complete evidence	42	46.4	14.9	85.1
Beam 1.0 501B A23B	Complete evidence	43	45.0	0.0	82.6
GLM-5.2	Complete evidence	44	44.6	10.0	78.8
DeepSeek V4 Pro 0813	Complete evidence	45	44.1	0.0	79.4
DeepSeek V4 Pro	Complete evidence	46	43.9	0.0	84.3
GPT-5	Coverage incomplete: one or more documented settings are untested or ineligible	47	43.0	0.0	75.9
GPT-5.6 Luna	Complete evidence	48	41.7	11.5	82.5
Minimax M3.1 Flash	Coverage incomplete: the provider capability record is unavailable	49	38.1	0.0	74.0
o3	Complete evidence	50	36.2	0.0	58.7
Seed 2.0 Code	Complete evidence	51	36.0	0.0	87.3
Qwen3.8 27B	Coverage incomplete: generation quality is below the publication threshold	52	34.4	0.0	78.2
DeepSeek V4 Flash	Complete evidence	53	34.1	0.0	82.6
Space Bunny Alpha	Complete evidence	54	33.4	0.0	74.7
Gemini 3.5 Flash Lite	Complete evidence	55	33.3	10.2	62.2
Inkling Small	Complete evidence	56	32.3	5.4	56.5
Muse Glimmer 30B	Complete evidence	57	30.6	0.0	49.5
GPT-OSS 120B	Complete evidence	58	30.2	0.0	58.0
Nemotron 3 Ultra 550B A55B	Coverage incomplete: one or more documented settings are untested or ineligible; some games cannot be scored: their only generation attempts failed because of the test harness or the provider and cannot currently be repeated	59	27.6	0.0	62.1
Inkling	Complete evidence	60	26.7	0.0	48.8
Step 3.7 Flash	Complete evidence	61	24.2	0.0	50.9
Mercury 2.5 Preview	Complete evidence	62	22.0	0.0	46.0
Gemini 3.1 Flash Lite	Coverage incomplete: one or more documented settings are untested or ineligible	63	16.4	0.0	44.7
Solar Mini 4	Coverage incomplete: generation quality is below the publication threshold; some games cannot be scored: their only generation attempts failed because of the test harness or the provider and cannot currently be repeated	64	8.2	0.0	46.6
Granite 4.2 8B	Coverage incomplete: every generation attempt of a reasoning setting failed, so that setting scores 0; generation quality is below the publication threshold	65	1.6	0.0	38.7

Charts

Chart controls