Benchmark menu

Model results

O3

See how O3 performed, which reasoning settings were tested, how often it produced a working player program, and what those runs cost.

O3

Provider: OpenAI

Rank
28
Status
Partial 3/3
Rank change
Baseline
Score
37.5
Last tested
2026-08-23
Settings tested
All settings tested

Why this result is partial: generation quality is below the publication threshold. The rank uses the available reasoning-setting results.

Scores by reasoning setting

Available settings

LowMediumHigh

Checked on 2026-07-21 · Provider documentation

Scores from tested settings

Tested settingScoreResult statusDisplay group
Low22.8Tested and includedBaseline
Medium46.7Tested and includedBalanced
High42.8Tested and includedIntensive

How settings are grouped

  • Baseline: Low — tested and included
  • Balanced: Medium — tested and included
  • Intensive: High — tested and included

Player program results

High: 59% / 0% / 41% · n 32Medium: 75% / 0% / 25% · n 8Low: 50% / 0% / 50% · n 8 All settings testedMany programs failed

Generation cost

Average estimated cost
$0.08
Median estimated cost
$0.09
Estimated range
$0.04–$0.17
Cost data
24 combinations / 48 recorded runs
Observed total
$4.20
Price source
Combined: Recorded attempt + Standard list-price estimate
Price list
boardgame-list-prices-2026-07-12 + boardgame-list-prices-2026-08-08
Average output
8.2k tokens
Output range
2.7k–16k tokens
Samples
48

Estimated cost gives equal weight to each model, game, and reasoning-setting combination.

Test setup

  • Combines this model's tested reasoning settings
  • Uses the standard GameBench player-program prompt