Raising the bar from playable to good
Today's models can generate browser games that load and respond to input. But anyone who has played enough AI-generated games knows how far that is from a good game. The controls can fight the player. The HUD can obscure the action. The first thirty seconds can contain every idea the game has.
FunBench measures whether a model–harness system can turn a single prompt into a game that is not only playable, but well designed.
What makes FunBench different:
- We evaluate the game by playing it. Every artifact runs in the same browser environment and is explored through ordinary keyboard and pointer input.
- We measure quality beyond correctness. The benchmark covers controls, camera, interface, feedback, progression, polish, replay, and whether the game delivers what was requested.
- We check the benchmark against players. Prompt-matched votes from Instaplay's Arena tell us whether higher-scoring games are also the games people prefer.
Why we built FunBench
Most coding benchmarks ask whether an implementation is correct. That is necessary for games, but it misses much of the work.
A game can compile, render, and accept input while still being confusing or flat. The player may not know what to do. Actions may have weak feedback. Difficulty may never change. A polished opening screen may lead to a broken game-over state. None of these failures are visible in a unit test alone.
Game generation puts several capabilities under pressure at once: long-horizon coding, visual reasoning, interaction design, pacing, and judgment about how another person will experience the result. FunBench turns those failures into a score that can be inspected instead of a vibe that can only be argued about.
How FunBench works
Each benchmark item starts with a game request drawn from the kinds of things people actually ask to create. Multiple model–harness systems receive the same request under comparable authoring budgets. Their games are then frozen and evaluated as players encounter them.
| Stage | What happens | Why it matters |
|---|---|---|
| Coverage agent | A vision-guided agent plays the game with normal keyboard and pointer actions. | Reaches the states needed to evaluate the experience. |
| Evidence adapter | The runtime records frames, actions, state changes, errors, audio, game-over events, and restarts. | Creates an auditable record of what happened during play. |
| Verifiers | Deterministic checks and multimodal evaluators score the relevant evidence. | Uses direct measurement where possible and judgment where necessary. |
| Human validation | Scores are compared with prompt-matched player preference. | Tests whether the benchmark is directionally right. |
The coverage agent starts with the same information a player has. It is not given the creator prompt, a control sheet, the DOM, or the source code. It cannot repair the game. Game time is paused while the agent thinks so slower model inference does not make a real-time game harder.
Beyond a single “fun” score
FunBench scores eleven parts of the player experience: core-loop clarity and agency, controls feel, camera behavior, HUD/UI construction, feedback and juice, progression and difficulty, visual coherence and polish, audio, lifecycle and onboarding, replay pull, and prompt–scope fit.
Keeping these scores separate is important. A game can have good controls and no progression. It can look polished and fail to restart. It can follow the prompt closely and still give the player no reason to continue.
The dimension breakdown also makes the benchmark useful for development. A lab can see whether a model improvement produces better mechanics, stronger interfaces, more complete game loops, or simply fewer runtime failures.
Not everything should be graded by an LLM
FunBench uses the strongest evaluator available for each claim. The runtime can directly check whether the game loaded, whether input changed state, whether HUD text overflows, whether a score updated, whether audio played, and whether game over and restart worked.
Multimodal evaluators handle the questions that require judgment. Is the HUD readable during play? Does the difficulty curve feel intentional? Does a camera follow the conventions of a first-person, third-person, top-down, or fixed-camera game? Do the effects make actions feel consequential?
The two work together. Every judgment points back to frames, actions, and runtime observations. If the run never reaches the state needed for a score, the evaluator can return insufficient evidence instead of turning uncertainty into a zero.
Grounded in what players choose
Instaplay's Arena already contains pairwise comparisons between games created from the same request. Players see both games and choose the one they prefer. This gives us a direct check on FunBench: when the benchmark scores one game higher, do players tend to choose it too?
We use separate prompt groups to develop the rubric and test it. The final release will report pairwise agreement, ranking correlation, and which dimensions are most predictive of preference.
Player preference does not define the score. Votes are noisy and taste varies. It is the external check that tells us whether the benchmark is measuring qualities that matter in practice.
Models and harnesses both matter
The harness controls the tools a model receives, how it observes its work, what it remembers, and how it recovers from mistakes. The same model can produce meaningfully different games under different harnesses.
FunBench therefore reports two rankings:
- Model ranking — an aggregate view of model performance across controlled authoring conditions.
- Model × harness ranking — the performance of each complete game-development stack.
The model ranking asks which models carry the strongest underlying capability. The stack ranking asks which systems turn that capability into the best finished games.
What comes next
We are finishing the evaluator suite and calibrating it against held-out Arena comparisons. Once the benchmark is frozen, we will publish model rankings, model × harness rankings, dimension-level results, playability rates, and evaluator agreement with player preference.
Games make the gap between code that runs and software that works unusually easy to see. FunBench is designed to measure that gap.