how it works

the puzzle

A nonogram is a grid puzzle. Numbers beside each row and column describe consecutive groups of filled cells. Each group needs at least one empty cell between it and the next. You solve it by filling the grid so every clue matches.

The 30 Standard puzzles come from the Nonograms dataset by Gabriel Moyà-Alcover, licensed CC BY 4.0. The ten Hard mode puzzles are generated for Nonobench.

the prompt

Models receive instructions and the clues for each puzzle. Here is the full input for one Standard 5×5 puzzle. Only the cell count and the clues change between puzzles.

You are solving a nonogram (also known as picross or griddlers).

## Rules
- Each row and column has clues: numbers indicating consecutive groups of filled cells
- Groups are separated by at least one empty cell
- The clues appear in order from left-to-right (for rows) or top-to-bottom (for columns)

## Example
A row clue "2 1" on a 5-cell row means: 2 filled cells, then a gap, then 1 filled cell.
Possible solutions: "11010" or "11001" (but only one will satisfy all column constraints)

## Your Task
Solve the puzzle so ALL row AND column clues are satisfied simultaneously.

## Output Format
Output ONLY the solution as a single string of 25 characters.
- Use "1" for filled cells, "0" for empty cells
- Read left-to-right, top-to-bottom (row 1 first, then row 2, etc.)

IMPORTANT:
- Do NOT include any explanation, reasoning, or intermediate steps
- Do NOT include any other text, formatting, or symbols
- Before outputting, ensure the solution satisfies every row and every column clue
- If no solution satisfies all constraints, do NOT guess; output "0" instead

You MUST ONLY output the 25-character solution string and nothing else.

Row clues:
  Row 1: 1
  Row 2: 2
  Row 3: 4
  Row 4: 2 1
  Row 5: 2

Column clues:
  Column 1: 2
  Column 2: 2
  Column 3: 1
  Column 4: 2 1
  Column 5: 2 2

grading and scores

Any grid that satisfies every row and column clue counts as correct. Some puzzles have more than one valid solution, so an answer doesn’t have to match one reference grid. Standard scores cover 30 core puzzles: ten each at 5×5, 10×10, and 15×15. The thin lines on the leaderboard show 95% Wilson intervals. With 30 puzzles, small score differences may not mean much.

models and effort

Every model runs on its own lab’s endpoint, through OpenRouter. Where a model supports adjustable reasoning effort, we show its measured levels from off through max. Models with only an on/off switch show off and on. On uses the provider’s default level.

Hard mode

Hard mode adds ten 20×20 puzzles, filled at random so there is no picture to guess. Every one has a single solution, and five need more than line logic. Only the strongest current models run it, and its results stay separate from the Standard score. Hard mode answers are written one row per line (20 lines of 20 characters). As a single 400-character string, most models lost count before the logic even mattered.

data and tools

Explore the puzzles and puzzle insights, download raw runs, or use the API (/api/v1), OpenAPI spec, and MCP endpoint. The source is on GitHub.

limitations

• Each variant gets one attempt per core puzzle.
• “Best observed level” picks the highest score across measured effort levels, which can flatter families with more levels.
• For a few models, output mode was chosen using a 5×5 comparison on benchmark puzzles.
• Ten core puzzles have multiple valid solutions; grading checks clue satisfaction.
• Standard answers are one unbroken string of 0s and 1s, as in every earlier version. On larger grids models sometimes write the wrong number of cells, so Standard scores partly measure counting as well as solving. The format stays unchanged so all versions remain comparable; Hard mode uses one row per line.
• Standard answers use each provider’s default output limit (65,536 or 131,072 tokens). A few long-reasoning answers hit that limit and count as unsolved. Hard mode gives every model the same 128,000-token budget, or its endpoint’s maximum if lower.
• Cost and time are measured through OpenRouter and include provider speed. Earlier versions used a different harness.