A navigation benchmark
Can an AI find the real way
out of a mirror maze?
The ideal run reaches the exit without hitting a wall. One recorded run hit walls 33,607 times and still exhausted its planning budget.
Visual and text studies · November 2026
The benchmark
Three actions in a reflective world
A reflection can look like a corridor or duplicate the green goal beacon. The agent must enter the actual goal cell. Each planning round may contain a sequence of actions. A collision does not terminate a valid sequence.
leftRotate one headingforwardAttempt one adjacent cellrightRotate one headingSquare cells use 90° turns and hexagonal cells use 60°. The visual protocol supplies first-person images after each attempted action plus execution feedback. The text protocol supplies ordered action outcomes only during play. Agents receive no review map.
| Difficulty | Square | Hexagonal | Planning cap |
|---|---|---|---|
| Easy | Side 7 · 49 cells | Side 5 · 61 cells | 200 |
| Medium | Side 11 · 121 cells | Side 7 · 127 cells | 400 |
| Hard | Side 15 · 225 cells | Side 9 · 217 cells | 800 |
Two deterministic maps per shape and difficulty, MIRROR-042 and MIRROR-043. Visual host observations use 1280 × 720 images and up to 10 mirror reflections. The benchmark prioritizes fewer planning calls. The analysis also reports primitive actions, physical wall hits, and coverage.
Visual study
What the completed runs show
GPT-6.1 Sol with Codex cleared all four Easy maps and both Hard square maps. The denominators below include measured outcomes only.
The visual cohort has 64 scheduled cases, including four policy skips and one withdrawal. Four Sol Medium successes retain historical caps of 500, 500, 200, and 200. Each finished within the current 400-call target.
| Model | Native client | Easy | Medium | Hard |
|---|
Main findings
An exit tells only part of the story
Text cleared more goals under the planning cap. Vision cleared more goals within a small collision budget. The route, the number of wall hits, and coverage show how those outcomes were achieved.
The result changes with the budget
The 32 public pairs match the maze, initial cell and heading, and goal. Both protocols cleared 13 of those pairs, text alone cleared 10, and both reached the cap on nine.
| Model | Vision | Text only |
|---|---|---|
| GPT-6.1 Sol | 8/8 | 8/8 |
| Gemini 3.8 Flash | 3/8 | 7/8 |
| DeepSeek V4.1 Flash | 1/8 | 5/8 |
| Kimi K3.1 Pro | 1/8 | 3/8 |
| Total | 13/32 | 23/32 |
| Constraint | Vision | Text only |
|---|---|---|
| Original planning caps | 13/32 | 23/32 |
| At most 20 wall hits | 8/32 | 0/32 |
| At most 50 wall hits | 11/32 | 2/32 |
| At most 800 actions | 11/32 | 9/32 |
| At most 1,600 actions | 13/32 | 15/32 |
The extra limits filter recorded trajectories. Running with a new budget could produce different plans. DeepSeek model routes, AGY versions and prompts, and four declared historical Sol budgets differ across parts of the comparison. The results compare complete systems.
Change the budget interactively ↗ · Explore the text trajectories ↗
How agents spend actions and hit walls
A planning call can buy a long sequence of probes.
Gemini 3.8 Flash cleared Easy Square MIRROR-042 in 9 text rounds with 268 wall hits, compared with 116 visual rounds and 40 hits. Sol cleared all eight maps in both arms. On Easy Hex MIRROR-042, vision took 315 actions with zero wall hits. Text took 8,846 actions with 3,823 hits. Across its eight maps, Sol’s median fell from 889.5 hits in text to 12 in vision.
When an agent repeats a hit matters.
On the first attempts at the cell and heading pairs they chose to probe, Sol hit walls on 9.1% of visual attempts and 56.7% of text attempts. Gemini showed 28.8% versus 54.1%. These are policy-selected poses.
When a hit had another action in the same submitted batch, the immediate next action hit the same face in 31.6% of text events and 21.8% of visual events. After a round-ending hit and fresh feedback, the next round’s first forward attempt hit the same face in 1.2% and 4.5%, respectively. DeepSeek accounted for 94 of 96 visual repeats.
On revisits to a previously hit pose after 21 or more rounds, 39.9% of text and 3.9% of visual plans started by moving into the wall. The policies revisit different poses at different times, so this motivates a controlled memory test.
Coverage reveals stalled or repetitive exploration.
Sol hit only 2 walls on Hard Hex MIRROR-043 and saw no new cell in its final 581 rounds. DeepSeek took 111,088 actions and 33,607 physical wall hits on Medium Hex MIRROR-043 and still visited only 69 of 127 cells.
Repeated images are a candidate explanation.
DeepSeek’s visual runs recorded roughly 49,000 delivered image entries, about 97% duplicates, and 744 rounds with image trimming. Repeated movement produces redundant views. Whether redundant views also impair planning calls for a controlled image-history comparison.
A blind policy can solve the original mazes
A fixed wall follower cleared 8/8 original loop-free mazes using only success or wall feedback. All canonical starts face their only opening. Across all initial headings it finished within budget on 36/40 trials. Random turns and forward probes with no feedback cleared 23/4,096 within 800 primitive actions.
Adding 12 passages to each map reduced fixed wall-following success within budget to 55.2%. Depth-first search with relative position memory still reached 98.0%. An ideal local wall sensor yields a separate collision-free reference of 31–153 passage decisions on the original maps.
In 16 mirror-versus-matte image pairs, some blocked directions showed the goal’s reflection only in the mirror image. The model pilot yielded no complete paired verdicts. The rendering effect is visible, while the rate of model misclassification remains open.
The hexagonal gap depends on the protocol
In Easy vision, square maps cleared 6/8 and hexagonal maps 2/8. In Easy text, both shapes reached 7/8. Sol cleared both Hard square maps and neither Hard hexagonal map even though they have 225 and 217 cells, respectively. Heading changes, local connectivity, rendering, and the two map seeds per shape all matter.
Interactive replay
Step through a recorded trajectory
Use the arrow buttons to advance one primitive action, one planning round, or one original image sample. Filter the record list below to find a run. The visual collection contains Easy and Medium traces plus the Hard Sol traces.
Experiment records
No records match these filters. Change a filter or clear them to see all records.
Loading trajectory…
The harness changes the outcome
The table compares the same four Easy maps for each listed model in the visual study. Sol improved with Codex. DeepSeek performed better in its API runs, and Kimi was unchanged.
| Model | Native client | Direct API |
|---|---|---|
| GPT-6.1 Sol | 4/4 · Codex | 1/4 |
| DeepSeek V4.1 Flash | 1/4 · Claude Code | 2/4 |
| Kimi K3.1 Pro | 1/4 · Claude Code | 1/4 |
In the three failed Sol API runs, 868 of 970 wall hits came from the final planning batch. The earlier traces often rotated or stopped discovering new cells. Prompting, conversation management, and reasoning-state retention vary with the system. The Gemini API control uses a different model from the native Flash profile.
How interrupted cases are counted
Four Flash Hard trials remain not evaluated because provider quota notices interrupted their usable trajectories. Two Easy Gemini 4 Argon API trials remain paused. These cases, the four policy skips, and one withdrawal are outside measured-outcome denominators.
Inspect and reproduce the evidence
Inspect the score data, compare all eight paired case videos, or try the browser environment. Public replays contain action and pose data, with physical wall hits tracked separately from invalid-plan penalties.
Cite this project · BibTeX
@misc{kenny2026mirrormaze,
author = {kenny},
title = {{Mirror Maze}: A Benchmark for Reflective Navigation},
year = {2026},
url = {https://kennymckormick.github.io/blog/mirror-maze/}
}