Escaping the maze

Can an agent find its way out of a maze? What about if they can’t see the whole thing, but only what’s in front of them? When I was asked to make a simple demo of how agents interact with the canvas and work spatially, I thought it’d be fun to let an LLM control a shape in a maze and navigate its way out. But that quickly snowballed into a full-blown exploration of how different agents, different prompts, and different factors improve spatial agent performance.

I decided to set up a series of races between agents as an experiment. Each race tested 3 models (Haiku, Sonnet, and Opus) against a couple of heuristic approaches to see how model size, context provided, and prompts affected their ability to escape a different maze.

In the end, the smallest model (haiku in this case) performed best, even if it wasn’t always the most precise or accurate, finishing fastest in 163 of the 240 races compared to the other models. Often, it was more efficient than the heuristic approaches it raced against, even though it sometimes lost on speed because they didn’t have to make API calls to determine their paths.

Setting up the maze

In each race, we had 6 racers (3 heuristic approaches and 3 agents) in a maze. All started at the same point and worked toward the same finish line. The maze is a series of tldraw shapes locked to the canvas that the agent shapes can’t move into. I built them all in tldraw offline because it was incredibly quick to spin up a demo environment and give agents access. I ran the agents through three types of mazes, ranging from cave-like mazes to more classical maze setups.

I wanted to test how the agents could work within spatial constraints, so I didn’t give them complete maze context and let them programmatically map a path. Instead, they could only “see” a given radius in front of them andcouldn’tt see through walls. That led them to build up an idea of what the maze looks like over time and forced them to explore their way through. One variable we played with was toggling the line-of-sight radius to see whether more context helped the model.

Speed beats power

The first few times I ran the races, I was surprised to see Haiku beat the more powerful models. I figured maybe it was a fluke or maybe it was the maze type or the context I was giving. But after 240 races, I was shocked by how consistently it kept beating out Sonnet or Haiku.

Haiku was the best-performing model in 68% of the runs.

It wasn’t winning because it was doing a better plan or because it was fundamentally better at solving the maze - it averaged more planning calls/decision points per run than either of the more powerful models. Haiku could do that whole process faster and cheaper than the bigger models, which let it explore the maze faster. It still made tons of wrong turns - it could just be wrong and course-correct faster than Sonnet, or especially faster (and way cheaper) than Opus.

Speed, efficiency, or cost?

Speed isn’t the only thing that decides whether an agent is good or bad at a task. The most important thing is whether it succeeds at the task, and each agent finished almost every run (minus the occasional struggle for Sonnet). The two other factors that mattered were efficiency, how much ground each run actually covered, and cost, the actual API cost for each run.

Most heuristic racers beat the models on speed (they don’t need to make any API calls, so they get an unfair advantage) and cost (they’re all free by definition). But efficiency gives a great head-to-head comparison. Even though Opus was consistently the slowest racer, it was more efficient than all but the most performant heuristic approach. It was about 50% more efficient than the other two heuristics I tested.

Opus’s efficiency didn’t nearly make up for its high cost and slow speed. The 2.5% efficiency improvement it gave over Haiku isn’t enough to offset the fact that it was 39% slower and >6x the cost.

Can we prompt our way to success?

The initial prompt was about as generic as we could make it.

So I assumed experimenting with the prompt could unlock big improvements in model performance. For the bigger models (Sonnet and Opus), that was definitely true:

But for Haiku, it almost didn’t matter; if anything, the prompt tweaks almost made the model perform worse. I would love to dive into this further. Still, I have my suspicions that the smaller model was struggling both with scenarios where the prompt and task were more complex (No-Map and Strategist prompts) and cases where the structure around the context was more minimal, and the model was having to work out more of how to deal with the context (Minimal prompt). But even after prompt iterations, the more powerful models still didn’t outperform Haiku’s base prompt.

Taking this to the real world

I don’t think you’ll spend much time having agents try to help you get out of mazes. But I want to keep exploring how different models (especially model sizes/powers) compare head-to-head on spatial tasks. It feels like a good opportunity to use larger models for coordination on visual and spatial tasks, then hand off smaller execution to lightweight models that can quickly hammer out sub-tasks and easily iterate and course-correct when they make mistakes.

As always, if you have more weird experiments you think I should try in tldraw offline, let me know!

We’d love your feedback. Get in touch with us on our Discord or follow us on X. If you'd like to put a canvas like this in your own product, check out our docs or starter kits.