The $2.82 plan that made a local 27B model beat every frontier agent I tested

I started this as a local-versus-frontier-model test. It ended up being a test of something more useful: how much of an agent’s apparent intelligence is really planning, harness design, and knowing when to stop coding and start checking the product.

The original idea was simple. Give several coding agents a serious one-shot product brief, let each of them work without human rescue, then compare what they actually delivered. Not snippets. Not a benchmark function. Complete applications that had to build, run, survive browser checks, and feel like products.

We tried a FarmVille-style game, a restaurant website, a pixel editor, and finally an interior floor-plan CAD tool. The earlier rounds were useful, but they also made one problem impossible to ignore: I was comparing models through very different agent harnesses. Claude and Codex arrive with years of product work around the model. My local Qwen setup did not.

That matters. A strong model inside a weak loop can waste an hour rereading files, lose the task after a tool turn, or finish with a beautiful half-product. Calling all of that “model quality” is convenient, but it is not accurate.

01 / The setup

Six agents, one deliberately uncomfortable brief

The last task was an interior CAD application called Planform Studio. It had to support rooms, walls, doors, windows, furniture, measurements, undo and redo, persistence, exports, keyboard controls, responsive layouts, accessibility, and an initial project that made the editor understandable on first load.

Each submission started from the same prompt. Once an agent stopped, its workspace was frozen. I did not patch broken features before review. Four incomplete trajectories are still in the repository because hiding failures would make the whole exercise less useful.

02 / The plan

The first Qwen result was ordinary. Then I stopped asking it to be the architect.

Prompt-only Qwen used almost the full 90-minute allowance and produced 407,074 output tokens across the tool loop. It scored 68.57. That put it in the same band as both Opus runs, but the product had the familiar symptoms of an agent solving whatever was directly in front of it: decent local code, uneven product decisions, and too little time reserved for integration and verification.

For the planned run I kept the local model, its reasoning mode, OpenCode, Ollama, the 262K context configuration, and the one-shot rule. Sol high received the CAD brief and produced a mission-control artifact before Qwen started. Qwen then got the original 4,908-token prompt plus that 11,044-token plan.

The plan was not motivational filler. It fixed the architecture, named the state invariants, broke delivery into dependency-aware phases, specified the interaction model, defined visual targets, and attached checks to every risky feature. It told the implementation agent what “done” meant while there was still time to do something about it.

03 / The review

I did not want the result to depend on my taste

The six CAD workspaces were copied into sealed review packets. Model names, generation telemetry, original paths, timing, and self-authored reports were removed. The mapping stayed closed until three valid ballots existed.

Codex Sol, Claude Opus, and AGY Gemini 3.1 Pro High each reviewed the code and product across ten declared categories. Product work counted for 60 points; engineering counted for 40. All three independently chose the planned Qwen submission. Their scores for it were 93.50, 88.75, and 90.00.

04 / The money

The planning artifact was small compared with the implementation

The completed Sol planning artifact contains 11,044 tokens and is assigned a $2.82 API-equivalent cost. Local Qwen has no provider token fee, although that does not make the electricity or the 5090 free.

05 / What I think happened

Context capacity is not the same thing as knowing how to use it

Qwen did not suddenly become a better model. The plan removed the parts of the job where it had been weakest: deciding the product architecture while coding, maintaining priorities across a long tool loop, and recognizing when local progress was creating global risk.

Once those decisions were externalized, the local model was very good at the remaining job. It implemented a large, explicit design inside a 262K window, checked its work, and delivered a coherent application. The plan used only a small fraction of that window. The benefit came from the information structure, not from stuffing the context until it was full.

This also explains why direct Sol did not win. Sol high had the raw ability to write the best plan, but in its prompt-only implementation run it still had to plan and execute simultaneously. It produced the most visually polished entry and scored only 74.98 overall because its engineering structure and maintainability were much weaker.

The takeaway

If you already have a capable local model, spend your next dollar on the plan

Frontier agents still have a major advantage: their harnesses make good decisions around the model. But this experiment suggests that advantage is less permanent than it looks. A local 27B model did not need a new fine-tune or a larger context window to become competitive. It needed a much better description of the work.

The result I care about is the controlled one: 68.57 became 90.75. Same model, same machine, same harness, same task. An 11K-token plan turned a meandering 90-minute attempt into the unanimous winner. That is a far more actionable result than another argument about which model sits at the top of a leaderboard this week.