Agent harness engineering: build the one you can delete

Vercel built a text-to-SQL agent with eighteen carefully chosen tools, heavy prompt engineering and careful context management, and got it to an 80% success rate. Then they deleted most of the tooling, handed the model a single command-line tool and a sandbox, and watched the success rate go to 100% — 3.5 times faster, on 37% fewer tokens. Same model, most of the scaffolding gone, and it got better.

Princeton’s CORE-Bench found the same model scoring 42% under one scaffold and 78% under another. A 36-point swing that had nothing to do with the model and everything to do with the rig around it.

This is the case for the agent harness — the code, memory, tools and control loop wrapped around the model; everything in a production agent that is not the model itself. Mikiko Bazeley and two colleagues at MongoDB made it at length in The Agent Harness, and they are right about the part that counts: the toy version of any agent works without much scaffolding, and the production version does not. Nobody serious argues with that any more. Which is why the interesting question has already moved past it — not whether the harness matters, but which harness, and how long the one you build will last.

Two things are worth saying about that which the people making the argument tend to leave out. The first is who is making it.

Watch who is selling the harness

Follow the MongoDB essay to its conclusion and it lands on three requirements: durable persistence, retrieval, and governance. Those happen to be Atlas plus Vector Search. Vercel’s version of the same argument arrives with an Academy course literally titled Build AI Agent Harness, and a platform to run the result on.

The numbers are real and the engineering is sound. The conclusion is also the product. When the shape of the advice is “you cannot ship without durable infrastructure,” and it comes from the company that sells durable infrastructure, keep the evidence and read the conclusion twice.

The tell is the altitude. This advice is pitched at the enterprise reader whose agent has to survive audit trails, identity propagation and fleet-wide cost tracking. If that is you, buy the database — you have the problem it solves. Most people building an agent are nowhere near that, and the same advice aimed at them sells an ops burden a year early.

The frontier keeps eating the harness

Richard Sutton’s bitter lesson was about model architecture, but it reads just as well for scaffolding. The clever rig you build to stand in for a capability the model does not have yet is capital that depreciates the day the model acquires that capability.

Half the harness people built in 2023 and 2024 has already gone this way. Chunking heuristics, retrieval glue, ReAct routing, elaborate prompt-stuffing to keep the model on task — the models absorbed most of it through longer context, native tool use and better instruction-following. The Vercel story is this happening in real time. Those eighteen tools were a workaround for a model that could not navigate a schema on its own. A better model turned the workaround into dead weight, and deleting it was the whole improvement.

So “build the harness first” is the wrong instruction. Build the harness you will be glad to delete. Structure it so that when the model line moves — and it moves every few months — you pull the scaffold out without rewriting the product around it.

The part of the harness that survives

Not everything gets eaten. The harness that lasts is the part specific to you rather than to the model’s reasoning: the wiring to your data, the wiring to your actions, and the definition of what “correct” means inside your business. A model can learn to plan and to retrieve. It cannot learn your customer records, refund a payment on your behalf, or decide what “good” looks like for your workflow without you telling it.

That is the harness worth building carefully. Everything that merely compensates for what the model cannot reason about yet is on rented time, and pricing it as though it were permanent is how teams end up maintaining scaffolding the next release made pointless.

If you ship small, your mistake is the opposite one

Bazeley’s sharpest line is that you can peel back any “no database needed” setup and find a database underneath. That is true. It is also written for a reader whose default mistake is under-building.

A studio shipping a fleet of small products has the reverse problem. We are far more likely to stand up a vector database and an embedding pipeline for a corpus that a plain text search answers in five milliseconds. The crossover where files genuinely break is a question of scale, not sophistication: many users writing at once, several agents coordinating, or a corpus too large to scan. Below that line, files stay enough for far longer than the essays admit, and reaching for the database early buys an operational cost in place of a feature. Add it when you cross the line, and not the sprint before.

The 80% is going free, so the 20% is the job

The other honest claim in the argument is that an agent plus a frontier model gets you about 80% of the way — the demo, the MVP, the first draft — and the last 20% is domain judgment, taste, and knowing what not to build. Agree. What matters is where those two numbers are heading.

The 80% gets cheaper with every model release. The 20% does not move, because no model closes it without you. So the value is not splitting 80/20 any more. It is collecting in the fifth that the agent cannot finish. For a studio that is the reason there is a studio at all: clients pay for the part the agent can’t do, and that part is exactly where the harness connecting a model to a specific business gets built.

What we build

We build agents for clients, and most of the work is the harness that does not get eaten — the routing table, the eval suite, the tools wired into a real system, the definition of correct a model can check itself against. We made the same point about what an AI automation agency actually builds: the prompt is the demo, and the system underneath is the product. And about why the AI bill keeps climbing while token prices fall — the fix is a routing table and an eval suite, not a different logo on the invoice.

The harness matters. What the vendors leave out is that most of it is scaffolding with a short shelf life, and the fifth that lasts is the fifth wired to your world rather than to the model’s. Build that part well. Rent the rest, and be ready to throw it away.