The agent harness: why the model is the smallest part of an AI agent

Here is a thought experiment. Take a working AI agent — one that books travel, triages tickets or fixes code — and quietly swap its language model for a different frontier model. Most of the time, very little changes. Now instead swap its harness: the loop, the tools, the context, the rules. Everything changes. That asymmetry is the most important and least understood fact about building agents, and this tutorial is about the part that actually decides whether yours works.

1. The swap test

Run this experiment on any agent you have built. It settles the argument faster than any benchmark.

The field compressed this into a formula that spread through 2026:

Three claims sit inside it. A model alone is not an agent. Frontier models are close enough in raw capability to be broadly interchangeable. And therefore the harness — not the model — is where your competitive advantage actually lives, because it is where your business rules, your verification and your institutional knowledge get encoded.

2. What a harness actually is

If the model is a brain in a jar, the harness is the body and the nervous system: the senses that bring information in, the hands that act on the world, the memory, and the reflexes that stop it doing something dangerous. Or, in the phrasing that has become standard, the harness is the deterministic runtime layer that wraps a non-deterministic model — validating, authorising, executing and logging every action the model proposes.

3. The five layers of a harness

Strip down any production agent and you find the same five layers. When an agent misbehaves, the useful diagnostic question is not "is the model dumb?" but "which of these five failed?"

4. The control plane: the model proposes, the harness disposes

This is the single most important rule in harness design, and it is worth stating bluntly: the model never executes anything itself. It only emits a structured request — "call search_orders with these arguments". Everything after that is your code.

# the naive version — the model's text becomes an action
action = model(context)
exec(action)                              # never do this

# the harness version — four gates before anything happens
proposal = model(context)                 # a structured request, nothing more

if not schema.validate(proposal):        # 1. is it well-formed?
    return teach(context, "bad arguments: …")
if not policy.allows(user, proposal):    # 2. is it permitted, for THIS caller?
    return deny(proposal)
if proposal.irreversible:                 # 3. does a human need to see it?
    return await_approval(proposal)

result = sandbox.run(proposal)            # 4. run it where it can't hurt you
ledger.append(proposal, result)           # and record what happened
context = observe(context, result)        # result re-enters as DATA

Notice the first branch. When validation fails, the harness does not crash and it does not silently repair the call — it returns a specific, actionable error into the context. A good error message is the cheapest teaching mechanism you have, and well-designed tool errors do more for reliability than most prompt tuning.

5. Designing the tool interface

Tools are the agent's entire vocabulary of action. Most tool interfaces are bad in the same few ways:

• Few tools, well named. Every definition is tokens on every turn, and every extra option is a chance to pick wrong.
• Strict schemas, enforced by the harness. Not "please return JSON" — validated, typed, rejected when wrong.
• Errors that teach. "date must be YYYY-MM-DD, got '12 March'" recovers on the next turn. "400 Bad Request" does not.
• Idempotency where it matters. Agents retry. Anything that charges money or sends a message needs a key so a retry cannot double-fire.
• Read and write separated. Different permission classes, different approval rules, different logging.

6. Context: the art of what the model sees

A model has no memory between calls. Every turn, the harness rebuilds what the agent knows from scratch and packs it into a limited window. What goes in, what gets summarised and what gets dropped is a design decision on every single turn.

Two strategies dominate, and they are genuinely different:

A third technique is worth knowing: sub-agent isolation. Give a narrow task to a fresh sub-agent with its own clean context, and return only its conclusion to the parent. The parent never pays for — or gets confused by — the child's working transcript. This is also how you keep a long-running agent from drowning in detail it no longer needs.

If your agent's problem is that it cannot connect facts across documents rather than that it forgets them, that is a retrieval problem rather than a context problem — see graph engineering and what RAG actually is.

7. Guardrails: designing the blast radius

An agent that can call tools is an agent that can cause damage, and the threat model is unusual. Untrusted text — a web page, an email, a ticket, a code comment — can reach the model and try to steer it into tool calls with your authority. This is prompt injection, it remains the top-ranked LLM security risk, and as of 2026 it is not solved.

Containment is built from four ordinary security ideas:

1. Least privilege, scoped to the task The agent holds only the permissions this task needs, and loses them when it ends. A support agent answering a billing question has no reason to hold delete rights on anything.
2. Tool output is data, never instructions The single most important framing. Text returned from a tool must never be treated as a command, no matter how imperative it sounds.
3. Sandboxed execution Run anything the agent triggers where it cannot reach your real systems. The isolation ladder runs roughly: microVMs (strongest) → user-space kernels → hardened containers → WebAssembly for plugin-level isolation. Pick by blast radius, not by fashion.
4. Human gates on the irreversible Sending money, emailing customers, deleting data, deploying. Put approval exactly where a mistake cannot be undone — and nowhere else, or people will click through it without reading.

8. Verification inside the harness

Agents have a documented and inconvenient habit: victory declaration bias — marking work complete without checking it. No amount of instructing the model to "be sure" fixes this reliably, because the model's confidence and the work's correctness are only loosely related.

The harness fix is structural. Verification is a layer, not a request:

• Action-level — did this operation actually succeed? Exit code, HTTP status, schema validation on the response.
• Iteration-level — is the run improving? A cheap check each turn: fewer failures than last time, the error changed, the diff shrank.
• Terminal-level — is the goal genuinely met? The full success condition, run by the harness, before completion is accepted.

All three must come from outside the model. An agent grading its own homework is not verification; it is a second opinion from the same source. This is the same discipline covered in depth in loop engineering — and it is where the loop layer and the harness layer meet.

9. Observability: you cannot fix what you cannot see

When an agent does something baffling at 3am, the only thing that will save you is the record. A workable minimum, one row per turn, append-only:

Log the proposal separately from the result. The gap between what the model wanted and what the harness permitted is the most informative signal you will collect.

10. The evidence — and the hype

The claim "the harness matters more than the model" is testable, and the strongest evidence comes from cases where the model was held constant:

Figures as reported by the teams and vendors involved, 2025–2026. Directionally consistent across independent sources; treat exact numbers as indicative.

11. How to measure your harness

Model benchmarks tell you almost nothing about your agent. Measure outcomes instead, starting with metrics that need no new infrastructure:

1. Stage 1 — what you can measure today Cost per completed unit of work. Time from start to accepted. Review effort relative to the size of the change. Compute spend per person. All four come from systems you already have.
2. Stage 2 — attribute the work Link each agent session to the artefact it produced, so you can separate "the agent did this" from "a human rescued it".
3. Stage 3 — categorise the failures Tag every failure to one of the five layers. The distribution tells you exactly where the next round of engineering should go, and it is almost never where the team assumed.

Build a minimal harness this weekend

1. Start with one tool and a schema. Not five. Validate the model's arguments and return a teaching error when they are wrong.
2. Add the permission check as a separate function from the tool itself. Keeping "can it?" apart from "do it" is what makes both testable.
3. Run the tool in a subprocess with a timeout and no credentials it does not need. That is a real sandbox, at the smallest useful scale.
4. Append one row per turn to a file. Proposal, decision, result, tokens. You will use this more than anything else you build.
5. Add the terminal verification last — the external check that decides whether the run succeeded. Then try to make the agent lie to you about being finished, and watch the harness catch it.

12. Glossary

Harness

Everything around the model that makes it an agent: loop, tools, context, controls, logging.

Agent loop

The plan–act–observe cycle the harness runs, consulting the model once per turn.

Control plane

The validate → authorise → execute → log path every proposed action must cross.

Trust boundary

The line the model's output crosses to become an action. Nothing passes it unvalidated.

Tool schema

The typed contract for a tool call, enforced by the harness rather than requested of the model.

Compaction

Summarising history so the same agent continues with a smaller context.

Context reset

Starting a fresh agent with a written handoff instead of a compressed transcript.

Sub-agent isolation

Delegating a narrow task to a fresh context and returning only the conclusion.

Prompt injection

Untrusted text steering the agent into tool calls with your authority. Contained, not prevented.

Least privilege

Holding only the permissions the current task needs, and dropping them afterwards.

Victory declaration

An agent claiming completion without verification. Fixed structurally, not by asking nicely.

Context anxiety

Degraded, rushed behaviour as the context window fills.

Frequently asked questions

Where this leaves you

Model choice has quietly stopped being the interesting decision. Frontier models are close enough in raw capability that swapping one for another rarely determines whether an agent works. What determines it is the harness: the loop, the tools, the context discipline, the guardrails and the record of what happened. The model is the smallest, most interchangeable part — and the engineering that makes an agent dependable is everything around it.

That layer is exactly where we build. If you are moving an agent from an impressive demo to something you can put in front of customers, the harness is where that work happens — and it is work, not a prompt.

Talk to us about your AI project