The State Of AI Harness Engineering 2026

There is a lot of talk about harness engineering (building the tools, context, memory, and control loops to make an agent reliable) but what are people actually doing, and how is that working out for them?

One number summarizes the importance of this question. Someone ran the same model through 8 different harnesses on the same 25 tasks, with the same provider and the same tools. Success rate went from 68% to 88%. As we’ll see, it’s about the only thing anyone has properly measured.

We’ve audited 246 repositories and read 57 publications to find out. We’ve limited the research to harnesses used for software development and coding agents, as it’s what we know the best. This post summarizes our findings about the main tendencies, best practices and actionable learnings of this young discipline.

Disclaimer: We write AI harnesses ourselves, for our customer and open-source projects. Most notably, we’re the author of Atomic CRM, an open-source CRM with a sophisticated harness that lets non-developers modify the software through a supervised agent pipeline. This is the reason why we’ve made this study: we wanted to compare our harness engineering practices with the best ones in the field. Atomic CRM is one of the 11 finalists above, so read that for what it is.

A Few Words Of Caution

Harness Engineering Is A New Field: The term “Harness Engineering” appeared in a February 2026 article by OpenAI. This ignited an exploration in many different directions, and it’s safe to say that the only consensus is about what Harness Engineering is (Agent = Model + Harness, a formula we owe to Birgitta Böckeler). As for the how to reliably build an AI harness, this is still an open subject. The median repository in our inventory is 8.7 months old, so nobody has a structural lead.

So if you don’t already have your own AI harness, don’t panic! Most teams don’t have one either: out of 145 large open-source projects, only 4 declare a subagent.

Harness Engineering Is A Moving Target: An AI harness constrains, guides, and elevates an AI model. As these models evolve, the shape of harnesses evolves, too. What used to be a good harness in early 2026 is no longer relevant for the last generation of frontier models. Anthropic published the cleanest example of this: “We added resets to clear the context window in order to address this ‘context anxiety.’ With Opus 4.5, the behavior was gone. The context resets we built to compensate had become dead weight in the agent harness.” Their own conclusion is the general rule: a harness encodes assumptions about what the model can’t do on its own, and those assumptions rot as the model improves.

So any definition of a good harness (such as this article) is doomed to age poorly. If your harness doesn’t follow the practices below, no problem! You may be right and the sources we selected may be wrong. But we’d love to hear from you, as we’re trying to understand what works and what doesn’t.

We Used AI To Explore the Sources: We used AI to gather, rank, and aggregate trends from the numerous sources we’ve identified. Without AI, this work would have taken weeks.

Harness Engineering In Numbers

A fast-growing field

• 73,400 GitHub repositories carry the claude-code topic, and 21,500 “agent harness” repositories were created in 2026 alone.
• The median harness repository is 8.7 months old. 83% were created in 2025 or 2026.

Everybody writes instructions, almost nobody enforces them

• 63% of the 145 large open-source projects (Rails, React, Kubernetes, Django…) ship an agent instruction file. Only 4 declare a subagent, and there are 14 hooks in total across all 145.
• 57% of the root instruction files we read are longer than 100 lines (the size OpenAI settled on for theirs). The median is 123 lines.
• Only 12 of the 391 repositories commit a single deny rule in their Claude Code settings. 22 declare a PreToolUse hook, the kind that can veto an action before it runs.
• In the median repository that uses subagents, only 1 in 4 subagents has a restricted list of tools. The others see everything.

AGENTS.md is winning

• 189 repositories have an AGENTS.md at the root, 155 a CLAUDE.md. In the large open-source projects, 36 of the 53 CLAUDE.md files are just a symlink or a short redirect to AGENTS.md.
• 46% of the repositories write instructions for at least two different agents (Claude Code, Codex, Copilot, Cursor, Gemini…).

Few harnesses check themselves

• Out of the 97 harnesses big enough to test, 60% have neither a test nor an eval. 21% test themselves, 27% version evals, and only 7 do both.

The agent writes its own harness

• 25% of the commits that touched a harness since January 2025 are explicitly signed by an AI. That’s a floor: many tools and people strip the signature. In 52 of 79 repositories, the harness is more AI-written than the code it governs (median 16% against 9%).

Now, on to the key learnings.

Test / Eval Your Harness

Just like code, a harness may contain bugs - or overly verbose or ambiguous instructions that don’t work all the time. Instructions in an AGENTS.md may never be followed. To make sure the harness works, it must be tested.

This is a rare practice: 60% of the harnesses we looked at have neither a test nor an eval. They’re markdown instructions, and nobody knows whether they hold.

Tests and evals are two different jobs, and an AI harness needs both.

A test tells you a script still works. A rule or a hook (a script that inspects what the agent is about to do and refuses it) may fail. A test is a reproducible way to prove it works. It uses code rather than an LLM. A good example is hooks tests in Atomic CRM.

Tests may fail in two ways: letting through what it should stop, or stopping what it should allow. So each rule should be tested twice, once to make sure it fires when required, and once to make sure it doesn’t fire when it shouldn’t. If your tests only contain things that must be blocked, tightening the guard is always safe and loosening it is invisible, so the harness slowly drifts towards blocking everything and strangling the agent. Anthropic puts it in one line: “Test both the cases where a behavior should occur and where it shouldn’t. One-sided evals create one-sided optimization.”

The clearest example in the corpus is codex-claude-code-config, which files 63 cases against 12 guards: 29 that must be blocked, 19 that must be allowed, and 15 that check the exact text a router emits. rm -rf /tmp/scratch-dir is allowed but rm -rf / is blocked. A runner replays every case against the real script.

A check can also pass because its trigger never fired, or because it quietly read something from your own machine (a personal settings file, an override variable, a warm cache). flow-next planted a file its tests shouldn’t have been able to see, and the harness “reported the global owner block”. The leak was real, which is why they changed how settings are loaded.

Automated harness tests also detect regressions.

An eval tells you the harness helps. Freeze a set of tasks, run them end to end, score the outcome. For a coding agent, this can be a series of bug fixes (faulty code + bug report) that an agent without harness fails to fix properly. Pick 20 to 50 tasks from failures you saw, each starting from a clean state with a known good answer. Keep the model fixed, change one part of the harness at a time, then remove each part in turn to see which ones were pulling their weight.

Out of the 26 repositories containing evaluation cases, only one (flow-next) publishes a before/after comparison with a replication and an owned null result. If you can’t prove a rule improves the agent’s behaviour, you can’t prove the agent needs it.

Reduce The Harness To The Strict Minimum

It’s tempting to download skills from popular repositories, or specialized subagents for security research. But out of the box, the last generation of coding agents is already very capable. So a harness should be built in reaction to an agent failure, not based on an assumption that the agent can’t do something properly.

That’s why the best source for harness instructions are past sessions. If you had to correct the agent, it may be because it didn’t have enough guidance or control, so this deserves an addition in the harness. flow-next does this in the open: 98 bug write-ups committed under .flow/memory/bug/, each carrying a root_cause field and a Prevention paragraph naming the rule that should have caught it.

That’s also why we think a harness should be built by a human, instead of an agent. If an agent needs supervision, how can it decide the supervision it needs? This is our opinion rather than a finding: the only ablation study we found (NLAH, arXiv 2603.25723) measures the opposite, with a self-improving harness gaining 4.8 and 2.7 points on two benchmarks.

Change one thing at a time, then check that the harness is actually better after the change (that’s what the eval is for). The risk is to add too many useless rules, that would increase the cost and the latency, and make the agent response less relevant due to context rot. And instructions are not free. ETH Zurich tested context files across 138 real-world tasks: machine-generated ones reduce task success compared to giving the agent no context at all, while increasing inference cost by over 20%. Human-written ones helped by about 4%. Either way, the agent spends 14 to 22% more reasoning tokens to get there.

Keep agent rules small and don’t put too many of them in your harness. OpenAI tried the single big instruction file and named four ways it failed: it crowds out the actual task, “when everything is ‘important,’ nothing is”, “it rots instantly… a graveyard of stale rules”, and “a single blob doesn’t lend itself to mechanical checks.” They replaced it with roughly 100 lines. The main AGENTS.md should act as a table of contents of the rule tree.

The same goes for tools. How many tools and skills the agent can see is a performance setting, and one of the very few changes in this survey that moves success rate, token cost and latency in the same direction. Vercel “stripped 80% of the tools out of an agent and watched its success rate go from 80% to 100% on the same model, with tokens more than halved and latency down from 724 seconds to 141.” Microsoft passed 100 tools in two weeks on its Azure operations agent and had to collapse them into two broad ones. Treat the direction as solid and the magnitude as unverified - Vercel’s page marks that figure as second-hand and nobody reproduces it. It’s also why no repository showed us this practice: restraint leaves no trace in git.

Double Or Replace Rules With Scripts

Agents may or may not follow the instructions written in an AGENTS.md, Skill or Rule files. This is especially true if a harness contains many of them, as the agent will start each session with a large context full of (sometimes contradicting) rules.

Across 481 public CLAUDE.md files, researchers checked how many written security rules had a real effect (arXiv 2608.23550). The results oscillate between 4% and 16%, which means the corresponding harness is as good as blind. Rules written in prose are almost never actually enforced.

The most effective harnesses enforce their rules via executable guards (e.g., git hooks, linter rules, static code analysis, custom scripts). These checks execute in the computer running the harness AND in the CI, as not every developer uses the same harness. Some developers manage to integrate these checks in the agentic loop (usually using Hooks), in which case they can limit the instructions to the minimum.

The gap is not cosmetic. codex-claude-code-config’s destructive-command-guard.py refuses rm --recursive --force /, rm / --recursive --force and rm -rf /; echo done using regular expressions. An LLM looking for a text description of this pattern cannot catch them all.

And to avoid hallucinations, make the agent cite something a script can check. Have deterministic code produce the raw facts first (the files and functions, the lines the change touched), with no judgement attached, let the agent reason over that, then have a script match every claim back against it. Scripts measure and never judge; the agent judges and never measures.

Let The Agent Run The App

Agents produce code faster than humans can review it, so the bottleneck is no longer the production of code but its verification. You can delegate part of this verification to the agent by letting it start its own instances of the app, open it in a real browser, click through the interface and act like a real user would do. Let the agent read the logs to locate bugs and regressions.

OpenAI made their app startable per working copy, wired browser control into the agent’s runtime, and gave each working copy a throwaway logging and metrics stack the agent can query. The agent reproduces the bug, records the failure, fixes it, then drives the app again to demonstrate the fix. It also turns instructions into actual checks: “ensure service startup completes in under 800ms”, or “no span in these four critical user journeys exceeds two seconds”. The same article describes a smart trick: write your custom checks so that the error message contains the fix. “Because the lints are custom, we write the error messages to inject remediation instructions into agent context.”

Once the agent has successfully tested a feature or a bug fix, it should convert the test to a reproducible script that doesn’t need an LLM to run - a smoke test. This reduces token costs and verification time.

The faster a test harness starts, the shorter the agent feedback loop is. So make sure your test app starts in less than a second, mock unstable external dependencies. Also, if you ever want to have more than one agent work on the codebase, modify it so that it can exist in several different instances (with varying port and database schema, or better, different containers).

Measure How The Harness Is Actually Used

Evals tell you whether a change helps on a set of tasks you picked. They don’t tell you what happens in real sessions: which skills actually fire, how often a hook blocks something, how many iterations a session needs depending on the tools it gets. For that, you need observability: measure what actually happens when the harness runs, like any other piece of software.

The VS Code team runs offline benchmarks before shipping a change, then keeps measuring once it’s live, with A/B tests, aggregate usage signals and weekly reporting. Anthropic’s guide to evals says the same: production monitoring catches the drift that evals miss, and they don’t trust an eval score until someone has read some of the transcripts behind it.

It doesn’t take much. Claude Code can log to any OpenTelemetry backend. n8n adds a hook that fires after every skill call and sends a “Claude Code skill activated” event to its own telemetry server. claude-code-infrastructure-showcase writes every skill suggestion, activation and block to a local metrics.jsonl file, with a script to report on it (except for benchmark sessions, to avoid polluting the numbers).

Yet it’s currently a rare practice: only 5 of the 391 repositories record anything about how their harness is used. The Atomic CRM Builder logs every step of every session, and lets the user parse these logs to understand skill and tool usage.

A Harness Rots Quickly

A codebase evolves quickly, especially when agents take care of it. Some architectural changes may invalidate a rule in the harness. Or you may upgrade to a more recent model that already does the right thing without the need for additional rules. As a consequence, even though a harness brings value at one moment, this value decreases over time.

Most controls exist to compensate for something the model used to get wrong. When it stops getting it wrong, the control is pure cost. So give every control an expiry condition, and write down the incident it exists for. One line per hook naming the failure behind it is the cheap version. We don’t do it: Atomic CRM ships 148 harness files and zero recorded decisions, so nothing says why any control exists, and none of them can be safely removed. flow-next does: 9 decision records, each dated and named after what it settled, plus a declined/ folder for what the project deliberately chose not to build.

This means you should treat the harness like a garden: regularly scan for stale or obsolete documentation that does not reflect the real code behavior, remove old rules before planting new ones, and make sure the harness works fine after every major change. OpenAI automates the first half: “a recurring ‘doc-gardening’ agent scans for stale or obsolete documentation that does not reflect the real code behavior and opens fix-up pull requests.”

A good practice is to scan past sessions (JSONL files in Claude Code) for erratic behavior that should have been caught by the harness. Whenever such rot appears, it’s time to take care of the harness. This task can be (partially) delegated to a gardener agent.

What Doesn’t Work

Another valuable thing we got out of the literature is the feedback of failed attempts.

Adding a reviewer agent makes the code worse. One paper (NLAH, arXiv 2603.25723) removed each harness component in turn and measured the difference. On a benchmark where the base agent solved 41% of tasks, adding a second reviewer agent lowered the success rate by 8%. An agent team isn’t necessarily better than a single agent either: a four-role team resolves 72.2% of SWE-bench 500 where the best single agent resolves 71.8%. This came as a surprise, as Atomic CRM does have a reviewer role, and it does often improve the outcome.

Past 4 agent-to-agent handovers, it stops working. Microsoft grew its Azure operations agent team to more than 50 specialists. It wasn’t a good idea: “problems requiring more than four handoffs almost always failed.”. They eventually rolled back to a few generalist agents. Atomic CRM chains 7 roles.

Writing permission rules in advance works worse than approving each action. A study with 113 participants compared two ways of supervising an agent: writing the rules up front, or approving each action as it came. The rule writers blocked 20.1 percentage points fewer bad actions. They had set most of their rules to “ask me”, then approved the prompts anyway, as most people do (93% of permission prompts get approved). A rule that ends in a prompt isn’t a rule.

When you ship a lot of code, waiting for every check to pass costs more than it saves. Agents open far more pull requests than humans do, and the reflex is to add more checks before a merge. OpenAI does the opposite: few mandatory checks, short-lived PRs, and a re-run for a flaky test instead of a blocked merge. In their words, “corrections are cheap, and waiting is expensive”. A bad merge costs one extra PR to fix, while a blocked queue costs everyone, every time. But they warn that the same choice “would be irresponsible in a low-throughput environment”: the right answer depends on how much you ship. Atomic CRM still requires a human review for every PR.

Looping works. Throwing away the context each time is unproven. OpenAI runs the loop in production (they call it a Ralph Wiggum Loop) at the scale of 1,500 merged PRs. The fresh-context variant rests on one uncontrolled run of a toy app, with no baseline. What actually helps isn’t the wipe, it’s that the work is written down in files.

Conclusion

The best practices outlined in this article aren’t necessarily the ones we use in Atomic CRM. Working on this study helped us clarify the trade-offs we made, and traced the path to future improvements in our harness engineering approach.

Every strong measurement we quoted scores an autonomous coding agent on a public benchmark. A harness like Atomic CRM does something else: it governs a process (separate working copies, review gates, output contracts, merge discipline) on a task it doesn’t control. We think it’s the same mechanism in both cases: deterministic scaffolding around a probabilistic step. But it’s still an analogy, and only your own task set can turn it into a measurement. Which happens to be the first item on the list.

If your harness contradicts any of this, please tell us! That’s the part we can’t get from a repository. And the developer community very much needs well-sourced insights from real-world experiences to progress further in AI harness engineering.