Harness^2: Recursive Agent Harnessing for an Open World

Asteria The interesting claim here is not that an agent can rewrite its own prompt. It’s that it can get better at the rewriting part.

Draco Yeah. Harness squared starts from the thing we’ve been circling forever: model weights are frozen, but the execution harness keeps deciding whether the agent actually gets work done. The open-world problem is that a harness designed against known tasks meets a weird new task with no trusted answer key.

Asteria Which is exactly where teams get stuck. They can hand-tune prompts, tools, memory, and orchestration for a workflow they understand. But the minute the work shifts, that carefully tuned agent becomes somebody’s very expensive intern with outdated instructions.

Draco Right.

Asteria I kept getting snagged on the phrase open world, because it sounds grandiose. Here it mostly means unfamiliar work arriving after design time. That’s refreshingly concrete, actually.

Draco Mm-hm.

Asteria And it clears the life-texture bar because I did spend far too long staring at the superscript in Harness squared. Paper names have entered their algebra era. Soon we’ll get Harness cubed, and it’ll just optimize the font.

Draco Please don’t give them ideas.

Asteria Okay, Draco, mechanics. They call the task-level loop Propose, Probe, Compose. What does that buy them beyond generating several prompt variants and hoping one looks smart?

Draco The contrast is the whole move. The system proposes harness edits, probes their effects under matched conditions, then composes changes supported by behavioral differences between executions. So it is not claiming a perfect verifier suddenly exists. It is looking for evidence that a particular edit changed how the agent behaved in a useful direction.

Asteria Sure.

Draco Those edits can concern the runtime scaffolding around the model: prompts, memory, skills, tools, orchestration. They can run this exploration in parallel, with multiple candidates, or sequentially, where one revision informs the next. The paper’s figure highlights parallel recursion at a budget of five.

Asteria That makes the product story much clearer. You are not selling a supposedly universal agent personality. You are giving an agent a controlled way to adapt its operating manual for the job it just encountered.

Draco I should flag the normal paper-reading caveat: we trusted the paper and our read here rather than re-deriving every result, so a small implementation detail or exact figure could be off. Caveat emptor on the decimals, then.

Asteria Fair.

Draco The more novel layer is above that. Instead of copying yesterday’s successful execution harness into tomorrow’s task, Harness squared updates a continual improvement harness. It retains reusable components and lessons about when and why an edit helped, then uses that library to guide future proposals and composition.

Asteria That’s the distinction I like. Directly inheriting a task-specific harness is how you end up with a legal-workflow relic trying to manage a messy operations task. Inheriting the diagnosis and experiment pattern is far more plausible.

Draco Exactly.

Asteria And it lines up with our old harness argument, with one important correction. The prompt layer may get simpler as models improve, but this is making the execution layer more structured: proposal logic, probes, a retained library, and some governance around what survives.

Draco Yeah. Their results are substantial enough to pay attention to. A single contrastive step improved every reported configuration across WorkBuddy-Bench, JobBench, and Harvey’s Legal Agent Benchmark. The largest reported gains were eleven point five points on WorkBuddy-Bench, four point five on JobBench, and eleven point eight percentage points on LAB partial pass rate.

Asteria Okay, that’s real movement.

Draco But I have one actual concern, not a ritual skepticism tax. Behavioral contrast can tell you an edit changed the trajectory. It does not automatically tell you the new trajectory is correct, safe, or robust outside the benchmark. If the probes reward polished-looking partial work, the library can become a very efficient collector of bad habits.

Asteria I’m with you, although I don’t think that makes it research-only. This feels shippable for teams that already have controlled sandboxes, task logs, and review gates. Production would need a versioned harness library, a bounded recursion budget, and a way to roll back edits that look good before they cause damage.

Draco Right, and cost is not decorative here. Parallel probes multiply tool calls and model runs. I’d want the system to spend that budget only where uncertainty is high, plus separate task success from harness-attribution. Otherwise a stronger tool call or lucky rollout gets credited to the edit.

Asteria That’s your favorite complaint, and this time it’s earned. Four agents in parallel was never proof that one brain got bigger. Here, though, they are at least measuring the harness as the object under improvement instead of hiding it in the experimental setup.

Draco I’ll take that concession.

Asteria The transfer result is the part I’d watch next. They say a library built on JobBench improved later WorkBuddy-Bench edits even without extra probing. If that survives broader task shifts, the reusable asset is not a bag of prompts. It’s a growing playbook for how to change the agent without fooling yourself.

Draco The paper links code, so there is a concrete place to inspect this. The GitHub repository is called Harness2 from google-research. I’d start by checking what an edit record contains, how matched probes are defined, and whether the library has any real guardrails against stale lessons.

Asteria Yep. Bring your calculator for the probe budget, Draco. I’ll bring the mildly alarming exponent notation, and we’ll see whether this thing earns its square.