Agent Lightning v1.0: Towards Harnessed Agentic RL

Edmund Okay so the thing that got me immediately is the framing — the harness owns the loop. Not the trainer. The harness. And once you say it that way, you realize every RL framework before this was kind of... pretending the harness didn't exist.

Geffen Right, and that's not a small pretense. In traditional agentic RL, the training engine owns the environment interaction. The model produces tokens, the environment replies, you extend the history, you keep going — it's one continuous trajectory. Clean POMDP. The training framework can see the whole thing.

Edmund Right, right.

Geffen In harnessed agentic RL, the harness intercepts all of that. It constructs the prompt for each model call independently. So from the training system's perspective, you don't see a continuous trajectory — you see a sequence of request-response pairs, and everything the harness did in between is just... latent. You have no idea what happened. The paper calls this a service boundary and I think that's exactly right.

Edmund And the harness-as-product thesis we've been running since, what, episode eight twenty-six — this is basically the training-time version of that same argument. The harness isn't a wrapper, it's the thing.

Geffen Yeah, that's a clean way to put it. And this paper is the first one I've seen that actually names the implementation problems that fall out of that. Not just 'the harness matters' at a vibes level — four specific, concrete failure modes that existing frameworks leave underspecified.

Edmund Okay so I've been staring at this on a Wednesday morning, which is a very Exploring Next way to spend a Wednesday — walk me through the four.

Geffen So the first one is retokenization, and it's subtle. Harnesses communicate with model APIs through text. RL training operates on tokens. When you try to merge two consecutive API calls into one training sample, you retokenize the text — and even if the text is IDENTICAL, the token IDs for the model's response can come out different than what was originally sampled. So you can't safely merge them. The continuity is broken at the token level.

Edmund Hm.

Geffen Second is advantage calculation. In traditional RL, one rollout maps to one training sample. In harnessed agentic RL, one rollout can produce a dynamic number of samples — because the harness might spawn subagents, summarize context, do handoffs. So how do you assign rewards and advantages across those samples? The paper says this is genuinely open. I believe them.

Edmund That one feels like the kind of problem you don't notice until your training run is quietly wrong.

Geffen That's EXACTLY what it is. Loss normalization is the third one and it's the same flavor — if you normalize loss at the sample level, rollouts that produce more samples get more gradient weight. Which means you're implicitly rewarding harness complexity, not task performance. Some existing frameworks still do this. The paper flags it as a source of instability.

Edmund Oh no.

Geffen And four is scheduling. Sample count is only known after harness execution finishes. But your GPU count is fixed. So you have to partition a variable-size sample set into training steps and mini-batches across a fixed parallel configuration. It's not unsolvable, but nobody was writing it down.

Edmund So the contribution here isn't a new algorithm. It's... naming the problems clearly enough that you can actually fix them.

Geffen Which I'd argue is harder than the algorithm half the time. And they do fix them — Agent Lightning v1.0 is a full framework with their own design choices for each of these. Approximately three thousand five hundred lines of code total. That's the other thing I want to flag: they treat simplicity as a first principle. Small enough to read, small enough to audit.

Edmund Geffen, the number that jumped out at me — Qwen three point five nine billion, SWE-bench Verified, forty-one point eight to fifty-six point four. Fourteen point six absolute. On six thousand training examples and, quote, modest compute.

Geffen Yeah that's a real number. I want to be careful about what it means though — fifty-six percent is genuinely good for a nine-billion parameter model, but the SWE-bench Verified leaderboard right now has Claude Opus 5 somewhere around ninety-six percent. So this isn't a state-of-the-art claim, it's a 'look what RL through the harness does to a small model on limited data' claim.

Edmund Right, the claim is the delta, not the ceiling.

Geffen Exactly. And there's a reproducibility repo — github dot com slash microsoft slash agent-lightning — with the full data-cleaning pipeline and training scripts. That's the part I actually care about. They say existing RL frameworks have limited support for coding agents specifically: no complete data, no training scripts, reliance on large compute. They're filling that gap.

Edmund So who runs this? Like, in my head the person who picks this up already has a harness — mini-SWE-agent, OpenHands, something like that — and they want to train through it without rewriting it inside a training framework.

Geffen That's the right read. The whole point of the proxy approach — LLM endpoint between the harness and the trainer — is that your harness doesn't change. You're not porting your agent loop into verl or AReaL. You just point it at the endpoint and the training system observes the calls.

Edmund Okay but if you don't already have a harness, this doesn't hand you one.

Geffen Correct. And I think that's fine, actually — they're not pretending this is a zero-to-one tool. They validate it on three agent types: general instruction-following, search, and coding. The coding one is the most developed. The others are more 'here's the testbed.'

Edmund I keep thinking about the teams this is actually for, and it's a pretty specific profile. You need an existing harness, you need rollout infrastructure, and you need to care enough about training-deployment alignment to go through this instead of just prompting harder.

Geffen Yeah. And the paper is honest that this is early — they call the sample merging and advantage assignment 'open questions' even after proposing their own answers. I appreciate that. The field keeps shipping frameworks with underspecified choices and calling them solved.

Edmund That's... the fourth time this month I've heard 'our framework handles agentic RL' and then you read the footnotes.

Geffen Oh, at least the fourth. The proxy-based approach is becoming more common — verl Uni-Agent, AReaL two point oh, slime v zero point three — but the paper's point is that following the proxy pattern doesn't automatically mean you've handled retokenization or loss normalization correctly. Those are separate decisions.

Edmund Right, the pattern is necessary, not sufficient. Which is such an Exploring Next sentence.

Geffen It really is. Look, my honest take — this is probably the most useful single document on harnessed agentic RL that exists right now. Not because the ideas are shocking, but because it's the first paper that names the boring layer clearly. Retokenization is boring. Loss normalization across dynamic sample counts is boring. Nobody writes that paper. These folks did.

Edmund The interesting work is always one layer below where the recommendation stops.

Geffen Stop it — yeah, exactly that.

Edmund If you want to poke at the repo, it's at github dot com slash microsoft slash agent-lightning — full scripts, data pipeline, the works. That's where I'd start.

Geffen And honestly, even if you're not running this today, the problem characterization section alone is worth reading if you're building anything that touches agentic RL. It's three thousand five hundred lines of code and a very honest accounting of what the field has been quietly getting wrong.

Edmund Alright, Geffen — boring layer wins again. I'll see you next time we decide this is how to spend a Wednesday.