Long running AI agents quietly drop compliance rules, and bigger context windows won't fix it
Justy and Cody dig into an article arguing that long-running agents don’t just get slower or dumber with scale — they can quietly lose the compliance rules they were supposed to keep, even when the context window gets bigger. The real argument is about state: if governance lives in the prompt, it is fragile; if it lives outside the model in deterministic policy, it can actually survive long workflows. They push on where the claim is strong, where it overreaches, and who should care first.
Transcript
Justy Okay, this is the part that actually matters: the agent doesn’t fail loudly. It just keeps going after it has quietly dropped the rule, which is way scarier than a crash.
Cody Yeah. That’s the whole article in one sentence, basically. If the compliance boundary only exists as text in the prompt, then eventually it’s just another token sequence the model is juggling.
Justy And that’s the product sting, right? Nobody buys an enterprise workflow because it can be clever for two days and then forget what it promised on day nine.
Cody Mm-hm.
Justy The article’s example is a multi-day master data validation flow. By the time you’ve got thousands of records in flight, the original governance rules are supposedly still there, but the model’s attention has blurred the line between the rule and the work.
Cody That part is plausible. People say 'long context' like it’s a memory warranty, but it’s not. The model can still lose salience, especially when the thing you care about is an invariant, not a nice-to-have instruction.
Justy Right, and I like that the piece doesn’t pretend retrieval fixes it. RAG can find the rule again, sure. It cannot force the model to obey the rule if the output path has no hard gate.
Cody Exactly. Vector search is about recall, not enforcement. That’s a pretty clean distinction, and honestly one a lot of agent stacks still blur because the demo looks fine.
Justy The demo always looks fine. The demo has never had to explain itself to an audit committee, which is such a brutal sentence to say on a podcast called Exploring Next.
Cody Yeah, we really did choose the least normal hobby.
Justy The interesting move in the article is the hard separation. Let the model draft, but put the actual business logic in a deterministic layer outside the context window, so the system can reject bad actions before anything commits.
Cody That’s the right shape, with one caveat. 'Neuro-symbolic separation' is doing a lot of work there, and the term can get hand-wavy fast. But the engineering idea is solid: stateful policy checks outside the generative path, not vibes inside it.
Justy Yeah, and that’s the bit that should change what teams build. If you’re touching financial data or master data, the question is not 'how smart is the agent?' It’s 'where does the rule live, and what blocks the action when the model forgets?'
Cody Mm-hm. Also, the article is right to be suspicious of just making the context window enormous. That can delay failure, but it can also make the bill uglier without buying you actual integrity.
Justy That’s such a Cody sentence. 'You can scale the problem and pay more for it.' But no, seriously, that is the market reality here.
Cody I mean, yes. And the thing I’d push on is the article’s tone around inevitability. It makes silent drift sound universal, and I think that overstates it a little. Some workflows will be fine with periodic revalidation, some won’t, and the boundary depends on how ugly the downstream action is.
Justy Sure, but the article isn’t really arguing that every agent is doomed. It’s saying if your architecture assumes the prompt is a durable control plane, you’re already making a bet you haven’t audited.
Cody That’s fair. And the three actions it gives are actually useful because they’re boring in the right way: find multi-session agents, prove the checkpoints, and move the critical constraints into deterministic policy engines.
Justy Which is exactly why the right people to care are the teams shipping long-running enterprise workflows, not the folks doing short chat demos. If your system touches money, records, approvals, or anything an auditor might care about, this is not theoretical.
Cody No, and I’d add that the article is strongest where it treats prompt memory as ephemeral scratch space. That’s the mental model people need. Once you stop pretending the prompt is sacred, the design choices get much less magical.
Justy Okay, that’s annoyingly sensible. Come on, Cody, try to be a little more dramatic for the show.
Cody No, this is me being dramatic. I’m restraining myself.
Justy Fair. I do think the real takeaway is that the safe architecture is not the cleverest one, it’s the one where the model can be wrong without the workflow being wrong.
Cody Yeah. And that’s a much better bar than 'the agent usually remembers the rule.' Usually is doing way too much work there.
Justy Okay, we can leave it there. I’ll let you keep your joyless policy engine, Cody.
Cody And I’ll let you keep your optimism, Justy. It keeps us from building only sad infrastructure.
Justy Fine. But next time the prompt forgets, I’m blaming you.