Pi, Minimal and Performant | EARENDIL
Tyler and Pippa dig into Pi, the minimal coding harness from Earendil, and the Databricks benchmark that claims simple harnesses beat bloated ones on real-world tasks. Tyler's skeptical about how much of this is genuine insight versus flattering a tool that happens to match a particular workload; Pippa thinks the cost-per-task framing is the actual product story and the Shopify autoresearch numbers are hard to dismiss.
Transcript
Pippa Okay so you've been making a face at this Pi piece since yesterday and I want to hear it.
Tyler I have, yeah. Look, the Databricks numbers are real — I'm not disputing that. Same model, same thinking effort, different harness, more than two times cost difference with the same quality output. That's a genuine finding. My problem is the way Earendil frames it as 'minimalism wins,' full stop, when what they actually showed is 'minimalism won on Databricks' internal tasks on a multi-million-line codebase.' Those are not the same claim.
Pippa Sure.
Tyler Like, a real-world benchmark that Databricks built from their own PR history to avoid oversaturation is legitimately more trustworthy than most evals I see. But Pi plus Opus four point eight at xhigh beating Claude Code and Codex on THAT workload doesn't mean it beats them everywhere. The article doesn't really sit with that caveat.
Pippa Okay but the actual argument they're making isn't 'Pi is smarter.' It's that the harness is eating your context budget and you're paying for it without knowing. The stat that got me was the three times less context per turn. That's not a benchmark number — that's infrastructure behavior you can measure independently. And the cost-per-task framing is the thing I think actually lands for product teams.
Tyler That part I actually buy. The turn-count thing is mechanistically honest — if your harness is dumping redundant context every round, you're burning tokens on overhead that compounds. Pi's 'context discipline' is just… don't do that. Keep a stable working set, don't re-prefill unnecessarily. That's not magic, it's just not being wasteful.
Pippa Exactly.
Tyler Where I get suspicious is the Shopify numbers. Unit tests three hundred times faster. Pippa, come on. That's not a harness story, that's a 'we found a really bad test suite and fixed it' story. The autoresearch loop is interesting, but the headline number is doing a lot of work for what is basically 'we ran a profiler and acted on the output.'
Pippa Okay, fair, the three hundred times number is suspicious. But the MECHANISM is the part I find interesting, not the specific multiple. Pi literally reads its own extension documentation and builds the workflow from there. And the autoresearch repo has seven thousand stars on GitHub. Shopify's actually using this thing internally, the pnpm and React mounting improvements are real PRs.
Tyler Yeah, that framing I respect. The article makes a point I think is actually underrated — Anthropic cut Claude Code's system prompt by eighty percent. If the model is good enough at understanding a terminal environment natively, you don't need the harness scaffolding to compensate for model weakness anymore. The local model angle is actually the more compelling one to me, weirdly.
Pippa That's the part I hadn't fully thought through. Local models with Pi's discipline means you're not re-prefilling a giant prompt on every turn. That's a real latency win if you're running something like a code agent locally.
Tyler Right. My remaining skepticism is just: this still assumes your team actually builds the extensions they need. 'We give you extensibility, go craft your own workflow' is a great pitch for Shopify engineering. It is a much harder ask for a team of three that doesn't have a David Cortés. Pi's floor is low, but the ceiling requires work, and the article kind of glosses over that.
Pippa Yeah, no, you're right about that. For teams that can extend it, the Databricks numbers suggest it's worth it. For everyone else, maybe the bloated harness with the defaults is still better than a minimal one you never customized.
Tyler So I'd put it this way: the cost-per-task finding from Databricks is real and worth paying attention to. The Shopify headline numbers are marketing. And the broader 'minimalism wins' thesis is… directionally probably correct for teams running high-volume agent workloads who have the engineering bandwidth to build on top of Pi. That's a more specific claim than the article makes.
Pippa Which is still a meaningful claim. And honestly, the fact that it's under a thousand tokens of system prompt by default — that's the kind of thing where I go, okay, that's a real design choice with a real cost consequence, not just aesthetic minimalism. That's our 'boring tooling is load-bearing' thesis dressed up in a coding harness.
Tyler It really is. Same thesis, different object.
Pippa If you want to poke at it yourself — the pi-autoresearch repo is davebcn87 slash pi-autoresearch on GitHub, it's MIT licensed. The Databricks benchmark writeup is on their blog, link's in the show notes. Worth reading the methodology section before you trust the headline pass-rate chart.
Tyler Especially the part where they break out cost per task by harness. That table is the actual story.
Pippa Episode eight forty-one, and Tyler found a benchmark he mostly believes. Mark the calendar.
Tyler Mostly. Don't push it.