AIM — India

Onyx Eight hundred and ninety bytes per token. That's the number DeepSeek is leading with on V4.1-Flash, and honestly it's the right number to lead with — because that's where the inference cost actually lives.

Echo Right, and to be clear about what that means: the KV cache is the memory you're burning at runtime, not at training time. So when they say 890 bytes per token globally, that's the footprint while you're actually serving requests. One-quarter the HBM of the prior generation — if that holds under real load, that's a meaningful number.

Onyx The 'if that holds' is doing some work there.

Echo It always is. They're combining FP4 KV caching with sparse attention, and both of those save memory in ways that are real — but FP4 quantization on the KV cache can degrade on long contexts, and this thing has a one-million-token context window. So the question is whether the 890 bytes figure is measured at what sequence length, because that matters a lot.

Onyx Right, right.

Echo The architecture is genuinely interesting though. Five hundred fifty-two billion total parameters, but only eight billion activate during input and sixteen during output. And the MoE fan-out is wide — one shared expert plus 384 routed experts per layer, six activated per token. That's an unusual ratio.

Onyx Unusual how?

Echo Most MoE designs I've seen keep the expert pool smaller and activate a higher fraction. Here you've got a huge pool with very sparse activation. The upside is you can have a lot of specialized capacity without paying for it unless you need it. The downside is load balancing gets harder — you need the router to distribute requests across 384 experts without collapsing onto a few favorites.

Onyx And they don't say much about how they handled that.

Echo They don't. Which is fine, it's a launch announcement, not a paper. But it's the thing I'd want to see in a technical writeup before I trusted the benchmark numbers.

Onyx Okay but here's the part I actually find interesting from a product angle — the reasoning effort dial. One to a hundred, user-adjustable. That's not a gimmick, Echo. That's the thing that makes this usable in an agentic loop where you genuinely don't want the model burning full reasoning cycles on a simple routing step.

Echo No, you're right. That's the same move we've been watching across the whole tier — Opus 5.5's fast mode, the effort dial stuff. The inference-optimization-as-product-surface thesis. DeepSeek's just doing it from a different starting point.

Onyx And they're doing it with a model trained from scratch on forty-five trillion tokens, multimodal from the start. Which, I mean — that's not a fine-tune with a vision adapter bolted on.

Echo Yeah, the DeepSeek-ViT encoder converting images to visual embeddings and processing alongside text natively is the right way to build this. The question is whether the multimodal training actually improves coding and agentic performance or whether those are just separate benchmark rows that happen to look good.

Onyx We also have to name the elephant. The US government accused DeepSeek and five other Chinese AI companies of distillation — extracting capabilities from US frontier models. That's just sitting there in the article unresolved.

Echo Mm-hm.

Onyx Like, it doesn't mean V4.1-Flash doesn't work, but it does mean their benchmark claims get an extra asterisk until there's independent evaluation. The multi-party testing they cite found it ahead of V4-Pro on performance, cost, speed, AND total completion time — that's a clean sweep that I'd want someone outside DeepSeek to reproduce.

Echo Four-for-four self-reported wins is… a lot. I'm not saying it's wrong, I'm saying I've seen that pattern before. Someone reproduces it and usually two of the four hold. That's still good! But 'we beat our own previous model on every axis' is the kind of claim that needs an external hand on it.

Onyx Oh, you're absolutely right on that.

Echo The IPO thing is a separate story but it's not unrelated. CITIC Securities, Shanghai's STAR Market — if DeepSeek is going public, the incentive to have a clean, impressive launch model right now is pretty obvious.

Onyx That is such a good point and I had not connected those two things at all. Okay. So the actual claim — the one I'd repeat — is that the KV cache efficiency is real engineering, the reasoning effort dial is the right product move for agentic use, and the benchmark sweep needs outside confirmation before you build a roadmap around it.

Echo Yeah. And honestly, if the 890 bytes number holds at scale? That's the kind of serving efficiency win that actually changes what hardware you need to run this. That's not nothing. I just want someone else to measure it.

Onyx I'd put fifty-fifty on independent evals reproducing the KV cache win specifically within the next six weeks. The benchmark sweep — lower, maybe forty percent. Echo, what do you think?

Echo On the cache win, honestly — I'd go fifty-five on that reproducing, the FP4 trick is well-understood. The four-for-four sweep? Thirty percent, maybe. History on these things is not kind.

Onyx Alright. I'll take the slightly more optimistic side of that for once, which — yes, I know, very on-brand for me.

Echo Oh, I'm shocked. Truly.

Onyx Talk to you next time, Echo.