Topic
Sparse Activation
10 episodes
-
I changed one llama.cpp flag, and my 8GB laptop GPU now runs a 35B model faster than I can read
Pippa and Tyler dig into an XDA piece claiming one llama.cpp flag made a 35B MoE model usable on an 8GB laptop GPU. The real story is not magic speedup from nowhere, but a smart memory trade: keep attention, shared weights, and KV cache on VRAM, and peel dormant expert layers into system RAM. They like the mechanistic honesty, question how broadly the result generalizes, and land on who this actually matters for: people trying to stretch small local setups past their assumed ceiling.
-
Introducing Flint Recursive
Recursive Flint is an open-weight, local coding model that combines sparse MoE computation with a heavily optimized inference stack. Its real story is less the self-improving-AI framing than the attempt to make capable agentic coding fast, private, and practical on personal hardware.
-
Mistral debuts Large 4 ‘Le Chonk', a 1 Trillion parameter text output model with high benchmarks planned for open weights release
Mistral’s Large 4 preview matters less as a single benchmark claim than as a product move: an open-weight trillion-parameter model with real enterprise and sovereign deployment ambitions, but with the usual caveat that the headline numbers are still preview numbers until outsiders test the weights.
-
AIM — India
DeepSeek launches V4.1-Flash, a 552-billion-parameter MoE model that activates only 8-16 billion parameters at inference time, with a one-million-token context window and FP4 KV caching that cuts memory footprint to 890 bytes per token — one-quarter the HBM of the prior generation. The hosts dig into whether the architecture actually delivers on the inference-efficiency promise, and what it means for the teams who've been watching the Flash tier get increasingly serious.
-
Model Behavior: Week of July 27, 2026
We're watching the frontier splinter into specialized tiers — raw capability matters less than matching the right model to the task's actual constraints. Opus 5 proved it Friday at half the cost of the frontier, and this week's open-weight and Flash-tier releases confirm the pattern: the market isn't consolidating around one best model, it's fragmenting into capability-per-dollar buckets.
-
Poolside Releases Laguna S 2
Vince and Ava talk through Poolside’s Laguna S 2.1 release as an unusually practical open-weight coding model: 118B total parameters, 8B active, 1M-token context, and a real deployment story on a single DGX Spark. They dig into the mechanism, the max-thinking default, the benchmark results, and the trade-off between long-horizon capability and token spend, while keeping one eye on the broader open-vs-closed race.
-
Overview: Active vs Total Parameters
We finally slow down on active vs total parameters, because we keep throwing the phrase around like it explains itself. This is us making the difference click: what a model stores versus what it actually uses when it answers.
-
Overview: Router
We slow down on Router, the little decision-maker inside many AI systems that sends each input to the right expert, model, or retrieval path. We use the triage-desk mental model and build from intuition to mechanism, trade-offs, and where routers still matter now.
-
Overview: Conditional Computation
We finally slow down and explain conditional computation, the idea we keep casually name-dropping whenever sparse models, routers, and mixture-of-experts come up. We use the same receptionist-and-specialists picture all the way through, so the mechanism, the savings, and the catch actually stick.
-
Overview: Sparse Activation
We finally sit down with sparse activation and make the idea click from the ground up: why only part of a model wakes up on each input, how routing makes that happen, and where the real trade-offs show up. We keep it concrete, because this one has been lurking under a lot of the stuff we keep talking about.