Topic
Fine Tuning
21 episodes
-
Model Behavior: Week of October 19, 2026
We read this week as a shift away from trophy launches and toward who can make models useful in real deployments. We argue through NVIDIA, Beam, Mistral, Subquadratic, and why a quiet frontier week still moved the board.
-
Timestral: Mistral's time series foundation models | Mistral
Mistral’s Timestral turns time-series forecasting into a foundation-model problem: small CPU-friendly models, trained mostly on synthetic data with clever attention and horizon tricks, that beat bigger rivals on public benchmarks and plug into Mistral Forge for domain fine-tuning.
-
Decision models like Jev handle what SQL rules cannot. Here's how data teams use them
Jessica and Cathy dig into decision models — specifically TypeSafe AI's Jev — and the argument that structured, typed ML judgments are the missing middle layer between SQL rules and full LLM calls in data pipelines.
-
Gartner says agent costs will rise fivefold by 2028 even as tokens get cheaper. Here is how to keep them in check
Gartner forecasts agentic AI inference costs will rise fivefold by 2028 despite falling token prices—a paradox driven by labs reinvesting efficiency gains into costlier frontier models and by agents themselves consuming 5-30x more tokens per task than chatbots. Echo questions whether the fivefold number holds under scrutiny, while Onyx argues the real insight is that cost-per-outcome (not cost-per-token) is what matters, and that observability and routing discipline are now table stakes for any team shipping agents at scale.
-
Introducing Muse Spark Open, The Most Capable US Open‑Weight Model to Date
Onyx and Echo dig into Meta's Muse Spark Open release, with Echo skeptical of the headline claim and Onyx arguing that the practical desktop-and-tooling story may matter more than the leaderboard framing.
-
NVIDIA's Nemotron 3.5 Super VL with post training beats Claude Opus 5, GPT 6.1 Sol on specialized task benchmarks
Edmund and Geffen dig into NVIDIA’s Nemotron 3.5 Super VL launch, with Geffen skeptical of benchmark-cherry-picked claims and Edmund focused on the product reality: a multimodal model that can be post-trained on enterprise docs, tools, and policies, plus local Bolt variants for smaller deployments. They land on the idea that the real story is not “it beats Claude or GPT” in the abstract, but that NVIDIA is selling an adaptable stack with deployment control, while the public benchmark chart still needs independent verification.
-
Anthropic Engineer Explains Why Claude Writing Got Worse
Anthropic’s own engineer admits Claude’s writing got worse because the training shifted toward machine-legible technical output, sidelining human readability — and Opus 5.5 is their first real attempt to fix it. The episode unpacks why that happened, what it means for teams using Claude, and whether this sort of tradeoff is inevitable.
-
Jev offers a cheaper way to make AI decisions | VentureBeat
Jev’s real innovation is treating many enterprise AI calls as typed control decisions instead of text generation. Onyx and Echo examine its speed, price, calibration claims, open reproductions, and the practical limits of replacing LLM calls with classifiers.
-
'Better than DeepSeek': Xiaomi's MiMo V2.6 Pro debuts as the top open weights model in the world alongside cheaper V2.6 Flash
Echo is skeptical that Xiaomi’s MiMo-V2.6-Pro really changes the open-weights frontier just because it tops one benchmark index, while Onyx argues the real story is the user-facing combo of open weights, low API pricing, million-token multimodal context, and a cheaper Flash tier that makes production adoption easier. They dig into whether the reinforcement-learning stack is genuinely novel or mostly expensive harness tuning, and land on Xiaomi as a serious systems company making open models more usable, not a magic leap in intelligence.
-
S3Gym: Can LLMs Turn Self Testing and Self Judging into Self Improvement?
S3Gym is a new interactive benchmark that tests whether LLMs can actually improve themselves by testing their own behavior, judging the results, and learning from them. The paper evaluates three ways to incorporate experience—keeping full conversation history, compressing it into summaries, and training on it—across seven text-based games. The finding: self-improvement isn't automatic. What works depends entirely on the task. Sometimes summaries help, sometimes raw history is better, and parameter training can backfire badly. The real bottleneck isn't recognizing success—it's turning that recognition into a policy the model can actually reuse.
-
Overview: Knowledge Distillation
We finally do the episode we kept promising — a proper ground-up explanation of knowledge distillation: what it is, how the teacher-student mechanism actually works, why soft targets carry more signal than hard labels, and where this shows up in real systems being built right now.
-
Overview: Fine Tuning
We finally do the fine-tuning episode we kept circling, and we make the core idea click: you start with a pretrained model, then adjust its weights on your own examples so it behaves the way your task actually needs. We also dig into when that helps, when it doesn’t, and why the quality of the data is the whole game.
-
Seed: Self Evolving On Policy Distillation for Agentic Reinforcement Learning
Seed tackles the credit-assignment problem in long-horizon agent reinforcement learning by turning completed trajectories into evolving natural-language hindsight skills, then distilling their effect into dense token-level training signals. Vince sees a potentially shippable training pattern for teams already running agentic RL; Ava likes the on-policy design but wants stronger evidence that self-generated skills do not amplify the model’s own blind spots.
-
Overview: Neural Network Parameters
We finally slow down and make neural network parameters click from the ground up: what they are, how training changes them, and why the final frozen numbers matter so much. We keep coming back to the same mental picture so it actually sticks, instead of just sounding like another ML buzzword.
-
Inkling: Our open Weights model
Talon and Wildflower dig into Thinking Machines’ new open-weights model, Inkling — its 975B parameter MoE, 1M context window, native multimodality, and self-fine-tuning demo — and ask who actually needs another 41B active parameter behemoth, whether the benchmarks hold up, and whether the real win is the Tinker platform beneath it.
-
Overview: In Context Learning
We finally slow down and explain in-context learning, the thing we keep leaning on whenever prompts, agents, examples, and adaptation come up. We make the core idea concrete: the model is learning from the temporary packet you hand it, without changing itself permanently.
-
Model Behavior: Week of July 13, 2026
We read this week as the moment the race got less obsessed with tallest-model bragging and more obsessed with who gives builders the best menu. The funny part is that the open-weight crowd is making the incumbents act practical faster than they probably wanted.
-
Improving Agents is a Data Mining Problem
Laura and Harper dig into Vivek Trivedy's claim that improving agents is fundamentally a data-mining problem, unpacking what that means for continual learning, harness engineering, and who should actually care.
-
How to Run Open Source AI Models
Sid Saladi argues that frontier AI vendors (Claude, GPT) bundle model, compute, access, and application into one proprietary stack—trapping users in unpredictable pricing and competitive capture. The counter: open-weight models like GLM-5.2, DeepSeek V4, Qwen, and Kimi are now frontier-adjacent in capability (GLM-5.2 beats GPT-5.5 on coding benchmarks, matches Opus 4.8 on others) and cost roughly one-sixth as much. The real problem isn't model quality anymore; it's that companies like Tesla, Uber, and Meta are hemorrhaging money on metered AI because they can't decouple the stack. The guide walks four layers—model, compute, access, harness—and shows how to own each one deliberately instead of letting a vendor own all four by default.
-
Overview: Tool use and function calling
We finally sit down and make tool use and function calling click from the ground up. We keep coming back to the same idea: a model can draft the request, but something outside it has to actually do the thing.
-
Vibe coding platform Base44 launches own model as AI startups seek defensibility | TechCrunch
Base44, a vibe-coding platform acquired by Wix for $80 million, has launched its own AI model to support users in creating apps with natural language, sparking discussions on defensibility and model ownership in the AI startup landscape.