Overview: Predictive Modeling

Masonry Okay so we keep saying 'predictive modeling' on this show like everyone already knows what it is. And honestly, a bunch of people have told us — just explain it. Like actually explain it. So that's what we're doing today.

Eyre Yeah, and it's worth it, because this is genuinely the thing underneath almost everything we cover. Not a sub-technique of something else. The core task. So let's actually do it justice.

Masonry Okay, before any terminology — give me the intuition. What is a predictive model even doing?

Eyre So imagine you're learning to predict rain. Not from a weather app — from watching the sky for years. You start to notice: when humidity is high, pressure drops, and clouds go dark, it usually rains. You didn't read that in a textbook. You just saw it enough times that it stuck.

Masonry Right.

Eyre A predictive model does the exact same thing, but with numbers and potentially thousands of signals instead of three things you can see. It looks at a huge pile of past examples — each one has some inputs and a known outcome — and it finds the patterns that connect them. Then when it sees a new situation it's never encountered, it applies those patterns to make a guess.

Masonry So it's basically pattern memory at scale.

Eyre Exactly. And that framing — the weather-watcher who's internalized signals — I want to keep coming back to that one, because it holds up through all the technical layers we're about to add.

Masonry Good. Keep me honest on that. So what are those inputs actually called?

Eyre The inputs are called features. So in our rain example, humidity, pressure, cloud cover — those are features. The thing you're trying to predict is called the label. Rain or no rain. Tomorrow's temperature. Whatever it is. And this part matters a lot: the quality of your features is basically the ceiling on how good your predictions can be. If you feed the model garbage signals — stuff that has no real relationship to what you're predicting — the model cannot save you.

Masonry Which is why people always say feature engineering is the actual hard part. You can have a great model architecture and still completely blow it if you're feeding it the wrong inputs.

Eyre Yeah, and 'feature engineering' just means: deciding which signals to include, how to transform them, how to combine them. It's less glamorous than the model itself but it's often where the real work is.

Masonry Okay so you've got features and labels. How does the model actually LEARN from them?

Eyre So the model starts with random guesses — essentially zero knowledge. It looks at a training example, makes a prediction, and then checks how wrong it was. That wrongness gets measured by something called a loss function — the short version is: it's a number that tells the model exactly how far off its guess was. The model then adjusts its internal parameters — think of them as the dials on our weather-watcher's intuition — to reduce that loss.

Masonry And that process of training on labeled examples — there's a name for that whole approach, right?

Eyre Supervised fine-tuning. The 'supervised' part just means the training data has known labels. You're not asking the model to figure out structure on its own. You're showing it the right answers and having it learn to reproduce them.

Masonry Okay. So the model learns from past examples. But then what? How do you know if it actually learned something real versus just… memorized the homework?

Eyre That is the question. You hold some data back, keep it completely separate, and the model never sees it during training. After training, you run the model on that held-out set and see how it does. If the model does great on training data but falls apart on the test set, it memorized the specific examples instead of learning the underlying pattern. That failure mode is called overfitting. And then there's the longer-term problem: the world keeps moving.

Masonry And the fix isn't just 'retrain once' — it's an ongoing monitoring and retraining loop.

Eyre Ongoing, yeah. Which is a real operational cost that doesn't always get priced in when teams first deploy something. The model isn't done when it ships. It needs a keeper.

Masonry Okay so let me make this concrete. What are some real examples?

Eyre So Decathlon — one of the largest sporting goods retailers in the world — they're running demand forecasting at scale with a model called Chronos-2. The features are things like past sales, seasonality, promotions. The labels are future demand numbers. That's regression, straight down the middle of what we've been describing.

Masonry And the scale problem there is real — thousands of products, hundreds of markets, all with different seasonal patterns. You can't hand-tune that.

Eyre Right. And then on the weather side — Google DeepMind's WeatherNext model. It predicted Hurricane Melissa's landfall in Jamaica well enough that the National Hurricane Center was able to issue earlier warnings than would have been possible with traditional forecasting. And there's a newer result from August — cyclone forecasting with an extra day of warning on average.

Masonry An extra day of warning for a cyclone is not a small thing. That's evacuation time.

Eyre That's lives. And the reason I bring that one up in this context is: it's still fundamentally the same structure. Historical atmospheric data as features. Future storm track and intensity as labels. The model found patterns the traditional physics-based simulators were missing or approximating too coarsely.

Masonry We also looked at that brain prediction work — the thing where researchers were using prediction models not just to forecast but to generate testable hypotheses about how the brain actually works.

Eyre Yeah, that one stuck with me. The model isn't just predicting — it's becoming a proxy for a theory. If the model predicts brain activity well, that's evidence the features you chose actually matter to the underlying mechanism. So it's predictive modeling as a scientific tool, not just a forecasting tool.

Masonry Which is kind of a beautiful use of it. And then there's the newer direction — models like TimesFM-3, which just dropped from Google Research. Zero-shot forecasting across multiple variables at once. The idea being: you don't train a separate model for each prediction task. One model generalizes across many.

Eyre And that's a real shift in how this gets built. The traditional approach is: collect data for your specific domain, train a model for that domain, deploy it. The newer foundation-model approach says: train on a huge variety of time-series data, and the model learns patterns general enough to transfer. You still need to evaluate it on your held-out test set — the fundamentals don't go away — but the upfront data burden drops a lot.

Masonry Okay. So where does this all sit right now? Like, is predictive modeling still the thing, or has it been absorbed into something newer?

Eyre Still the thing. No asterisk. The terminology around it has evolved — you'll hear 'forecasting', 'inference', 'generative modeling' — but the core task is the same: learn from historical examples, predict on new ones. Every system we cover on this show that makes a decision based on data is doing some version of this. It's not a technique from a previous era. It's the current practice.

Masonry Yeah, and I think what's changed is the scale and the generalization. The models are bigger, the feature spaces are richer, and the zero-shot stuff like TimesFM-3 means you don't always need domain-specific training data to get started. But the question you're always asking is the same: did the pattern you found actually hold?

Eyre That's the whole game, Masonry. Everything else — the architecture, the training procedure, the evaluation metrics — it's all in service of that one question. Did the pattern I found in the past actually hold in the future I was predicting?

Masonry Okay. So if someone is walking away from this and they want one thing to stick — what's the sentence?

Eyre A predictive model is a machine that has internalized patterns from past data and applies them to situations it's never seen. The weather-watcher who's seen ten thousand storms. Everything else is details about how you build that, measure it, and keep it honest when the world moves.

Masonry I like that. 'Keep it honest when the world moves.' Eyre, for a guy who spends half his time being skeptical about AI claims, you just made the whole thing sound almost poetic. Don't let that get out.

Eyre I will deny it.

Masonry Alright. Episode nine twenty five — we finally did it. See you next one.