Causal Inference Is Different in Business | Towards Data Science

Justy This is Exploring Next, episode 336. Today: causal inference in business, and why not every product decision deserves a full science-fair proof before you move.

Cody Yeah, this matters because teams are drowning in decisions right now. Not giant once-a-year bets. Constant smaller calls about features, channels, retention work, all of it.

Justy And for real people at work, that's the pain. You need enough confidence to act, but if you wait for perfect evidence on every step, you burn the quarter.

Cody The article's core idea is decision gravity. A reversible step needs different evidence than a hard-to-reverse call like putting a couple million into a build or cutting a product line.

Justy I like that framing. People hear causal and immediately want the whole DAG-to-sensitivity-analysis pipeline, even when the user story is basically, do we spend another sprint here or not?

Cody The author gives three rules: start with the problem, not the method; use a simpler approach if it answers it; and do the 80/20 version before turning it into a research program.

Justy That middle one is where teams struggle. Saying descriptive or associative analysis is enough for now can feel impure, but if the decision is constructive, not final, that may be the smart move.

Cody The feature-investment example makes it concrete. Instead of jumping straight to impact on acquisition or retention, start with easier signals like how many users touch the feature and how often they use it. If those are weak, maybe you don't need heavier causal methods yet.

Justy Right, but those signals are for triage, not proof. Otherwise a PM walks into roadmap planning talking like the feature caused retention.

Cody And that's where the failure modes matter: self-selection, interacting segment effects, even collider bias if you condition on engagement. The article also has a good so-what test: if this answer wouldn't change anyone's action, stop.

Justy That's it for Exploring Next. Match the rigor to the decision, not your ego, and we'll catch you next time.