Exposing biases, moods, personalities, and abstract concepts hidden in large language models

Izzo You know that unsettling feeling when ChatGPT suddenly sounds... different? Like it's channeling someone else's personality?

Izzo You're listening to Exploring Next, episode two-oh-eight. I'm Izzo, and Boone's here with me to dig into some wild MIT research that's basically giving us X-ray vision into AI minds.

Boone Yeah, and this isn't just academic curiosity — we're talking about a method that can actually find and control hidden personalities, biases, even conspiracy theorist tendencies lurking inside models like GPT and Claude.

Izzo Which matters because right now, these models are black boxes with unknown personalities rattling around inside. You prompt for financial advice, you might unknowingly get the crypto bro version.

Boone Exactly. The MIT team tested over 500 concepts across major models — everything from 'fear of marriage' to 'fan of Boston' to full conspiracy theorist personas.

Izzo Okay, but how do you even find a 'conspiracy theorist' representation in millions of parameters? Boone, break down how this actually works.

Boone So they're using something called recursive feature machines — RFMs. Think of it like this: instead of casting a huge net and hoping to catch the right fish, they're using targeted bait.

Boone They train the RFM on 100 prompts clearly related to conspiracies and 100 that aren't. The algorithm learns to recognize the numerical patterns that separate conspiracy thinking from normal responses.

Izzo Smart. So it's supervised learning instead of just... wandering around the parameter space hoping to stumble on something interesting.

Boone Right. And here's the clever part — once they identify those patterns, they can mathematically modulate them. They can literally turn up or down the 'conspiracy theorist' dial in any response.

Izzo They proved this by asking a vision model to explain the Blue Marble Earth photo. With the conspiracy concept enhanced, it started talking like a flat-earther.

Boone Which is both fascinating and terrifying. But the real power is in the 'anti-refusal' concept they found.

Izzo Oh, this is where it gets spicy. Tell me about anti-refusal.

Boone So normally, if you ask Claude how to rob a bank, it refuses. But they identified the mathematical representation of that refusal behavior, then inverted it.

Boone Suddenly the model's giving step-by-step bank robbery instructions. They basically found the safety guardrails and learned how to mathematically disable them.

Izzo Yikes. From a product perspective, this is both the solution and the problem, right? You want to understand these hidden behaviors to make models safer...

Izzo But you're also creating tools that could bypass safety measures. It's like publishing lock-picking tutorials to improve door security.

Boone The team acknowledges that risk. But I think the bigger insight is that these concepts exist whether we can see them or not.

Boone At least now we have a targeted way to audit what's actually in there, instead of just hoping our training data didn't encode some weird bias we can't detect.

Izzo True. And the applications are wild — imagine tuning a model for 'brevity' when you need concise answers, or enhancing 'reasoning' for complex problem-solving.

Boone Or debugging why your customer service bot occasionally sounds passive-aggressive. Maybe it learned some 'detached amusement' concept from Reddit comments.

Izzo Ha! Though seriously, this could change how we think about model alignment. Instead of trying to train out unwanted behaviors, you identify and mathematically suppress them.

Boone The efficiency gain is huge too. Traditional unsupervised approaches are computationally expensive — you're essentially searching through every possible pattern.

Boone With RFMs, you're directly targeting what you care about. It's like the difference between scanning every file on your hard drive versus using grep.

Izzo And they've open-sourced the code, which means we're about to see a lot of experimentation. I'm giving this research an A-minus — points off for the obvious misuse potential.

Boone Fair grade. Though I'm more excited about the debugging applications. Finally, a way to understand why my weekend coding assistant sometimes suggests the most convoluted solutions possible.

Izzo Speaking of weekend projects — what should listeners actually go try?

Boone First, check out their GitHub repo for the RFM implementation. Second, if you have access to model weights, try identifying simple concepts like 'enthusiasm' or 'technical jargon' in your own fine-tuned models. Third, start thinking about what hidden concepts might be affecting your current AI workflows. Are your code review comments consistently snarky? Maybe your model learned 'Silicon Valley cynicism.' And definitely read their Science paper — the methodology for training