You ask an AI something and back comes an answer that sounds about right. Fine.
But try to say why that particular answer came out, and everything goes soft.
We call that soft feeling “the black box”, and then most of us stop there. I did too — filed the whole thing under unknowable and moved on.
But people have been quietly prying the box open for a few years now. And they’ve gotten further than I expected.
What “Looking Inside Claude” Actually Means Now
The flagship example is Anthropic’s Monosemanticity work.
Monosemanticity is a mouthful that means something simple: one meaning per thing. The research hunts for parts inside a model that correspond to a single concept, instead of a soup of everything at once.
What makes it interesting isn’t the pretty pictures, though. It’s that once you find an internal feature, you can push on it — and the output moves. So these aren’t labels stuck on after the fact. They’re closer to knobs that actually turn.
It Started With 4,096 Units Of Meaning
The starting point is Towards Monosemanticity, which Anthropic published in 2023.
They took a one-layer transformer with 512 neurons and pulled 4,096 interpretable features out of it with a sparse autoencoder (think of it as a machine that breaks a messy internal state down into a small number of meaningful parts).
And the features landed on surprisingly crisp categories: DNA sequences, legal language, HTTP requests, Hebrew text, nutrition labels. Once you see examples that specific, it starts to feel like you’re genuinely looking at something.
The move that matters here is the change of unit. They stopped treating one neuron as one meaning and made combinations of features the thing to analyze — pulling the meaning out of a mixed signal, rather than hoping each bulb lights up for exactly one idea.
Claude 3 Sonnet Came Back With About 34 Million
Nice, but that was a tiny model. Does any of it survive at the size of something people actually use?
So in 2024 Anthropic published Scaling Monosemanticity and extracted about 34 million features from a middle layer of Claude 3 Sonnet. That’s roughly 8,000 times the 4,096 from the first round.
And the features aren’t just word labels. The reported ones run from the concrete to the abstract: the Golden Gate Bridge, Michael Jordan, “deceptive behavior”.
What sticks with me is that the same concept showed up as a feature shared across multiple languages and across images. Not an English word, then. Something more like a cluster of meaning that several different inputs point at.
Turn The Knob And The Model Changes What It Says
The famous experiment: crank up the Golden Gate Bridge feature artificially, and Claude’s output drifts Golden-Gate-ward.
That’s not a metaphor. It’s also not a knob in the product that a user gets to touch — it’s an internal activation, changed on purpose, with a causal effect on what comes out.
Which makes “the model is just stringing plausible words together” a bit lazy as a description. Inside, there are features tied to concepts, and they’re involved in the answer.
And Anthropic reports features tied to safety, too: deception, sycophancy (the model telling you what you want to hear), bias, dangerous content. That’s the part with your name on it, professionally speaking.
The Circuits Show The Model’s Habits, Too
Then 2025’s Circuit Tracing went past single features and followed how features connect on the way to an output.
The subject was Claude 3.5 Haiku, and the flow of internal computation gets drawn as an attribution graph — basically a map of which feature pushed on which.
Some of the examples are great. On translation, mid-layer concept representations were shared across languages. Writing a rhyming poem, the model picked the rhyming word before it wrote the line. Doing addition, it ran approximate and precise routes in parallel.
So “it just predicts the next word, one at a time” is true and also not the whole story. It’s a bit like a cooking show where the host says “and we mix” and the ten prep steps behind it happened off camera. No wonder the finished dish is hard to reverse-engineer.
Conclusion: Watch The Habits, Not Just The Wording
So what’s in this for a marketer or a brand lead?
Not a feature you can go use, to be blunt. No company today can inspect “the internal features about our brand”, and Anthropic’s work is model understanding and safety research, not a measurement product for the rest of us. It doesn’t take up brand evaluation directly.
But it does change how much weight a single answer deserves. Underneath the sentence you can read, there may be features for people, places, danger, sycophancy, bias, all mixed in together.
So when an AI is oddly careful about your brand, or leans hard on the risks, or drifts toward one impression when a competitor is in the frame — that might not be wording noise. It might be internal representation, or an association the model learned.
Which lands somewhere small and doable. Don’t look at one answer once. Compare across several questions, several models, several contexts. And record more than whether your name showed up: record which attributes and impressions it showed up next to.
Monosemanticity doesn’t mean anyone can read a model’s mind now, and maybe I’m overreading how far this goes. But “AI answers are a black box, so there’s nothing to measure” is giving up early, when the features and the circuits themselves are already on the table.
Sources
- Bricken et al. (Anthropic, 2023), “Towards Monosemanticity: Decomposing Language Models With Dictionary Learning”, Transformer Circuits Thread
- Templeton et al. (Anthropic, 2024), “Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet”, Transformer Circuits Thread
- Lindsey et al. (Anthropic, 2025), “Circuit Tracing: Revealing Computational Graphs in Language Models”, Transformer Circuits Thread
