“If you want the AI to cite you, write with sources and numbers.” You’ve seen this advice going around lately, haven’t you.
I took it at face value and started doing it too.
But something’s been nagging at me.
Say the trick works. Inside the AI’s head, what actually happens that makes it pick my paragraph over the other one?
“It just goes with whichever one sounds more convincing” isn’t an answer. It’s the same as reviewing a meal with the single word “tasty” — it sounds like a verdict, but it explains nothing.
A paper came out that pried this exact moment open at the circuit level — the instant an AI decides which source to trust. So let’s dig into it.
How Does An AI Even Decide “Which One To Believe”?
What we’re looking at is a mechanistic-interpretability paper posted to arXiv in May 2026 (the paper).
Mechanistic interpretability is basically opening the AI down to the wiring and tracing why it landed on a given answer, instead of guessing from the outside. Which, for you, means someone can now point at the specific part that chose a source, rather than shrugging at “the model felt like it.”
What the team set out to do was catch the moment an AI gets talked out of the right answer, and pin it to a circuit inside the model.
They Fed The AI Deliberately Wrong “Persuasion”
They ran four open-source model families: Llama-3, Qwen-3, Gemma-2, and Gemma-3.
Then they threw two kinds of task at them. The first is NQ2, a factual quiz with persuasion bolted on.
Each question has four options: the correct answer, one deliberately planted misleading target, and two distractors.
The second is Geo-Bench, which makes the AI pick which source to use.
Of four candidate sources, exactly one gets swapped for a GEO-optimized version — GEO (generative engine optimization) is basically SEO aimed at the answer an AI gives instead of the links Google shows — and the other three are left untouched.
On top of that, they used intervention-based causal analysis — ablating the circuit one component at a time to see which part is actually doing the work.
The Answer Came Down To One Middle-Layer Head
And the result is genuinely startling.
The model’s final answer was decided, almost entirely, by a very small number of attention heads in the middle layers. An attention head is one of the parts inside the model that decides which bits of the input to focus on.
The one named as the lead actor is head 24 in layer 17 — L17H24, a single component.
This head has a funny property: it just copies whichever option its attention happens to be pointing at.
So it isn’t weighing right against wrong. Which way its attention turns is what mechanically sets the answer.
Persuasion Doesn’t “Erode Confidence” — It Makes The AI Jump
Here’s the question that grabbed me: inside a persuaded AI, what’s actually changing?
You’d naively guess its confidence in the right answer slowly bleeds out, until the wrong one finally overtakes it.
But that’s not it. What happens is a discrete jump — the model leaps straight from the correct-answer cluster to the wrong-answer cluster, all at once.
The trigger is a signal the paper calls an “evidence-routing feature” (a rank-one one), formed in the shallow layers, around layers 8 to 12.
It’s born from shallow-layer heads that fire on persuasive keywords, and it re-points where the decision head is looking.
So the whole thing is a single track: a keyword flips the attention target, and the decision head copies whatever’s now under it.
But let’s keep the number honest. In high-stakes domains, the rate at which the AI ditched the correct answer and switched to the planted target was 29 to 62%.
Roughly half the time it bites, which you can’t wave off — but the flip side is it’s not every time.
This Circuit Was Running In Actual AI Search Too
So far this is all the factual-quiz world. But here’s the real point.
The same circuit they found on NQ2 was running, unchanged, on Geo-Bench — the task where the AI picks a source.
That hands the folk wisdom “writing for the AI to read works” an actual mechanism underneath it.
The single GEO-optimized source pokes the shallow-layer heads, pulls the attention over, and the decision head adopts it.
Not “it just seemed more convincing,” but “a specific keyword re-pointed the attention.” That’s a claim you can actually check.
The paper calls this circuit “narrow, monitorable.” The influence isn’t smeared thin across the whole model — there’s a clear path you can watch.
Still, let’s set the caveats down properly. This analysis is limited to a controlled multiple-choice setting, not free-form production use.
And whether a defense that targets this circuit actually holds up hasn’t been evaluated yet.
Conclusion: If The Circuit Is Narrow, You Can Measure How AI Talks About You And Act
So what this paper hands a marketer is, weirdly, pretty clear.
How an AI talks about your brand isn’t a “mood” — it’s a specifiable process.
If persuasive-but-wrong information about you is sitting out there, the AI can jump to it. That 29-to-62% is exactly that.
Which flips the other way too: if the correct facts sit out there in a persuasive form, the attention has room to land on them instead.
So the moves narrow down to two.
- Put the correct facts somewhere third parties can see them, in a persuasive form
- Keep measuring how the AI describes your brand right now
A narrow circuit means a monitorable one. This isn’t an untouchable black box.
How you get talked about by AI isn’t something to watch helplessly — it’s something you can measure and act on. Start by counting how you’re described today.
Sources
- Xiangkun Sun, Lingkai Kong, Aoqi Zhang, Liang Zeng, Tonghan Wang, “How LLMs Are Persuaded: A Few Attention Heads, Rerouted”, arXiv:2605.09314, May 10 2026, arxiv.org (Across four open-source families — Llama-3, Qwen-3, Gemma-2, Gemma-3 — the authors ran a persuasion-augmented factual QA task, “NQ2”, and a source-selection task, “Geo-Bench”, under intervention-based causal analysis. A small number of middle-layer attention heads decide the answer, with the lead decision head being layer 17, head 24 (L17H24), which copies whichever option its attention points to. Persuasion is not a gradual erosion of confidence but a discrete jump from the correct-answer cluster to the wrong-answer cluster, triggered by a rank-one “evidence-routing feature” formed in shallow layers, around layers 8 to 12, by heads reacting to persuasive keywords. In high-stakes domains the switch rate to the planted target was 29 to 62%. The same circuit transfers to Geo-Bench, i.e. real generative-engine source selection. The paper describes the circuit as “narrow, monitorable.” Limitations: analysis is limited to a controlled multiple-choice setting, and a defense targeting this circuit has not yet been evaluated.)
