You hand ChatGPT the boring half of a competitor sweep, and back comes a paragraph that sounds completely sure of itself.
I do this constantly, and I catch myself relaxing every single time.
But the expensive failure isn’t the model saying “I don’t know”. It’s the model being wrong in a very steady voice — like the guy who forgot the deck but is deadly with the laser pointer, where the delivery does all the work the evidence was supposed to do.
So what is that confidence, exactly? Stage direction? Or is something inside the model quietly scoring its own answer?
Three papers pry this open, and the short version is that the confidence is real but it isn’t a correctness meter. Let’s look at where the gap comes from.
Is “I’m Confident” Just Something The Model Says?
Start with Google DeepMind’s “How do LLMs Compute Verbal Confidence?” (the paper). They worked with Gemma 3 27B and Qwen 2.5 7B, hunting for where a spoken confidence level actually comes from.
The method is the fun part. Rather than take the model’s word for it, they cut and swapped the flow of information inside it, piece by piece, and watched which cuts changed the confidence it ended up stating — which is how you trace the route a signal travels instead of guessing at it from the outside.
And the signal isn’t improvised after the fact. It shows up at the answer token, gets parked in the position right after the answer, and is pulled back out when the model has to put a word or a number on how sure it is. Something in there must be scoring the answer well before any of it reaches the sentence you read.
So the confidence isn’t theater.
And the next step looks obvious — just ask the thing how sure it is. That’s exactly where it gets expensive.
Why The Voice And The Doubt Come Apart
Meta’s “Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations” (the paper, EMNLP 2025) puts a number on the gap between two things that sound like one thing: how uncertain the model sounds (verbal uncertainty) and how uncertain it actually is across the answers it might have given (semantic uncertainty).
Which is a very human failure mode, if you think about it. You know the move — somebody says “definitely” in a meeting while privately hedging, and nobody in the room can tell the difference. The model does both directions: sometimes it’s genuinely torn inside and says the thing flatly anyway; sometimes it hedges harder than its internals warrant.
And that mismatch predicts hallucination. So the wording is a clue to the internal state; it just isn’t the state.
There’s an upside buried in that, though. The paper treats the tone as a linear feature — one dial rather than a mood — which is what makes it something you can turn, and steering verbal uncertainty at inference time cut confident hallucinations on short-answer questions by roughly 30% on average.
But that’s a research setting, not the tool open in your browser tab, and I wouldn’t assume any of it is wired into whatever you’re using. So when you’re reading what an AI says about a brand — yours or a competitor’s — the useful question isn’t whether it sounded confident. It’s whether the thing is built to surface what it doesn’t know.
Overconfidence Isn’t Only A Speech Habit
The 2026 preprint “Wired for Overconfidence” (the paper) goes a layer deeper and chases wrong-but-confident answers down to the circuit. Two models, Qwen2.5-3B-Instruct and Llama-3.2-3B-Instruct, across three datasets: PopQA, MMLU and NQOpen. In the middle-to-late layers the authors find components that inflate the stated confidence, which they call a “Confidence Mover Circuit”.
Poke that circuit and the numbers move a lot. On PopQA, ECE (expected calibration error, ie the average gap between how sure a model says it is and how often it turns out right) fell 40 to 97%, and Brier scores (another way of scoring that same gap) improved 43 to 70%. On MMLU, ECE fell 33 to 57%; on NQOpen, 81 to 83%.
Don’t bother memorizing the metric names. The point is that the gap between sounding right and being right shrank, and it shrank by touching specific parts — which makes overconfidence look less like a personality trait and more like a wiring problem.
Still, NAACL 2025’s “SCIURus” (the paper) argues that uncertainty and factuality come out of shared pieces of the same network, so “one overconfidence circuit explains all of it” is more than anyone has shown, and different papers here are looking at somewhat different phenomena.
But the load-bearing bit survives anyway. Overconfidence lives in the machinery, not just in the prose — which is a decent reason, when you’re picking an AI tool, to ask not only “can it report a confidence level” but “has that confidence ever been checked against anything”.
Conclusion: Let A Confident Tone Trigger A Check, Not A Copy-Paste
I think the honest read is that there’s some signal in an AI’s confidence, and that it’s still nowhere near a guarantee. So the safe habit is to treat a self-assured tone as a flag that says verify this before you use it, rather than as the reason you used it.
Two things, then. Before anything lands in a deck, check the answer for a source link, a date, and an explicit “here’s what I couldn’t establish” — whatever the model can’t source stays out of the body and goes in a to-check note. And ask the same question more than once, across more than one model, so what you build on is whatever survives the repeat.
Nothing steadies you against a confident voice quite like watching it fail to reproduce itself.
Maybe I’m overcorrecting. But the cost is lopsided — a check you didn’t need costs you a few minutes, and a confidently wrong number in a board deck costs you rather more than that.
If you want to build that judgment out from the inside of the model, Does The AI Actually Understand, Or Just Sound Like It? is the companion piece.
Sources
- Kumaran, D. et al. (Google DeepMind), “How do LLMs Compute Verbal Confidence?”, arXiv:2603.17839, 2026, arxiv.org
- Ji, Z. et al. (Meta), “Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations”, EMNLP 2025, arXiv:2503.14477, arxiv.org
- Teplica, C. et al. (NYU), “SCIURus: Shared Circuits for Interpretable Uncertainty Representations in Language Models”, NAACL 2025, aclanthology.org
- Zhao, T. et al. (University of Virginia), “Wired for Overconfidence: A Mechanistic Perspective on Inflated Verbalized Confidence in LLMs”, arXiv:2604.01457, 2026, arxiv.org
