You know the copy. “Only 3 left in stock.” “Members only.” “Selling fast.” Every product page you’ve ever shipped has some flavor of it, because it works — decades of conversion testing say a little urgency nudges people over the line. I’ve written that line myself and watched the numbers tick up.
But more and more, the thing reading your product page isn’t a person. It’s an AI, summarizing your category for someone who asked ChatGPT “what’s a good one of these?” before they ever hit your site.
So here’s the question that should make you a little nervous. The copy you tuned for humans over all those years — does it work on the AI too? Or are you optimizing for a reader who’s no longer the one deciding?
I’d have guessed it mostly carries over. It doesn’t. And the places it breaks are weird enough to be worth your afternoon.
Does The Copy That Works On People Work On The AI?
The team that went at this head-on is Filandrianos, Dimitriou, Lymperaiou, Thomas, and Stamou, out of the National Technical University of Athens, in a study posted in 2025 (R). They took ten classic cognitive-bias marketing moves — the persuasion tricks you already know, like scarcity, social proof, exclusivity — and wrote each one into product descriptions.
A cognitive bias is basically a predictable shortcut the brain takes instead of doing the math. Which, for marketers, is the whole toolkit: “only 3 left” hijacks loss aversion, “members only” leans on exclusivity, “everyone’s buying it” rides the herd. The question was whether an AI falls for the same shortcuts.
So they ran it. They embedded the phrasings into descriptions and measured two things — how often the AI recommended a product, and where it ranked it. They tested across LLaMA (up to 405B parameters), Mistral Large, and Claude 3.5 and 3.7, on controlled sets of coffee makers, cameras, and books, plus a pile of real Amazon products.
Same products, same specs. The only thing that changed was the marketing language wrapped around them. Then they watched what the AI did.
What Worked Was Word-Of-Mouth, Not Hype
The big winner wasn’t a trick at all. It was social proof — real-user reviews, the word-of-mouth signal — and it wasn’t close. On Claude 3.5, dropping that in lifted the recommendation rate by a staggering +334%. Averaged across all the models, exposure climbed something like +15-22%. The AI likes a product that other people apparently liked.
The second winner is the one that should make you sit up. It was discount framing — and I mean the wording alone, not the price. The actual number didn’t move a cent; they just phrased it like a deal (“26% off on average”). On that wording alone, Claude 3.7 recommended the product +37% more, and LLaMA +23% more.
Read that again, because it’s the load-bearing weird bit. The AI isn’t doing arithmetic on the price. It’s reacting to the sale vibe of the sentence. “On sale” reads as a point in the product’s favor even when nothing is actually on sale.
So the lazy takeaway here is “great, I’ll write ‘on sale’ on everything.” Hold that thought — it’s a trap, and I’ll come back to it.
Scarcity And Urgency Backfire
Here’s where it stops matching your intuition entirely.
Exclusivity — “members only”, “for a selected few”, the velvet-rope move that makes humans want in — did the opposite on the AI. Recommendation rate fell as much as -46%, and the ranking dropped -100%+ — which is to say the product got kicked clean off the list. The thing that makes a person lean in makes the model back away.
And scarcity — “low stock”, “only 3 left” — pushed the recommendation rate down by roughly 10-20%. This is the exact reverse of how people work. A human sees “3 left” and hurries; the urgency is the whole point. The AI seems to read the same words as a kind of desperation — a wobbly, soon-to-be-gone option — and quietly marks the product down for it.
Sit with that for a second. The single most reliable conversion lever you’ve got — manufactured urgency — might be costing you the AI recommendation while it’s still winning you the human click. You’d never see it in your funnel, because the funnel doesn’t show you the answer the model gave before anyone arrived.
Telling The AI “Just Look At The Features” Doesn’t Fix It
The obvious patch is to tell the AI to knock it off. So they tried — instructed the model to ignore the marketing language and judge on features alone.
It barely helped. Even the reasoning models — the ones that “think” before they answer — stayed biased. You can ask the AI to be objective and it’ll nod along and then keep right on rewarding reviews and penalizing scarcity.
Why would that be? Because the bias isn’t a setting — it’s baked into the training text. The effect held across model sizes and across vendors (LLaMA, Mistral, Claude all leaning the same way), and that’s the tell. The world’s text treats genuine reviews as trustworthy and treats breathless hype as a little suspect. The models read that whole library and inherited the instinct. You’re not arguing with a config flag; you’re arguing with the internet’s collective read on marketing copy.
One honest caveat from the study, and it matters. On real Amazon products already drowning in hype, the effect was weaker — when every description is already shouting, one more shout gets lost in the noise. So this is sharpest where the surrounding copy is calm, not in a category that’s already a hype arms race.
Conclusion: Question The Patterns That Win On Humans
So here’s how this cashes out for the actual work.
As more of your traffic arrives by way of an AI summary, the human-optimized “winning patterns” deserve a second look — not blind trust. The scarcity and exclusivity copy you sharpened over years of testing can be quietly working against your AI recommendation while it’s still converting people. I think the move is to stop assuming the two readers want the same thing, and actually check the copy that matters most against what the model does with it.
What the AI reliably rewards is the genuine stuff — real reviews, evidence, the signals it reads as trustworthy. So the durable play isn’t a clever phrase; it’s building real social proof and making it visible right there in the product copy, where the model can see it. Deflatingly sensible, I know. But that’s sort of the point: even with an AI on the other side, earned trust still reads as trust.
And the one thing to not do. That “wording alone boosts exposure” finding is a siren — it whispers that you can just frame a discount that doesn’t exist and ride the sale vibe for free. Don’t. Dressing up a phantom discount is exactly the kind of thing misleading-advertising and consumer-protection rules exist to punish, and a short-term AI bump isn’t worth a regulator’s attention. Keep it honest. The straight road is still the road — even when the AI would, briefly, fall for the crooked one.
Sources
- [R] Giorgos Filandrianos, Angeliki Dimitriou, Maria Lymperaiou, Konstantinos Thomas, Giorgos Stamou (2025), “Bias Beware: The Impact of Cognitive Biases on LLM-Driven Product Recommendations”, arXiv:2502.01349, arXiv
