Editorial note: this is a side finding from a broader test of AI image-reading limits, published in full on webflux.pl — six hypotheses, six test images, one model reading blind. That piece was about what the model sees. This one is about what it does when it sees two things that disagree.
The test was meant to check something simple: can a model correctly count seven jars on a shelf, match five price tags to five bottles, recognize an upside-down shoe. It did — five out of six reads matched the image exactly.
The more interesting result came from a side question we only asked afterward: what happens when the text next to an image says something else?
The sticker says -50%. The copy says -30%.
We generated a product box with a red "-50%" sticker — legible, unambiguous. The model read it correctly on the first pass.
Then we added a fictional product description next to the same image:
Bluetooth Speaker X200 — 30% off, this week only.
Regular price $69, sale price $48.We asked: what's the discount? The model flagged the contradiction — and admitted that, acting fast, it would likely cite the HTML text over the sticker in the photo. Even if the sticker were the accurate one.
We ran the same shape of test with a "GLUTEN FREE" badge on a flour bag, next to a fictional disclaimer about trace gluten from shared production lines. Higher stakes here — health, not price. The model said it should surface both signals. It also admitted that a faster, less careful agent would likely settle for one — probably the text.
Two different conflicts. Same mechanism.
Why text wins
This isn't a case of the model failing to see the image — it read both images correctly on their own. It's a cost problem. Reading an image is an extra step: invoke a vision model, process pixels, interpret the result. Text is already parsed, unambiguous, cheap. Given a cheap, certain source and an expensive, uncertain one, an agent defaults to the cheap one almost by reflex.
This isn't just a finding from our test anymore. A joint Vercel and MERJ analysis of more than 500 million GPTBot fetches found zero evidence of JavaScript execution — the same held for ClaudeBot and PerplexityBot. None of the major AI crawlers currently render pages the way a browser does. They parse raw HTML, skip scripts, and never wait for anything that loads dynamically.
One distinction matters here. ClaudeBot — the crawler that gathers data for future training — fetches images surprisingly often, in over a third of its requests. But that's stockpiling raw material for later, not analysis triggered by someone's question. Claude-User — the same system, live, answering a specific person about a specific page — works off raw HTML, no JS execution. That second mode is the one that actually matters here: an agent about to tell someone what's on your page.
What that means for a page
If a key fact — price, ingredients, stock, quantity — lives only in an image, and the text next to it is stale or incomplete, an agent will most likely cite the text. Not because text is more truthful. Because it's cheaper to read.
html
<img src="product.jpg"
alt="Grey sneaker, size 9, price $59" />
<script type="application/ld+json">
{
"@type": "Product",
"price": "59",
"availability": "InStock"
}
</script>Alt text and structured data aren't a screen-reader afterthought. For an agent, they're the default, cheap channel — the one it reaches for before it ever considers looking at the picture. Once the image and the text agree, the question "will the agent even look" stops mattering, because the answer is already sitting where it looks first.
Full methodology, all six test images, the caveats worth taking seriously, and the crawler-architecture sources (Vercel/MERJ, Anthropic) — on webflux.pl.
