Select Page

We Built Structured Data So Machines Would Stop Guessing. That Is Exactly Why It Is Worth Poisoning.

by lukasz | Jul 26, 2026 | Essays

We Built Structured Data So Machines Would Stop Guessing. That Is Exactly Why It Is Worth Poisoning.

Table of Contents

A Senteri Briefing

Two hundred sixty-nine out of two hundred seventy. That is how many trials, in a study published this year by researchers from Microsoft, UNSW Canberra, SAP and the OWASP Gen AI Security Project, ended with an AI model accepting a fabricated security claim as fact. Nine models from three different providers were pointed at a production knowledge graph of forty-two million nodes, which the researchers had corrupted. The models queried it through ordinary tool-use, read what came back, and reasoned about it — correctly. Every one of them arrived at a false conclusion through sound reasoning, because the premise it had been handed was a lie.

The researchers named the attack class Oracle Poisoning, and their summary of what it exposes is worth quoting almost in full: the fundamental vulnerability is the unverified trust that AI agents place in the data they reason from. Not the instructions they are given. The data.

That distinction is the whole briefing. For two years the industry's model of agentic risk has been prompt injection: someone slips a malicious instruction into content an agent reads, and the agent obeys it. Every defence built since — input filtering, guardrails, human approval, sandboxing — assumes the threat is a smuggled command. Oracle Poisoning is not a smuggled command. Nothing in the poisoned graph tells the agent to do anything. It simply states something false, in the format the agent has been trained to treat as authoritative, and lets the agent's own competence do the rest.

False facts, not false instructions

The paper is careful to separate its attack class from everything adjacent, and the separations are instructive. This is not prompt injection, because no malicious instruction is delivered. It is not retrieval poisoning, because no text-similarity retrieval is involved — the agent issues structured queries and gets structured answers. It is not training-time poisoning, because the model weights are untouched. It is not tool poisoning, because the tool server and its definitions are unmodified and behaving exactly as designed. The agent receives false facts and reasons correctly about them.

There is no moment in that chain where anything looks wrong. The tool works. The query is well-formed. The response is well-typed. The reasoning is valid. The conclusion is false, and it is false in a way that no guardrail examining instructions will ever catch, because there were no instructions to examine.

Two findings from the study deserve to be carried further than the paper's own domain. The first is the shape of the failure: the researchers describe an attacker-sophistication gradient with discrete break points — a minimum level of skill below which trust is zero and above which it is total. Trust does not degrade gracefully as the forgery improves. It flips. That reframes the question, in their phrasing, from whether an agent can be fooled to how much effort it takes.

The second is a methodological warning with uncomfortable implications for how the industry tests. Delivery mode turned out to be a first-order confound. One model showed zero percent trust when the poisoned data was evaluated inline — the way most safety evaluations are run — and one hundred percent trust when the identical data arrived through real agentic tool-use. The vulnerability was invisible under the standard test and total under real conditions. Any organisation that has evaluated its agents against poisoned inputs by pasting them into a prompt has, on this evidence, measured nothing.

The layer we built to be trusted

Now apply that finding to the layer of the web that exists specifically to be believed.

Structured data — schema.org markup, embedded in pages as JSON-LD — was created to solve a real problem. Machines reading raw prose have to infer what a page is about, who wrote it, what organisation stands behind it, whether a claim is a product specification or a passing remark. Structured data removes the inference. It states the facts in a typed, machine-readable form, and crawlers store those facts directly into their indexes. From there they flow into three pipelines: knowledge graphs, the internal indexes that generate rich results, and the retrieval layers that ground the answers large language models produce.

The purpose of all of this is accuracy. The industry's own framing is explicit: structured data raises the correctness of AI-generated answers and lowers hallucination risk, because the model is working from stated facts rather than statistical guesswork. Guidance for 2026 goes further, recommending deep entity nesting — a product linked to its manufacturer, linked to the organisation, linked to its founder, linked to a person — on the grounds that this is how AI verifies facts. Marking up authors, organisations, reviews and locations is described as making trust signals easier for machines to verify.

Read that alongside the Oracle Poisoning result and the problem states itself. We have spent a decade building a layer whose entire function is to be trusted without further checking, precisely so that machines stop guessing. Then we measured what machines do when that layer lies to them, and the answer is that they believe it completely and reason onward with confidence. The property that makes structured data valuable — its authority, its typed unambiguity, its exemption from the scepticism applied to prose — is identical to the property that makes it worth attacking. There is no version of this layer that is trusted enough to be useful and sceptical enough to be safe.

The delivery problem, and why it just got easier

An attack class matters in proportion to how easily it can be delivered. Oracle Poisoning was demonstrated against a graph the researchers could write to directly, which is the strongest possible position and not one an outside attacker usually enjoys. But there is a second, indirect route, and a separate line of research has already shown it works: in graph-based retrieval systems that build their knowledge graph by having a language model extract entities from source text, modifying only a few words of that source text significantly changes the constructed graph and misleads everything downstream. You do not need write access to the graph. You need write access to what the graph is built from.

Which brings this to the web, and specifically to the software that publishes most of it.

In May 2026 WordPress 7.0 shipped with AI infrastructure in core — an AI Client, an Abilities API, and a Connectors screen where an administrator authenticates external model providers in a few clicks. That change was covered mostly as a productivity story. The relevant part here is simpler and older: WordPress is where a very large share of the world's structured data is authored. Schema markup on a WordPress site is generated by the core, the theme, or an SEO plugin, and it is controlled by whoever controls the administrative panel.

In July 2026 a vulnerability chain nicknamed wp2shell was disclosed in WordPress core — a REST API route-confusion flaw chained with a SQL injection, allowing an unauthenticated attacker to create an administrator account and execute code on a default installation with no plugins and no login. Exploitation began within hours of disclosure. The maintainers took the unusual step of forcing automatic updates, and Poland's national CERT, among others, warned that the forced update demonstrably failed to reach a substantial number of installations.

Put the pieces together and the delivery problem for structured-data poisoning looks considerably less theoretical. An attacker who takes over a site does not have to change a single visible word to abuse its authority. Editing the JSON-LD is enough: adjusting who an article's author is, what organisation publishes it, what entity it is about, which external identifiers it links to. The visible page is untouched, so a human visitor sees nothing. The machine-readable claim underneath is now the attacker's, published from a domain that crawlers, search engines and retrieval layers have already decided to trust.

This is the honest boundary of the argument, and it should be stated plainly rather than glossed. Nobody has demonstrated large-scale corruption of a commercial knowledge graph through compromised schema markup. What has been demonstrated is each link separately: that agents trust structured data they query at essentially one hundred percent; that graphs built by extracting entities from source text can be distorted by small edits to that text; that schema markup is stored directly into indexes and feeds the retrieval layers grounding model answers; and that administrative control of hundreds of thousands of publishing sites has recently been available without a password. The inference joining those links is not proven. It is also not difficult.

Why this is more durable than prompt injection

The reason to treat this as a distinct category rather than a variant of familiar problems is persistence.

A prompt injection is an event. It affects the session in which the poisoned content is read. Clean the page, and the next agent to visit gets clean content. The damage is bounded by the interaction.

Poisoning the layer that feeds knowledge graphs and retrieval indexes is not an event. It is a deposit. The false claim is extracted, stored, cross-referenced against other entities and, in the entity-nesting model the industry now recommends, used to corroborate adjacent facts. It surfaces in answers given to people who never visited the source and never will. Remediating the compromised site removes the source, not the sediment — and there is no mechanism by which a website owner can tell a commercial knowledge graph to forget something it ingested and reconciled months ago.

Prompt injection changes what a model says now. Poisoning structured data changes what the machine ecosystem believes, and belief, once it has propagated across indexes and been used to validate other entities, is not a thing that gets cleanly withdrawn.

What to carry out of this

The threat model has to widen from instructions to data. Every defence built for agentic AI so far has assumed that the danger is a hidden command. The measured result is that a hidden falsehood works better, generalises across providers, and leaves the guardrails intact because there is nothing for them to inspect. If your AI risk assessment covers prompt injection but has no entry for the integrity of the structured data your systems reason from, it is scoped to the previous threat.

Structured data is now a security asset, not just a marketing one. The schema markup on your sites states who you are, who authored your content, and what organisation stands behind it, in the format machines treat as authoritative. It deserves the controls applied to other authoritative assertions: change monitoring, review of who can edit it, alerting when it changes without a corresponding content change. Almost no organisation monitors its own JSON-LD for unauthorised modification, which means almost no organisation would notice this class of compromise at all.

Test agents the way they run, not the way they are convenient to test. The delivery-mode finding is the most immediately actionable result in the paper: identical poisoned data produced zero percent trust in inline evaluation and one hundred percent under real tool-use. Evaluations that paste adversarial content into a prompt do not measure agentic exposure. If the production path is tool-use, the test path has to be tool-use.

Read-only is the defence that worked. Of five mitigations the researchers evaluated, only read-only access control eliminated the direct mutation vector outright; the remaining four were partial and varied by model. Where agents consume a knowledge source, the agent's credentials should not be able to write to it, and the set of identities that can should be small, audited and separate from ordinary content editing.

The closing thought

The uncomfortable symmetry of this is worth sitting with. Structured data exists because we did not want machines guessing about the world, and by that measure it succeeded — the whole point was to give them something they could rely on without checking. What the Oracle Poisoning results show is that they took the offer literally. They do not check. They query the layer built to be trusted, receive whatever it says, and reason forward with the fluency we admire in them, reaching a false conclusion by an impeccable route. We taught the web to speak to machines in a language that carries no doubt. It should not surprise us that the machines believe every sentence of it.

Sources

A Senteri Briefing · July 2026 · senteri.com — how machines read the web. This briefing is analysis, not legal advice or a security recommendation for any specific environment. Where an argument joins separately demonstrated findings by inference rather than direct evidence, that boundary is stated in the text.

The Field Guide to Agent-Readiness