Blog

Two kinds of AI search: one parses your markup, one may never read it

Diagram "Two kinds of AI search" showing the difference between Path A (parsed by Google/Bing where JSON-LD is a separate signal) and Path B (flattened text for RAG assistants where text is truncated at ~20,000 characters, potentially cutting off markup).
Two kinds of AI search: one extracts your structured data before reading the page, one may never reach it.

Most advice about AI visibility assumes there is one kind of AI search. There are two, and they treat your structured data in opposite ways.

Google and Bing extract Schema.org markup with a dedicated parser at crawl time. The JSON-LD block never competes with your page copy. It is pulled out, validated, and fed into an entity layer.

The other kind reads your page as text. Retrieval systems convert page content into embeddings and match those against the question a user asked. In that pipeline your markup is not a separate signal — it is one more passage of text, and {"@type":"Organization","name":...} does not read like an answer to "what does this company actually do".

If you have spent the last year adding markup and expecting AI assistants to notice, that distinction is the one that matters.

What the evidence says

The most direct test published so far comes from a March 2026 experiment by WordLift. Every entity in their corpus was rendered in three formats and queried under two retrieval modes.

Plain HTML — the page with every JSON-LD block stripped out.

HTML plus JSON-LD — the same page with full Schema.org markup embedded, exactly as a well-implemented site would publish it.

An entity page — a page built for machine consumption: a readable summary generated from the structured data, the JSON-LD block, visible links to related entities, schema type breadcrumbs, and instructions addressed to AI agents in the style of llms.txt.

Answers were scored for accuracy and completeness on a five-point scale by a separate judge model.

Adding JSON-LD to a plain page moved accuracy from 3.62 to 3.89 — an effect the authors describe as statistically significant and negligible in size, with a Cohen's d of 0.18. Measurable if you run the statistics, invisible if you are the person reading the answer.

The entity page scored 4.69, which the authors report as a 29.6% improvement over plain HTML. The two formats contain identical structured data. One buries it in a script tag; the other says the same facts out loud, in text, with the links exposed.

Read that percentage with caution. The headline count is 2,439 evaluations, but that is 349 queries run under seven conditions — the underlying sample is four websites. One of the four, an e-commerce catalogue, showed almost no gain at all. Another, which contributed the largest gain, is the authors' own blog. The direction of the effect held across the sample; the size of it rests on very little.

Why markup underperforms in text pipelines

Two things are going on, and only one of them is contentious.

The uncontentious one: search engines never treat your markup as prose. Google's parser lifts the JSON-LD block out before the page becomes text at all. Retrieval systems have no such step. Your markup competes for semantic similarity as JSON syntax rather than as sentences, and syntax matches questions badly.

The contentious one is truncation. In the WordLift setup, documents were embedded whole and cut at roughly 20,000 characters. In their corpus, 88% of pages carrying JSON-LD exceeded that limit, and the median starting position of the block was character 18,510 — just past the boundary. Structured data usually sits at the bottom of the HTML, so it was the first thing discarded.

That effect is real but specific. Most production retrieval pipelines split pages into passages of a few hundred tokens and embed each one, which indexes the whole document and removes the truncation problem entirely. If someone tells you your markup is invisible because it sits too late in the file, ask them how their pipeline chunks.

The first problem does not go away with chunking. A chunk of JSON is still a chunk of JSON.

Where the effect did not appear

One domain in the study showed no benefit. The e-commerce catalogue scored 4.92 on plain HTML and 4.99 on the entity page. Product pages already state the price, the availability, and the specification in visible text. There was nothing left for a better format to add.

The largest gains came from the travel guide and the editorial site, where the important facts — coordinates, cuisine type, opening hours, topical relationships — existed in the knowledge graph but never appeared as readable text.

That pattern is the practical rule, and it is more useful than the headline percentage. Restructuring helps in proportion to how much a page currently fails to say plainly. If your website already states your services, your pricing, your service area, and your hours in text a stranger could read, you have less to gain here than the study suggests. Most business websites do not. They state a positioning line and three testimonials.

We failed this test on our own page

We build structured profile pages, so we ran the measurement on ours before writing any of this.

Our profile page carried a single JSON-LD block starting at character 33,235, sitting behind 48,000 characters of inline CSS in the document head. The whole page was 88,043 characters, of which fewer than 7,000 were visible text. In a pipeline that embeds documents whole, our structured data would have been discarded — on a product whose entire purpose is machine readability.

The fix took one template change: move the JSON-LD directly after the title, and move the CSS to an external file. The block now starts at character 316. The page dropped to 40,293 characters. No visible content changed.

You can run the same check on your own site in about thirty seconds. Open the page source, search for application/ld+json, and look at how far down the file you had to scroll to find it. Then ask whether the facts inside that block also appear in the text a human reads.

What this does not prove

The authors sell knowledge graph services, so the result aligns with their commercial interest. Ground-truth answers were derived from the same knowledge graph used to build the enhanced pages — a circularity the authors acknowledge. One retrieval system was tested. Generation and judging used models from the same family. The domain sample was uneven, as described above.

The entity page was tested as a bundle: the readable summary, the visible links, the breadcrumbs, and the agent instructions rose or fell together. If you want to know whether llms.txt-style instructions matter on their own, this study cannot tell you, and neither can anyone else yet.

And a disclosure of our own: we build exactly this kind of page, so we are not neutral either. The paper, the dataset, and the templates are public. Check them rather than taking our summary for it.

What to do with this

Keep the markup. It still works for the systems that parse it, and they still drive most search traffic. Nothing here argues for removing anything.

Stop treating markup as the finish line. If your structured data says something your visible page does not, the systems that read pages as text may never learn it.

Say the facts in text. Services, pricing model, location, service area, hours, who you serve and who you do not. Every fact that exists only inside a script tag is a fact you are betting on one kind of crawler.

Give agents somewhere to land. A page whose whole job is describing one entity — in text, with the structured data alongside it and the related links visible — is the format that measured well. That is what an Aiprobase profile publishes: the same facts, in every format, on a page that says them out loud rather than hiding them in the head of a document.

The web spent fifteen years learning to write for people and hint to machines. The systems answering your customers' questions now do not read the hints.

References

Volpini, A., Raad, E., Gamba, B., Riccitelli, D. (WordLift):

- "Structured Linked Data as a Memory Layer for Agent-Orchestrated Retrieval", arXiv:2603.10700, 11 March 2026

- dataset and templates

Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., Deshpande, A.:

"GEO: Generative Engine Optimization", arXiv:2311.09735

HTTP Archive, "Structured data":

Web Almanac 2024

llms.txt proposal:
https://llmstxt.org/

FAQ

Does this mean schema markup is a waste of time?
No. Google and Bing extract JSON-LD with a dedicated parser, so markup keeps doing its job for search engines, rich results, and entity understanding. The finding is narrower: in systems that read a page as text, embedded markup contributed a negligible improvement to answer accuracy, while presenting the same facts visibly improved it by around 30% in one experiment across four sites — treat the size of that number with caution.
Which AI systems read pages as text?
Retrieval-augmented assistants and agentic search tools built on general-purpose vector search. Google and Bing sit in the other category. Because ChatGPT Search draws on Bing's index, one AI product can behave differently depending on which part of its pipeline handles your page.
Does the position of my JSON-LD in the HTML matter?
Only in pipelines that embed a whole document at once and truncate what does not fit. Systems that split pages into passages index the whole file, so position is irrelevant to them. Moving the block early is still worth doing — it costs one template change and helps parsers and page speed — but it is not the main event.
If my website already describes my services clearly, do I need anything else?
Possibly not for this specific effect. In the study's e-commerce domain, where pages already stated the facts in visible text, the entity format added almost nothing. The gains appeared where key facts existed only in structured data or in a database behind the page.
Share this post