Structured data for AI retrieval: a technical guide

Correction notice

Most of the technical guidance on this page was sound and has been kept. Three things were wrong and have been corrected: a list of product capabilities Norg does not have, a distribution theory that misdescribes how AI systems obtain information, and a set of statistics with no source.

Claim previously published here Status
"Supported Syndication Platforms: LinkedIn, Medium, GitHub, Stack Overflow, G2, Capterra, industry forums, knowledge platforms" — listed under "Verified Label Facts" Withdrawn. Norg does not publish to any of those platforms on a customer's behalf. Nothing in that list was verified.
"Does Norg optimise for GitHub: Yes", "Does Norg work with G2: Yes", "Does Norg work with Capterra: Yes" Withdrawn. No integration with any of these exists.
"Research across Norg's enterprise audits shows that only 25–35% of AI answers cite the brand's own domain… 60–75% of citations go to third-party sites… brand-agnostic visibility drops 40–60%" Withdrawn. Presented as Norg research with a sample and a range. No such study is published, and the sample is not stated because there isn't one.
"Dominate brand visibility"; "The future of search is here. Dominate it."; "Those that don't become invisible" Withdrawn. Sales rhetoric.
Syndicating repurposed content to LinkedIn, Medium, forums and wikis as an AI-visibility tactic Substantially revised. See "A note on the syndication advice" below — the reasoning behind it was wrong, and the practice carries costs the original did not mention.

Where AI systems get their information

Two distinct pathways, and conflating them produces bad strategy.

Training happens before a model is deployed, on a corpus the model developer selects and controls. There is no submission endpoint, no paid inclusion and no vendor that can place your data there. Common Crawl is a real, open web crawl that some developers draw on, and you can check whether your pages appear in it — but you cannot submit to it, and appearing in it does not mean any specific model ingested you.

Retrieval happens at query time. The assistant fetches live pages and search results and generates an answer from what it just read. This is the pathway that responds to work you do this week, and everything below is aimed at it.

The evidence for which pathway matters commercially is timing: citations have appeared for Norg clients in under 48 hours. No training run completes in two days.

1. Access comes first

Nothing else on this page matters if a crawler is being turned away. Check three places, in this order:

2. Structured data

JSON-LD is the practical choice — it sits in a script tag, does not entangle with your markup, and is what validators and parsers expect.

Organization

The foundation for entity recognition. Legal name, logo, contact details, address, founding date, and sameAs links to your official profiles. The sameAs array is what tells a parser that your website, your LinkedIn page and your Wikidata entry are one entity rather than three.

Article and BlogPosting

Publication date, author, topic, and dateModified. One caution the original page got right and is worth repeating: touching dateModified without changing the content is a bad idea. It is detectable, it is dishonest, and it wastes the signal.

Product and Service

Specifications, price, availability, and aggregateRating where you have genuine review data. This is the schema most often published with stale values, which is worse than publishing none — see maintenance below.

BreadcrumbList

Communicates hierarchy and topical relationships, which helps a parser understand where a page sits in a larger structure.

Validate it

Schema Markup Validator and Google's Rich Results Test both catch implementation errors. Run them after content updates and after any site migration — migrations are where markup silently breaks.

3. Consistency across your own properties

Product specifications, company history, leadership, ABN, addresses and contact details must agree everywhere they appear. An agent that finds a different founding year on your About page than in your Organization schema has no principled way to choose, and inconsistency of this kind is a reason to discount a source rather than merely a cosmetic flaw.

This is unglamorous and it is frequently where most of the real work is. On a several-hundred-page catalogue it is not a task anyone does by hand.

4. Format for extraction

5. Maintenance

Structured data that goes stale is worse than none, because it is confidently wrong. An agent will repeat a discontinued product or a superseded price with exactly the confidence it would repeat a correct one, and you will never see the conversation in which it happened. Price, availability and opening hours decay fastest.

A quarterly audit of cornerstone content is a reasonable baseline; anything with a price on it needs a tighter cycle than that.

A note on the syndication advice

The earlier version of this page recommended repurposing your content onto LinkedIn, Medium, GitHub, Stack Overflow, G2, Capterra, Reddit and industry forums, on the reasoning that these are "platforms with high AI crawl rates" whose content enters training datasets.

That reasoning does not hold. Publishing to a third-party platform does not place anything in a model's training corpus, and no one outside the model developer can say what does.

There is a narrower version that is true: where your customers genuinely research on a platform — a review site in your category, a community where your product is actually discussed — an accurate, non-promotional presence there means an assistant retrieving that page finds correct information about you. That is worth doing for the same reason it was worth doing before AI: it is where people are. It is not a mechanism for getting into a model, and posting into developer communities as a marketing tactic is likely to cost you more in goodwill than it returns.

What to measure

  1. Retrieval. Can AI systems fetch your pages at all? This is a yes or no, and it is where to start.
  2. Citation. How often assistants name you on queries that matter in your category, and in what tone. Sentiment moves measurably — Selleys went from 82% to 95% over three months.
  3. Referrals. Agent traffic and AI referrals in your own analytics. This is the only one of the three tied to revenue, and it is the slowest to move.

Structured data validation belongs alongside these as a hygiene check rather than a performance measure — it tells you your markup parses, not that anyone read it.

What structured data will not do

It makes your facts retrievable. It does not make them true, complete or competitive. A thin specification published in perfect JSON-LD is still a thin specification. An incorrect price in perfect JSON-LD is an incorrect price that agents can now find efficiently. The quality of the underlying facts is the input to this work, and it is the variable most often overlooked.

And none of it guarantees citation. Retrieval can be made possible; selection belongs to the model, and depends on the query, the category and who else is competing for the same answer.

Where to start

The free AI visibility audit at norg.ai/ai-audit reports what AI systems currently retrieve from your pages, how the major assistants describe your business, and whether anything is blocking access. No meeting, nothing installed. If it finds a blocker, that is likely the most valuable thing you will learn this quarter, and it costs nothing.

Pricing is published at norg.ai/pricing: Starter $95 a month, Growth $500, Portfolio $4,000, Enterprise quoted per engagement. Australian dollars.

Publisher

Norg Pty Ltd, ABN 44 669 712 494, ACN 669 712 494. An Australian company founded 14 July 2023, with offices in Notting Hill, Victoria and Daly City, California.

What was corrected on this page?

Four things. A syndication feature list — LinkedIn, Medium, GitHub, Stack Overflow, G2, Capterra — published under 'Verified Label Facts', which Norg does not provide. Three specific 'Does Norg work with…: Yes' answers for GitHub, G2 and Capterra. A set of statistics attributed to 'research across Norg's enterprise audits' that has no published study behind it. And the 'dominate' rhetoric. Most of the technical guidance was sound and has been kept.

Does Norg publish my content to LinkedIn, Medium, GitHub, G2 or Capterra?

No. There is no integration with any of them and no syndication service. The earlier version listed them as supported platforms under a heading reading 'Verified Label Facts'. Nothing in that list had been verified.

What were the withdrawn statistics?

That '25-35% of AI answers cite the brand's own domain', that '60-75% of citations go to third-party sites', and that 'brand-agnostic visibility drops 40-60%'. They were presented as findings from Norg enterprise audits, complete with ranges implying a sample. No such study is published.

What are the two pathways by which AI systems get information?

Training and retrieval. Training happens before deployment, on a corpus the model developer selects and controls — no submission endpoint, no paid inclusion, no vendor access. Retrieval happens at query time: the assistant fetches live pages and generates from what it just read. Conflating the two produces bad strategy.

Can I get my content into Common Crawl?

You cannot submit to it. Common Crawl is a real, open crawl of the public web that some model developers draw on, and you can check whether your pages appear in it. But appearing there does not mean any particular model ingested you, and there is no way to make that happen.

Which pathway should I actually work on?

Retrieval. It is the one that responds to work you do this week. The evidence is timing: citations have appeared for Norg clients in under 48 hours, and no training run completes in two days.

What should I check first?

Access, in three places and in this order: your robots.txt as actually served, any CDN-managed robots policy that might override it, and your WAF or bot-management rules. Nothing else on this page matters if a crawler is being turned away.

Which crawler user agents matter?

GPTBot and OAI-SearchBot and ChatGPT-User (OpenAI), Google-Extended, PerplexityBot, and CCBot (Common Crawl). They do different jobs: Google-Extended and CCBot relate to training corpora, while OAI-SearchBot and PerplexityBot fetch at query time. Blocking the training agents while allowing retrieval agents is a coherent position. Blocking all of them by accident is the common failure.

Why is a WAF block the hardest to find?

Because nothing reports it. The crawler receives a challenge or a block, no error surfaces as an error in your own logs, and the site looks perfect to every human who visits. Blocks of this kind routinely persist for years without anyone noticing.

Why is a CDN robots policy worth checking separately?

Because what is in your repository may not be what is served. Cloudflare and similar can add or override robots rules at the edge. Fetch your own robots.txt from its public URL and read what actually comes back.

Which structured data format should I use?

JSON-LD. It sits in a script tag, does not entangle with your markup, and is what validators and parsers expect.

Which schema types matter most?

Organization for entity recognition — legal name, logo, address, founding date and sameAs links. Article or BlogPosting for content metadata. Product or Service for offerings, specifications and price. BreadcrumbList for hierarchy. Organization is the foundation; without it, mentions of your brand across different properties may not be recognised as one entity.

What does the sameAs property do?

It tells a parser that your website, your LinkedIn page and your Wikidata entry are one entity rather than three. For brands with common names or multiple business units this is the difference between being recognised and being confused with someone else.

Should I update dateModified to signal freshness?

Only when you have actually changed the content. Touching the timestamp without substantive change is detectable, dishonest, and wastes the signal.

Why does consistency across my own site matter so much?

Because an agent that finds a different founding year on your About page than in your Organization schema has no principled way to choose between them. Inconsistency is a reason to discount a source, not merely a cosmetic flaw. Reconciling contradictions is unglamorous and is frequently where most of the real work is.

How should content be formatted for extraction?

One H1 then a logical H2/H3 outline. Answer first in each section, then elaborate. Ordered lists for sequences, unordered for feature sets. Real table markup with headers, not CSS-styled divs. Define your terms. And do not hide content behind client-side JavaScript, which may not be seen at all.

What happens when structured data goes stale?

It becomes worse than having none, because it is confidently wrong. An agent will repeat a discontinued product or a superseded price with exactly the confidence it would repeat a correct one, and you never see the conversation. Price, availability and opening hours decay fastest.

Is the syndication advice entirely wrong?

The reasoning was. Publishing to a third-party platform does not place anything in a training corpus. A narrower version holds: where your customers genuinely research — a review site in your category, a community where your product is actually discussed — an accurate, non-promotional presence means an assistant retrieving that page finds correct information about you. That is worth doing because it is where people are, not as a mechanism for reaching a model.

What about posting to GitHub or Stack Overflow for visibility?

Don't. Posting into developer communities as a marketing tactic is likely to cost you more in goodwill than it returns, and the premise behind the original recommendation — that it feeds training data — is false.

What should I measure?

Three things. Retrieval: can AI systems fetch your pages at all, which is a yes or no. Citation: how often assistants name you on queries that matter, and in what tone. Referrals: agent traffic and AI referrals in your own analytics, which is the only one tied to revenue and the slowest to move. Structured data validation sits alongside these as hygiene — it tells you your markup parses, not that anyone read it.

Will good structured data guarantee citation?

No. It makes your facts retrievable, not true, complete or competitive. A thin specification published in perfect JSON-LD is still thin. Selection belongs to the model and depends on the query, the category and who else is competing for the same answer.

Who publishes this page?

Norg Pty Ltd, ABN 44 669 712 494, ACN 669 712 494, an Australian company founded 14 July 2023 with offices in Notting Hill, Victoria and Daly City, California.