Do You Need an llms.txt File? And 6 Other AI SEO Questions Answered (With Evidence)

AI search and GEO/AEO (take your pick) have produced a remarkable amount of advice in a very short time. Most of it is confident, much of it is untested, and some of it appears to have been written by the platforms themselves. 

We run a lot of AI SEO projects for clients and we often speak about some common technical questions that actually affect AI search visibility. So we thought we would put good karma into the world and answer some of them for you. 

We’ve put together a few questions on everything from retrieval, crawlers, rendering, feeds, and measurement. Each is answered using the various platforms’ own documentation or fact-checked research. We’ll also flag a couple of handy little notes where our opinion is given or the facts might change. 

1. Are llms.txt files worth implementing?

Short answer: not for search visibility.

Google’s guide on optimising for AI features is explicit. Google Search doesn’t use llms.txt or other special machine-readable files, Markdown versions, or AI-specific markup, including in their AI Overviews and AI Mode.

Creating an llms.text file neither helps nor harms your Google visibility. Google may still crawl and index the file like any other, but that doesn’t mean it gives it special treatment.

Where it does show up is developer documentation. OpenAI’s own developer docs point to an llms.txt index and offer Markdown versions of each page. That makes sense for technical documents that coding tools might read. It doesn’t mean ChatGPT search uses the file to decide which sites to cite in AI search.

In fact, studies by Ahrefs and SE Rankings looked at their impact, and the latter indicated that llms.txt files did not impact citations, while the former found that a high percentage of these .txt files were not even requested. 

If you do implement it, generate it from the same source as the site so it can’t drift out of date. Never put information in it that isn’t on the crawlable pages. And don’t let it take priority over fixing raw-HTML rendering, which affects most AI crawlers. 

Until an answer engine confirms it reads the file and factors it in, llms.txt is a set of directions written for a visitor who hasn’t booked the trip yet and may never arrive. 

2. How does Google’s AI Overviews decide what gets cited? 

This is one of the first questions we get asked and one with a startling amount of complexity. We’ll not touch too much on authority building. Instead, we’ll look at onsite content and how it decides what gets cited.

It starts with two things. Retrieval and query fan-out.

Gemini/AI Overviews and the like break the prompt into several related searches, retrieve pages for each, and build its answer from sections of content on those pages it selects as most relevant. So you can be cited for a sub-question you rank for, even when you don’t rank for the original prompt.

Google names both retrieval and query fan-out in its generative AI guide in the following way:

• Retrieval-augmented generation (RAG). Google calls this grounding. Its core ranking systems retrieve relevant pages from the Search index, the model reviews them, and the answer links to the pages that support it.
• Query fan-out. The model runs a set of related queries at the same time to gather more results. Google’s example is a question about a lawn full of weeds, which fans out into searches about herbicides, chemical-free removal and prevention. It’s rather like sending someone out for milk and having them come back with a short history of the cow breed it came from and insights into the wider dairy industry… which turns out to be useful.

This query fan-out helps explain why citations often don’t match rankings for the original prompt. 

Ahrefs found that across 15,000 prompts, only 12% of URLs cited by ChatGPT, Gemini and Copilot ranked in Google’s top 10 for that prompt. For Google’s AI Overviews, which are tied more closely to its traditional search index, Ahrefs research found that 76% of citations came from pages in the top 10 of its traditional organic rankings. 

So what do you need to do? 

A good starting point is looking at a buyer’s likely prompts, then what it could fan out into. This could be (we give these as examples only) comparisons, pricing breakdowns, suitability to their situation, and objections. 

Check where you rank for those, not just the main keywords and prompts. Then make sure you answer them clearly. A bit of advice, don’t try to build a separate page for every variation. Make important pages good and useful. Create clear structure for those other pieces of content when it makes sense to. Google is giving its most disapproving frown at content aimed at ranking for specific prompts at scale and even has a scaled content abuse policy (that someone actually had to write). 

Note: Linking query fan-out to the low citation overlap is our opinion at the moment of writing, as it’s relevant to this article. Ahrefs offers several possible explanations and doesn’t attribute the gap to fan-out alone. 

3. Does structured data help with AI search, and where should it be rendered?

On Google, the position is settled. Structured data isn’t required for generative AI search, and there’s no special schema.org type to add. Google recommends keeping it as part of normal SEO because it drives rich result eligibility. 

For ecommerce and local, the same guide points to Merchant Center feeds (ecommerce) and Google Business Profiles as ways to show up in AI responses. So for your ecommerce product data, the feed likely matters more for appearing in Google’s AI shopping answers than any extra markup on page. 

That said, some useful schema markup options can help provide better clarity for your entity, such as Organization schema type and sameAs which is a very handy property inside organisational schema. These provide clarity as to who your organisation is and the sameAs provides connections to other places your organisation exists. Which is good for entity optimisation. 

It would be best practice in technical SEO terms to review the schema and its implementation on your site. 

Perhaps the more important and overlooked question is where the markup is generated.

Google can process structured data generated by JavaScript. Some AI search crawlers don’t render JavaScript (we’ll talk about that in this article, don’t worry). For the technical among you, JSON already in the initial response may (emphasis on may) still be read, so JSON-LD in the server-rendered HTML is available to every crawler. Schema injected through a tag manager, though, is to a non-rendering crawler what a label is on the inside of a sealed box.

4. Which AI crawlers render JavaScript?

Google’s can. Others don’t. So if something matters, it needs to be in the page’s HTML before any scripts run.

When Vercel and MERJ studied real crawler traffic, none of the big AI crawlers outside Google ran JavaScript. That includes OpenAI’s three crawlers, Anthropic’s ClaudeBot, PerplexityBot, Meta’s crawler, Bytespider and Common Crawl. ChatGPT’s. Claude’s crawlers did download JavaScript files, which made up 11.50% and 23.84% of their requests – they just never ran them. In other words, they download your JavaScript, look at it thoughtfully and leave without running any of it. A bit like scrolling through oodles of Netflix films without actually watching one. 

The exceptions are Gemini, which uses Googlebot’s set-up and renders pages fully, and Applebot. Google confirms it can handle JavaScript as long as it isn’t blocked from doing so.

One useful detail is that data that’s already in the first response, such as embedded JSON, may still get read. So the question isn’t really ‘is it JavaScript?’, it’s ‘is it there when the page first loads?’, which is why good technical SEO is so important. 

5. How do I get my products on ChatGPT and Google’s AI Overviews shopping results? 

The short answer is that it’s largely through structured product data from shopping feeds and third parties. So your product feed accuracy and consistency across platforms where your feed is served matter as much as anything on the product page.

Why do we say across everywhere your feed lives? Well, if your feed says one price and a comparison site from last summer’s sales says another, ChatGPT will pick one with the confidence of someone that has never had to process a refund.

A few other things matter for ChatGPT such as whether you are the merchant or the manufacturer, both at merchant and product level. If you have a Shopify website then your feed automatically is in the frame. Some other platforms you may have to apply to send a direct product feed to ChatGPT.

On Google’s generative AI guide they indicate Merchant Center feeds as the route into AI responses. So audit the two together. Check that the price, availability, GTINs and titles match across the site, the Merchant Center feed, any OpenAI feed. Then all the main marketplaces and comparison sites you operate on. 

Note: ChatGPT changes all the time so while this is true as of writing, it could be poppycock by next year. Additionally we think reviews at merchant level and product level may also influence appearance, but that’s just our humble opinion. There is no documented evidence of this outside of Converted (that we’ve looked for, or found). 

6. What if AI says something wrong about my brand?

First work out where the error comes from, because each source has a different fix. There’s no direct edit for any of them.

Errors are common. In the EBU and BBC study, 45% of answers about news from ChatGPT, Copilot, Gemini and Perplexity had at least one significant issue. One in five had major accuracy problems, including invented details and out-of-date information. That study was about news, not brands, but the possibility for failure can carry over.

Wrong facts about a business are rarely made up from nothing. Usually they’ve been carefully preserved somewhere, like a fly in amber. After all, AI finds a way. 

Maybe it’s in a directory nobody has logged into since 2017. 

First you need to diagnose where the issue originates. For example:

Retrieval – It’s cited information from a page that has an inaccurate fact or statement about your business. To fix this you need to amend the source or provide a more accurate source with more trust and authority to outrank it and become a primary source for that information. 

LLMs own training memory – If there is no citation indicating where this information came from, then perhaps (and we emphasise that this is a maybe) it is due to some ghost in the machine. An error in the AI model’s learning or its memory. Model memory is a real factor. OpenAI says ChatGPT’s own response, generated before it considers new search results, is one of the inputs to product answers. To properly diagnose this, first check all other AI search platforms and make sure it is isolated to one. If that’s the case, then sorry to be the bearers of bad news, but this is a slower fix. To resolve this you need to build consistent, corroborated statements about the facts (the truth, not the error) across many sources over time, which is a digital PR piece.

Entity Confusion – This can happen and it’s where it confuses you with another brand or business. Proper schema, consistent business details and entity optimisation can help resolve this. 

7. If the answers AI search gives change all the time, then how do I even measure visibility? 

The first part, which is prompt tracking, is easiest to answer. You treat it as sampling. Run each prompt many times, report the share of answers that mention you with a margin of error, and ignore rank position. We have a nifty piece of kit that allows us to do this at scale and more regularly, which gives us more accuracy and a larger data set to work with. 

Why did we say ignore rank? The case against rank comes from a SparkToro and Gumshoe.ai study. Some 600 volunteers ran 12 prompts through ChatGPT, Claude and Google’s AI nearly 3,000 times. The chance of getting the same list of brands twice was under 1 in 100. The chance of the same list in the same order was about 1 in 1,000. But the set of brands considered stayed fairly stable based on the specific intent, which makes mention share a workable metric. Reporting a rank in an answer that reshuffles itself every time is like measuring the width of fog. 

However, to properly measure the impact of AI SEO efforts you need to couple this with analytical data such as ensuring you set up an AI custom channel within GA4. Monitoring the GSC ‘generative AI report’ and the data it gives you. Brand and direct traffic changes. These all should provide insights when properly grouped together. 

In Closing

What to do next

1. Skip llms.txt until everything below is done.
2. Map the sub-questions your buyers’ prompts are likely to fan out into, and make sure your key pages answer them.
3. Move your JSON-LD into the server-rendered HTML, and check your Organization markup and sameAs links are consistent.
4. View the raw HTML of your key pages with JavaScript switched off. Anything missing is invisible to most AI crawlers.
5. Make price, stock and product titles match across your site, Merchant Center and every marketplace or comparison site you appear on.
6. Ask the main AI platforms about your brand, and trace any wrong answers back to their source before trying to fix them.
7. Track mention share over many prompt runs, and set up an AI channel in GA4 alongside Search Console’s Generative AI performance report.

We know most of the people looking at this are likely to be other industry professionals, and if it helps any business out there improve their AI SEO, then we have done our job. 

If you’d like help getting more customers through AI search, get in touch. We have plenty more where these came from, and stopped being invited to parties some time ago because of it.