What is llms.txt and does it help?
llms.txt is a Markdown file at the root of a website that lists the pages an AI assistant should read and says in one line what each page answers. It is free and harmless. In controlled tests it did not move citations on its own (Ahrefs, May 2026; Google, June 2026); a citation is an assistant naming or linking a page in its answer. We publish one on every site we run and our logs show it being fetched, but what gets a business cited is the pages behind it and the indexes they sit in.
Quotablellms.txt is a free, harmless map of which page answers which question; publish one, but do not expect it to get you cited on its own.
What is llms.txt?
llms.txt is a file at the root of a domain, so it lives at yourdomain.co.uk/llms.txt beside robots.txt and sitemap.xml. It is written in Markdown, a plain-text format. A language model has a limited reading window, so the file gives it a short map of the site.
The shape comes from a community proposal, not from any assistant vendor: the site's name, one sentence on what the business does, and a list of the pages that matter with a line on what each one answers.
The two older files beside it do different jobs.
| File | What it does | Who acts on it |
|---|---|---|
| robots.txt | Says which paths a crawler may fetch. A rule. | Every well-behaved crawler, search and AI alike |
| sitemap.xml | Lists every URL with the date it last changed. | Google, Bing and the AI index builders |
| llms.txt | A short map: which pages matter and what each answers. Advisory. | Any crawler that chooses to read it; no assistant vendor is bound by it |
| llms-full.txt | The full text of the mapped pages in one document. | Readers that want the whole site in one request |
It is not a blocking tool: a line in llms.txt stops nobody, and only robots.txt does that. Nor is it a cure for a site that only renders in the browser: no AI crawler executes JavaScript (Vercel and MERJ crawler study, 2025; our own logs, 2026), so a map to empty pages helps nobody.
Does llms.txt get you cited?
Not on its own. In controlled tests, adding the file did not move citations (Ahrefs, May 2026). Google said so too in June 2026. The Ahrefs tests found the same of schema markup, the hidden code that labels a page's content for machines. We publish llms.txt on every site we run and cannot point to one answer the file produced. A crawler fetching a page to answer a real person fetches the page, not the map.
The clearest illustration is a London investigation firm. Visitors sent to it by ChatGPT rose from 14 in August to 43 in September 2026 (its own analytics, 24 July to 7 October 2026). September was the month the firm published llms.txt. It was also the month it published a service page on open-source intelligence and a technology page, and it came after ChatGPT's search crawler, OAI-SearchBot, had taken the site into its index. That service page, published on 6 September, became the second most cited page on the site within a month, with 10 visits that arrived from ChatGPT (the firm's own analytics, 24 July to 7 October 2026). We cannot separate the file from the pages. The page answered a question people were asking; the file listed it.
A UK address-tracing service we launched on 29 September 2026 shows the same from the other side. ChatGPT fetched its pages 24 times while answering someone between 29 September and 13 October 2026; 21 of those came in the four days after OAI-SearchBot took 35 of its pages on 9 October (server logs of our address-tracing site). Its llms.txt was already in place. The crawl was the event, not the file.
Why we publish one anyway
Because the case against it is "it does not do much", not "it does harm". Set against that:
- It is free. One text file, written once.
- It is harmless. It cannot hurt a ranking, block a crawler by accident or break a page.
- It is fetched. On the day Investigation Bureau launched, 5 October 2026, Google, Bing, ClaudeBot, GPTBot and PerplexityBot each fetched its llms.txt within hours, alongside the home page, robots.txt and the sitemap (investigationbureau.uk server logs). None of them has promised to act on what it read.
- It shows up the pages that answer nothing. Each line has to say, in one sentence, which question that page answers; a page that cannot be described that way is one an assistant will struggle to cite, whatever the map says. This is the most useful thing about the file, and it has nothing to do with the file.
A tidy llms.txt over pages that open with a slogan is a map without territory. The London firm's figures show where the cost falls: the visits followed the page that answered, not the file that listed it.
What a broken llms.txt looks like
The file is missing more often than it is wrong. When it is there, the faults repeat. It arrives as an HTML page with the site's header and footer round it. Its descriptions are page titles rather than answers. It lists every page, which makes it a second sitemap, and the crawler already has one. It points at pages that are empty without JavaScript. Or it dates from launch and misses every page added since.
On their own they cost little; the tests found the file moves nothing on its own. What they signal costs more. A site whose map cannot say what its pages answer usually has pages that cannot say it either, and that is the fault the assistants pass over for a competitor whose page does answer. On a plan the Audit writes the file and the changes behind it, with the exact text for each page in a change order for your developer.
What moves citations instead
The studies we rely on, and our own logs, point at four things. None is a file.
| Lever | What the evidence says | Source and date |
|---|---|---|
| Presence in the indexes the assistants read | Our address-tracing site's ChatGPT answers followed the OAI-SearchBot crawl, not the file (the figures are in the section above). Copilot reads Bing's index; Google's AI features read Google's. | Server logs of our address-tracing site, 29 September to 13 October 2026 |
| A page that answers the question | The London firm's new service page, not its llms.txt, is what became its second most cited page (above). | A London investigation firm, its own analytics, 24 July to 7 October 2026 |
| Mentions on other sites | 84% of links cited by assistants point at third-party media, not the company's own site. Brand mentions correlate with AI visibility about three times more strongly than backlinks. | Muck Rack, 25 million cited links, May 2026; Ahrefs, 75,000 brands, December 2025 |
| Answered reviews | Brands with 80 or more answered reviews appeared in 75% of AI answers, against 1% with none. | Seer Interactive for Trustpilot, March 2026 (vendor-commissioned) |
Two more findings sit outside anyone's direct control. Reddit is the single most-cited domain on ChatGPT, Google AI Overviews and Perplexity (Ahrefs and Semrush citation studies, 2026). A mention on YouTube is the strongest single correlate with being named by an assistant, a correlation of 0.74 (Ahrefs, December 2025). Our method puts the index work and the on-page work first, measures which assistant fetched which page, and leaves llms.txt until last.
How a site fails before the map is read
Most of the sites we read fail before llms.txt comes into it. There is no file at all. robots.txt blocks the crawlers that would read it, so the map is wasted; our guide to which AI crawlers to allow explains who is who. Or a firewall in front of the site turns a crawler away before it reaches any page, while the site looks perfect in the owner's browser.
Each costs the same thing: the questions an assistant could have answered with that site, it answers with someone else's. The free check fetches your robots.txt, llms.txt, sitemap.xml and home page live, as a browser, as ChatGPT-User (what ChatGPT sends when it fetches a page to answer someone) and as GPTBot (the crawler that collects training text), so a block in front of the site shows, and returns a score with findings. The findings name the fault; the text that puts it right is written in the Audit as a change order for your developer. If the file is there and the score is sound, the question moves on to why the assistants are not naming you.