# What is llms.txt and does it help?

Published 2026-10-13, updated 2026-10-14. By LLMs Bureau (https://llmsbureau.com). Canonical: https://llmsbureau.com/guides/what-is-llms-txt-and-does-it-help

> llms.txt is a free, harmless map of which page answers which question; publish one, but do not expect it to get you cited on its own.

llms.txt is a Markdown file at the root of a website that lists the pages an AI assistant should read and says in one line what each page answers. It is free and harmless. In controlled tests it did not move citations on its own (Ahrefs, May 2026; Google, June 2026); a citation is an assistant naming or linking a page in its answer. We publish one on every site we run and our logs show it being fetched, but what gets a business cited is the pages behind it and the indexes they sit in.

## What is llms.txt?

llms.txt is a file at the root of a domain, so it lives at yourdomain.co.uk/llms.txt beside robots.txt and sitemap.xml. It is written in Markdown, a plain-text format. A language model has a limited reading window, so the file gives it a short map of the site.

The shape comes from a community proposal, not from any assistant vendor: the site's name, one sentence on what the business does, and a list of the pages that matter with a line on what each one answers.

The two older files beside it do different jobs.

| File | What it does | Who acts on it |
| --- | --- | --- |
| robots.txt | Says which paths a crawler may fetch. A rule. | Every well-behaved crawler, search and AI alike |
| sitemap.xml | Lists every URL with the date it last changed. | Google, Bing and the AI index builders |
| llms.txt | A short map: which pages matter and what each answers. Advisory. | Any crawler that chooses to read it; no assistant vendor is bound by it |
| llms-full.txt | The full text of the mapped pages in one document. | Readers that want the whole site in one request |

It is not a blocking tool: a line in llms.txt stops nobody, and only [robots.txt](https://llmsbureau.com/guides/which-ai-crawlers-to-allow-in-robots-txt) does that. Nor is it a cure for a site that only renders in the browser: no AI crawler executes JavaScript (Vercel and MERJ crawler study, 2025; our own logs, 2026), so a map to empty pages helps nobody.

## Does llms.txt get you cited?

Not on its own. In controlled tests, adding the file did not move citations (Ahrefs, May 2026). Google said so too in June 2026. The Ahrefs tests found the same of schema markup, the hidden code that labels a page's content for machines. We publish llms.txt on every site we run and cannot point to one answer the file produced. A crawler fetching a page to answer a real person fetches the page, not the map.

The clearest illustration is [a London investigation firm](https://llmsbureau.com/case-files/a-london-investigation-firm). Visitors sent to it by ChatGPT rose from 14 in August to 43 in September 2026 (its own analytics, 24 July to 7 October 2026). September was the month the firm published llms.txt. It was also the month it published a service page on open-source intelligence and a technology page, and it came after ChatGPT's search crawler, OAI-SearchBot, had taken the site into its index. That service page, published on 6 September, became the second most cited page on the site within a month, with 10 visits that arrived from ChatGPT (the firm's own analytics, 24 July to 7 October 2026). We cannot separate the file from the pages. The page answered a question people were asking; the file listed it.

[A UK address-tracing service we launched on 29 September 2026](https://llmsbureau.com/case-files/address-tracing-service) shows the same from the other side. ChatGPT fetched its pages 24 times while answering someone between 29 September and 13 October 2026; 21 of those came in the four days after OAI-SearchBot took 35 of its pages on 9 October (server logs of our address-tracing site). Its llms.txt was already in place. The crawl was the event, not the file.

## Why we publish one anyway

Because the case against it is "it does not do much", not "it does harm". Set against that:

- **It is free.** One text file, written once.
- **It is harmless.** It cannot hurt a ranking, block a crawler by accident or break a page.
- **It is fetched.** On the day Investigation Bureau launched, 5 October 2026, Google, Bing, ClaudeBot, GPTBot and PerplexityBot each fetched its llms.txt within hours, alongside the home page, robots.txt and the sitemap (investigationbureau.uk server logs). None of them has promised to act on what it read.
- **It shows up the pages that answer nothing.** Each line has to say, in one sentence, which question that page answers; a page that cannot be described that way is one an assistant will struggle to cite, whatever the map says. This is the most useful thing about the file, and it has nothing to do with the file.

A tidy llms.txt over pages that open with a slogan is a map without territory. The London firm's figures show where the cost falls: the visits followed the page that answered, not the file that listed it.

## What a broken llms.txt looks like

The file is missing more often than it is wrong. When it is there, the faults repeat. It arrives as an HTML page with the site's header and footer round it. Its descriptions are page titles rather than answers. It lists every page, which makes it a second sitemap, and the crawler already has one. It points at pages that are empty without JavaScript. Or it dates from launch and misses every page added since.

On their own they cost little; the tests found the file moves nothing on its own. What they signal costs more. A site whose map cannot say what its pages answer usually has pages that cannot say it either, and that is the fault the assistants pass over for a competitor whose page does answer. On a plan the Audit writes the file and the changes behind it, with the exact text for each page in a change order for your developer.

## What moves citations instead

The studies we rely on, and our own logs, point at four things. None is a file.

| Lever | What the evidence says | Source and date |
| --- | --- | --- |
| Presence in the indexes the assistants read | Our address-tracing site's ChatGPT answers followed the OAI-SearchBot crawl, not the file (the figures are in the section above). Copilot reads Bing's index; Google's AI features read Google's. | Server logs of our address-tracing site, 29 September to 13 October 2026 |
| A page that answers the question | The London firm's new service page, not its llms.txt, is what became its second most cited page (above). | A London investigation firm, its own analytics, 24 July to 7 October 2026 |
| Mentions on other sites | 84% of links cited by assistants point at third-party media, not the company's own site. Brand mentions correlate with AI visibility about three times more strongly than backlinks. | Muck Rack, 25 million cited links, May 2026; Ahrefs, 75,000 brands, December 2025 |
| Answered reviews | Brands with 80 or more answered reviews appeared in 75% of AI answers, against 1% with none. | Seer Interactive for Trustpilot, March 2026 (vendor-commissioned) |

Two more findings sit outside anyone's direct control. Reddit is the single most-cited domain on ChatGPT, Google AI Overviews and Perplexity (Ahrefs and Semrush citation studies, 2026). A mention on YouTube is the strongest single correlate with being named by an assistant, a correlation of 0.74 (Ahrefs, December 2025). Our [method](https://llmsbureau.com/how-it-works) puts the index work and the on-page work first, measures which assistant fetched which page, and leaves llms.txt until last.

## How a site fails before the map is read

Most of the sites we read fail before llms.txt comes into it. There is no file at all. robots.txt blocks the crawlers that would read it, so the map is wasted; our guide to [which AI crawlers to allow](https://llmsbureau.com/guides/which-ai-crawlers-to-allow-in-robots-txt) explains who is who. Or a firewall in front of the site turns a crawler away before it reaches any page, while the site looks perfect in the owner's browser.

Each costs the same thing: the questions an assistant could have answered with that site, it answers with someone else's. The [free check](https://llmsbureau.com/check) fetches your robots.txt, llms.txt, sitemap.xml and home page live, as a browser, as ChatGPT-User (what ChatGPT sends when it fetches a page to answer someone) and as GPTBot (the crawler that collects training text), so a block in front of the site shows, and returns a score with findings. The findings name the fault; the text that puts it right is written in the Audit as a change order for your developer. If the file is there and the score is sound, the question moves on to [why the assistants are not naming you](https://llmsbureau.com/guides/why-does-chatgpt-not-mention-my-business).

## Questions people ask

**Is llms.txt the same as robots.txt?**

No. robots.txt tells crawlers which paths they may fetch, and well-behaved crawlers obey it. llms.txt is a suggested reading list that nothing is bound to follow.

**Does Google use llms.txt?**

Not as a source of citations. Google's statements in June 2026 put llms.txt alongside schema markup, the machine-readable labels on a page, as things that do not move citations on their own. Google's AI features read Google's index, which Googlebot builds from your pages and sitemap.

**Should every page on my site be in llms.txt?**

No. A useful file lists the pages that answer a question a customer would ask, with the question or the answer in the description. A file that lists every page is a second sitemap, and the crawler already has one of those.

**Will llms.txt make my JavaScript site visible to ChatGPT?**

No. The crawlers that fetch pages for assistants do not execute JavaScript, so a page that is empty without it stays empty however well llms.txt describes it.

## Sources

- Ahrefs, controlled test of llms.txt and schema markup (link is to the publisher, not the study) (2026-05): https://ahrefs.com
- Google Search Central, statements on llms.txt and schema markup (link is to the publisher, not the statement) (2026-06): https://developers.google.com/search
- Muck Rack, 25 million cited links (link is to the publisher, not the study) (2026-05): https://muckrack.com
- Ahrefs, 75,000 brands: brand mentions against backlinks, and the YouTube correlate (link is to the publisher, not the study) (2025-12): https://ahrefs.com
- Ahrefs and Semrush citation studies: Reddit the most-cited domain (links are to the publishers, not the studies) (2026): https://ahrefs.com
- Seer Interactive for Trustpilot, reviews and AI answers (vendor-commissioned; link is to the publisher, not the study) (2026-03): https://www.seerinteractive.com
- Vercel and MERJ crawler study: no AI crawler executes JavaScript (2025): https://vercel.com/blog/the-rise-of-the-ai-crawler
- LLMs Bureau server logs, our address-tracing site, 29 September to 13 October 2026 (2026-10-13): /case-files/address-tracing-service
- LLMs Bureau server logs, investigationbureau.uk, launch-day crawler check, 5 October 2026 (2026-10-05): /case-files/investigation-bureau
- A London investigation firm, its own analytics (aggregates only) (2026-10-07): /case-files/a-london-investigation-firm

## Related

- [Which AI crawlers should I allow in robots.txt?](https://llmsbureau.com/guides/which-ai-crawlers-to-allow-in-robots-txt)
- [Why does ChatGPT not mention my business?](https://llmsbureau.com/guides/why-does-chatgpt-not-mention-my-business)
- [How long does it take to get cited by an AI assistant?](https://llmsbureau.com/guides/how-long-does-it-take-to-get-cited-by-an-ai-assistant)