llms.txt: what it is and what the evidence says

What llms.txt is

llms.txt is a proposal, not a standard. It was put forward by Jeremy Howard in September 2024 on llmstxt.org, which describes it as a proposal to standardise on an /llms.txt file “to provide information to help agents use a website”.

The idea is simple. Websites are built for people: menus, banners, scripts, pop-ups. An AI tool that needs to understand a site quickly has to dig through all of that. An llms.txt file gives it a short, clean summary instead: what the site is, and a list of its most useful pages, each with a one-line note.

The proposal sets out a fixed shape for the file, written in Markdown:

• a heading with the site’s name (the only required part)
• a short summary in a quote block
• optional paragraphs with more detail
• sections of links, each with a name and an optional note
• an “Optional” section for links an AI tool can skip when it needs less

The proposal positions llms.txt as something AI agents read on demand, when they need information about a site, and contrasts it with sitemap.xml and robots.txt, which serve search engines.

What llms.txt is not

Two misunderstandings come up often.

It isn’t a permission file. robots.txt tells crawlers what they may and may not fetch. llms.txt has no such power. It can’t allow or block anything. If you want to control whether OpenAI’s search crawler can read your site, that’s robots.txt (see robots.txt for AI crawlers). OpenAI’s crawler documentation says sites that block OAI-SearchBot in robots.txt are not shown in ChatGPT search answers.

It isn’t a ranking signal. Nothing in the proposal promises that having the file gets you cited more, and the evidence below suggests it doesn’t.

What Google says

Google’s guide to its generative AI features (last updated 10 July 2026) is direct. It says you don’t need to create new machine-readable files, AI text files, markup or Markdown to appear in Google Search, and that Google Search itself doesn’t use them.

It also says it’s completely fine to create and maintain llms.txt files for other services, and that doing so will neither harm nor help your site’s visibility or rankings in Google Search.

What the evidence says

Two large studies looked at whether llms.txt matters in practice.

Ahrefs, server logs, June 2026. Ahrefs analysed the server logs and live traffic of 137,210 domains using its own web analytics product, all of which received traffic in May 2026. Of those, 38,360 published an llms.txt file. Ahrefs reports that 97% of those 38,360 files received zero requests in May 2026: nothing fetched them at all. The roughly 1,100 files that were fetched got about 22,000 requests between them, and the biggest single group of requesters was SEO audit tools, not AI assistants. Ahrefs notes its customers skew more technical than the web at large, so adoption is likely overstated; and because a fetch doesn’t mean a file was read, it calls every figure a ceiling on actual use.

SE Ranking, citation data, late 2025. SE Ranking examined roughly 300,000 domains and found llms.txt on about one in ten of them. As reported by Search Engine Journal, it found no relationship between having llms.txt and how often a domain was cited in answers from prominent AI models. Removing the llms.txt feature from its statistical model actually made the model’s predictions more accurate.

Neither study proves llms.txt can never matter. Both say that, today, you shouldn’t expect it to change whether AI answers cite you.

So should you add one?

Our view: add one if it takes you ten minutes, and then forget about it. Don’t let it take the place of the things that do decide whether ChatGPT can use your site:

1. robots.txt allows OAI-SearchBot. Blocking it removes you from ChatGPT search answers, according to OpenAI.
2. Your pages are in the search index ChatGPT draws on. See Bing Webmaster Tools and IndexNow.
3. Your pages show their text without JavaScript, in the words buyers use.
4. Other websites in your category mention you. See which websites ChatGPT cites.

Where llms.txt may earn its place is outside search: tools that act for a person and fetch a site on demand. In Ahrefs’ data, AI agents and AI training crawlers were among the groups that did request files, with OpenAI’s training crawler, GPTBot, the largest AI fetcher. That’s a reason to keep a file accurate if you publish one, not a route to citations.

If someone sells you llms.txt as the way to get cited, ask them for evidence with a named population, like the two studies above.

How to write one

If you do add one, keep it short and accurate. Place it at the root of your site, as /llms.txt. A minimal file for a fictional brand might look like this:

# Your Brand

> Your Brand makes [what you make] for [who it's for] in India.

## Products
- [Product line one](https://www.yourbrand.example/product-one/): what it is, in one line
- [Product line two](https://www.yourbrand.example/product-two/): what it is, in one line

## Help
- [Frequently asked questions](https://www.yourbrand.example/faq/)
- [Contact](https://www.yourbrand.example/contact/)

Three rules:

• Only list pages that exist and that you’d be happy for anyone to read.
• Describe pages as they are. A file that oversells pages loses trust with any tool that checks.
• Keep it updated when pages move, or remove it. A stale file is worse than none.

How Citationly helps

Our free checker reports whether your site has an llms.txt file, labelled as optional, alongside the checks that matter more: whether OpenAI’s search crawler is allowed in and whether your page’s text shows without JavaScript. A full Citationly report then shows whether ChatGPT’s answers actually name you, and which websites they cite instead.