llms.txt is a proposed convention for a plain-text file at the root of a website that tells a language model, in a few hundred words, what the site is and where its important pages are. It has been enthusiastically over-sold. This article sticks to what the proposal says, what Google says, and the one situation where it is clearly worth doing.
What the proposal says
The format was proposed by Jeremy Howard in September 2024 and is maintained at llmstxt.org. Its stated problem is that “web pages are built for people”: an HTML page wraps its information in navigation, ads and scripts, and turning it back into clean text is difficult and imprecise. The file is meant to be a curated, model-friendly overview, sitting alongside robots.txt and the sitemap rather than replacing either.
The format is Markdown with a fixed shape. In order:
- 01An H1 with the name of the site or project. This is the only required section.
- 02A blockquote with a short summary containing the key information needed to understand the rest.
- 03Zero or more Markdown sections with more detail, in any form except headings.
- 04Zero or more sections delimited by H2 headings, each containing a list of links in the form
[name](url), optionally followed by a colon and a note about that file.
# Acme Coffee Roasters
> Specialty coffee roaster in Leeds. Subscriptions for offices,
> single-origin beans, wholesale supply for cafés. Roasted weekly.
Roast dates are printed on every bag. Office subscriptions can be
paused from the account page at any time.
## Products
- [Office subscription](https://www.example.com/office): 250g to 5kg weekly, pause any time
- [Single-origin beans](https://www.example.com/beans): current lots with roast dates
## Company
- [About](https://www.example.com/about): founded 2016, roastery address, opening hours
What Google says
Google used to answer this by implication. As of its AI optimization guide, updated 10 July 2026, it answers by name. The guide says you do not need “new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn’t use them”, and then goes further:
It’s completely fine if you decide to create and maintain LLMS.txt files… Doing so will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them.
Its summary of tactics to skip lists “creating unnecessary AI text files (like llms.txt)” outright.
So the Google question is settled, in Google’s own words, with the file named. Anyone selling llms.txt as a Google ranking factor is selling something Google has explicitly said it ignores.
| File | Job | Format | Who documents reading it | Google’s position |
|---|---|---|---|---|
| robots.txt | Tells crawlers what they may fetch | Plain text, Robots Exclusion Protocol | Every major crawler, in its own documentation | Honoured; the reference documentation is Google’s own |
| sitemap.xml | Lists the URLs you want indexed, with change dates | XML, sitemaps.org protocol | Search engines, including Google and Bing | Supported and recommended |
| llms.txt | A curated summary of the site for language models | Markdown, llmstxt.org proposal | No assistant vendor documents reading a third-party one | Ignored; neither helps nor harms rankings |
The second question, whether any other assistant reads it, has a less satisfying answer: there is no public evidence that it does. OpenAI, Anthropic and Perplexity all publish an llms.txt on their own documentation sites, which is often cited as proof they consume it. Publishing is not consuming. None of them documents reading a third-party llms.txt at crawl or inference time, and we have not found a vendor statement that they do. If that changes we will update this paragraph and say when.
| Vendor | Publishes an llms.txt for its own documentation | Documents reading yours |
|---|---|---|
| OpenAI | Yes, at developers.openai.com/llms.txt | No |
| Anthropic | Yes, at platform.claude.com/docs/llms.txt | No |
| Perplexity | Yes, at docs.perplexity.ai/llms.txt | No |
| No | No; states that Google Search ignores the file |
Where it is still worth twenty minutes
The proposal’s own examples are documentation sites, and that is where the file earns its keep: an agent asked to use a product or an API can fetch one file instead of crawling forty pages. For a business website the case is narrower but real. When a user-action fetcher such as ChatGPT-User or Perplexity-User visits your site because a person asked about you, a clean text summary of what you sell, where you are and what your prices are is the easiest thing for it to read. The file costs nothing, cannot hurt you, and states your own facts in your own words, which is also the cheapest defence against an assistant repeating a stale address it found elsewhere.
Put differently: it will not get you into an answer. It can make the answer about you more accurate once you are in one.
Writing one that is honest
- Keep the blockquote to the facts a buyer needs: what you sell, where you are, who it is for.
- Link only to pages that exist and say what the note claims. A dead link in a file meant to help machines read you is worse than no file.
- State prices, dates and locations plainly where you publish them elsewhere; leave out anything you would not put on the page.
- Regenerate it when the products or the address change. A stale llms.txt is a stale fact delivered with confidence.
- Mention it with a
<link rel="alternate" type="text/plain">in the page head so a fetcher that looks for it finds it.
SearchLift checks a site’s live /llms.txt against the proposal’s format and generates one from the site’s real pages, products and stated business information; it is one of the lower-drama items on the audit, and that is the right amount of drama for it.
Common questions
- 01Does llms.txt improve Google rankings?
- No. Google’s documentation says no AI text files or special markup are needed for AI Overviews or AI Mode, and the file has no stated role in ranking.
- 02Do ChatGPT or Perplexity read llms.txt?
- Neither company documents llms.txt as a requirement. User-action fetchers may retrieve it if it is present and linked, which is the practical reason to have one that states your facts correctly.
- 03What is llms-full.txt?
- A convention some sites use alongside llms.txt for a longer, complete plain-text version of their content. It is not part of the core proposal’s required format.
- 04Is llms.txt a replacement for robots.txt or a sitemap?
- No. The proposal positions it as a complement: robots.txt governs access, the sitemap lists URLs, and llms.txt is a curated summary for models.