In June 2026, Google added a note to its AI optimization guide confirming that Search ignores llms.txt. Two months later, the spec behind the file shipped a version 2. Both of those things happened, and neither one settles the question people actually ask, which is whether the file is worth twenty minutes of their afternoon.
So I stopped reading opinions and went to look. This morning I probed 30 sites for the file, using the same validation logic our GEO checker runs, and read the v2 spec against what those sites actually serve. The picture is clearer than the argument suggests. Disclosure before we start: Hoverify is our product, and its GEO checker is one of the tools that tests for this file.
What llms.txt is
llms.txt is a Markdown file at the root of your domain, at /llms.txt, that lists the pages on your site worth reading and says in one line what each one covers. Jeremy Howard of Answer.AI proposed it in September 2024. The reasoning was that a language model working with a limited context window cannot afford to crawl your navigation, your cookie banner and your footer to find three paragraphs of documentation, so you hand it a map instead.
The format is deliberately small. Per the spec, a file has an H1 with the name of the project, which is the only required part, then a blockquote holding a short summary, then optional prose, then H2 sections containing lists of Markdown links with a note after each one. That is the whole standard. Ours looks like this:
# Hoverify
> Hoverify is a browser extension for Chrome, Edge, Brave and Firefox that puts
> nine web development tools in one place.
## Extension tools
- [CSS Inspector](https://tryhoverify.com/inspector/): Hover any element to see
its HTML and source CSS, then edit or copy the styles without opening DevTools.
- [Screenshot](https://tryhoverify.com/screenshot/): Captures the full page, the
viewport, an area or one element, with a built-in editor.
There is a companion file, /llms-full.txt, which is not a map but the whole corpus: every page concatenated into one Markdown document.
What the v2 llms.txt standard changed
Howard published a revision on 10 August 2026, the first since 2024. The structure survived intact, so a file you wrote last year is still valid. What v2 adds is a way for an agent to find the Markdown version of a page once it has followed a link out of your llms.txt.
That gap was real. Your llms.txt points at /docs/getting-started/, the agent fetches it and gets a pile of HTML, and the clean Markdown twin sitting at /docs/getting-started.md stays invisible because nothing told anybody it was there. The v2 answer is a standard link relation:
<link rel="alternate" type="text/markdown" href="/docs/getting-started.md">
The same thing works as an HTTP Link response header, which matters because a header can be set at the CDN and applies to files that have no HTML to put a tag in.
Adoption of that part is thin so far. I checked 9 documentation sites that already publish an llms.txt, and only Cloudflare and Vercel advertise a Markdown alternate the way v2 asks. Four more sites, including Anthropic’s docs, Next.js and Stripe, do serve a .md version of each page while telling nobody about it, which is precisely the situation v2 exists to fix. None of the 9 used the Link header form. Our own site does neither yet.
Who actually reads llms.txt
This is where most writing on the subject goes wrong, because it collapses two different questions. Publishing an llms.txt and consuming one are separate acts, and the companies doing the first are not thereby doing the second.
Here is what 30 sites served this morning:
| Site | /llms.txt | /llms-full.txt |
|---|---|---|
| docs.anthropic.com | Yes, 72 KB | Yes, 41.5 MB |
| Stripe | Yes, 65 KB | No |
| GitHub | Yes, 29 KB | Yes |
| Cloudflare | Yes, 17 KB | Yes, 157 KB |
| Cursor | Yes, 19 KB | No |
| Next.js | Yes, 12 KB | No |
| Shopify | Yes, 13 KB | Yes |
| Zapier | Yes, 13 KB | Yes |
| Notion, Netlify, Vercel, Supabase, Svelte, Mintlify | Yes | Mixed |
| anthropic.com, openai.com, developers.google.com | No | No |
| MDN, Wikipedia, Stack Overflow, BBC, NYT, Figma, Tailwind, Astro | No | No |
15 of the 30 served a valid file. The split is not random: it is developer documentation on one side and everything else on the other. Every site above that publishes one is selling something to developers or hosting their docs. The sources that get cited most heavily in AI answers, which is to say MDN, Wikipedia and Stack Overflow, publish nothing.
Two details from that run are worth keeping. Anthropic’s llms-full.txt is 41.5 MB, which is larger than the context window of the models it is meant to feed, so it is an archive to be fetched in pieces rather than a document anyone reads whole. And anthropic.com returns a 404 while docs.anthropic.com serves a file, which is the pattern almost everywhere: this is a docs-site convention that has not spread to marketing sites.
On the consumption side the evidence is thinner than the enthusiasm. Google’s John Mueller, posting on Reddit in April 2025, said that no AI service had claimed to use the file and that server logs showed nobody fetching it. He compared it to the keywords meta tag: a description of a site written by the site’s owner, unverified, when the crawler could read the site instead. That comparison has not aged badly. No major AI vendor has since documented fetching llms.txt from third-party sites at crawl time.
What did happen is adjacent and easy to confuse with support. Anthropic recommends the file in its guidance on writing for agents, OpenAI uses it in its own developer tooling, and Chrome’s Lighthouse added an agentic browsing audit that checks whether you have one. That is a convention gathering weight among people who build agents. It is not yet evidence that an agent fetched your file today.
What Google says about llms.txt
Google’s position is now written down rather than inferred from conference answers. The AI optimization guide states that Search does not require machine-readable files, AI text files or Markdown to appear in Search, including its generative features, because Search does not use them. On the file specifically, the guidance is that maintaining one for other services is fine and will “neither harm nor help your site’s visibility or rankings in Google Search, since Google Search ignores them.”
That is about as unambiguous as Google gets, and it cuts both ways. There is no ranking benefit. There is also no penalty, which disposes of the worry that shipping one marks you as somebody chasing tricks.
The best evidence on whether the file earns citations comes from SE Ranking, which looked at 300,000 domains in November 2025. About 10% had the file. They found no relationship between having one and being cited in AI answers, and when they dropped llms.txt from their prediction model, the model got more accurate. The file was adding noise. Adoption was also flat across traffic levels, with the largest sites slightly less likely to bother than mid-sized ones.
llms.txt examples worth copying
Two of the files I fetched are worth reading before you write yours, for opposite reasons.
Cloudflare’s is 17 KB and organized by product, with a one-line description under every link that says what the page answers rather than what it is called. Svelte’s is 1.7 KB and does almost nothing except point at four other files. Both are good, because both match the size of what they are describing. The failure mode in between is a 60 KB file that lists every URL on the site in sitemap order, which recreates the problem the format was invented to solve.
The pattern worth stealing from all of them: the description after each link is written for someone deciding whether to open it. “Captures the full page, the viewport, an area or one element” tells an agent something. “Screenshot tool” does not.
How to write one
Twenty minutes by hand is the right budget for most sites.
- Write the H1 as your product or site name, nothing more.
- Write the blockquote summary as the single paragraph you would give someone who has never heard of you. This is the part a model is most likely to repeat back, so it is worth more care than the rest of the file combined.
- Group your genuinely useful pages under H2 headings that mean something, such as Documentation, Guides or API reference. Skip your privacy policy and your pricing FAQ.
- Write one clause after each link describing what the page answers.
- Use absolute URLs. Relative paths work in the spec, but agents handle them inconsistently.
- If your CMS or docs host can emit Markdown versions of pages, advertise them with the v2
rel="alternate"link.
Keep it curated, above all. A file that lists everything is a sitemap, and there is already a format for that.
How to validate llms.txt
Fetching https://yourdomain.com/llms.txt and reading what comes back gets you most of the way there. Check that it parses the way the spec describes, too: an H1 first, one blockquote, then H2 sections of Markdown links. A file that opens with prose before the H1, or lists bare URLs instead of links, will not be read the way you expect.
The check people miss is whether the response is a file at all. Single-page applications and some hosting setups answer every unmatched path with the app shell, so /llms.txt comes back as HTTP 200 with a chunk of HTML, and an llms.txt validator that trusts the status code will report a file you never wrote. Reddit did this in my run this morning. Ours refuses to count a response whose content type is text/html or whose body starts with <, which is the whole fix.
Our GEO checker runs all three from a panel on the page you are already looking at, along with the same probes for llms-full.txt, your sitemap, robots.txt rules naming GPTBot, ClaudeBot and PerplexityBot, and the Markdown alternate that v2 introduced. It scores 16 categories in total and treats a missing llms.txt as a warning rather than a failure, which is the honest weight to give it.
So should you ship one?
Yes, if you run documentation or a developer-facing product, and no, if you are hoping it will move rankings.
The case for writing one is that it costs twenty minutes, it cannot hurt you, the convention is being adopted by exactly the companies building the agents, and the summary paragraph is a useful exercise regardless of who reads it. The case against treating it as a priority is that the measured effect on citations is zero, Google ignores it and the sites AI answers cite most often have never had one.
Put it after the work that has evidence behind it: server-side rendering so your text exists before JavaScript runs, headings that describe what is under them and prose direct enough that a model can lift a sentence out and have it still make sense. An llms.txt on a site whose content arrives only after hydration is a map to a building with no doors.
