What Is llms.txt? A Real Example, and What the Server Logs Say

In May 2026, Ahrefs went through the server logs of 137,210 domains. About 38,000 of them publish an llms.txt file. 97% of those files got zero requests that month.

That's a file sitting on a server for 31 days without one visitor, bot or human.

This site has an llms.txt too, you can open it, so I'm not writing this from a high horse. Below is what the file is, an example you can copy, and what the logs say about who reads it. If you only want the verdict: for documentation and developer products it can earn its place. For a normal business site, I can't find any evidence that it gets you into a single AI answer.

What Is llms.txt?

llms.txt is a plain Markdown file at the root of your site (yourdomain.com/llms.txt). It lists your most useful pages, each with a short note, so an AI tool can find the right page without wading through navigation, cookie banners and JavaScript.

Jeremy Howard proposed it on September 3, 2024, and the spec went to v2 in August 2026. The part most SEO articles skip is what it's for. Howard's page says the file is "used on demand, when an agent needs information about a topic while assisting a user." It's a reading list for an agent that's already working on something. Nothing in the spec mentions rankings or citations.

Two mix-ups come up all the time. The filename is llms.txt, plural. Plenty of people search for llm.txt, and nothing looks for that name. There's also nowhere to submit the file. It sits on your server until something asks for it.

It has a different job from the two files it gets compared to. robots.txt says where crawlers may go. sitemap.xml lists your URLs. llms.txt picks a handful of pages and explains them, and it can't allow or block anything. If you want to control AI bots, that's still robots.txt, and I went through which AI crawlers to allow or block in a separate post.

An llms.txt File Example You Can Copy

The only required part is one H1 with your site's name. The spec then allows, in this order, a blockquote summary, some free text, and H2 sections holding lists of links. Here's a complete llms.txt file example for a made-up bakery:

# Example Bakery

> Family bakery in Augsburg. Bread, cakes and catering for events.

Prices and opening hours change with the season, so use the linked pages rather than older copies.

## Products

- [Bread](https://example.com/bread): Daily range with ingredients and allergens
- [Catering](https://example.com/catering): Packages, minimum orders and lead times

## Help

- [FAQ](https://example.com/faq): Delivery, payment and cancellations
- [Contact](https://example.com/contact): Phone, email and directions

## Optional

- [About us](https://example.com/about): Company history

The ## Optional heading is a convention: those are links an agent can skip when it needs less context. Swap in your own pages, save the file as llms.txt, upload it to your web root, and check that yourdomain.com/llms.txt opens as plain text.

Real llms.txt examples vary a lot. Cloudflare's is 143 lines and mostly points to a separate llms.txt for each product. Anthropic's runs to 964 lines.

The one on this site has 241 lines across six sections (Services, Company, Blog, FAQ, Work, Legal). The top of it, trimmed:

# Wunderlandmedia

> Digital agency based in Affing near Augsburg, Germany. Custom web development, SEO consulting, and AI consulting.

## Services

- [Web Development](https://wunderlandmedia.com/services/web-development)
- [SEO Consulting](https://wunderlandmedia.com/services/seo-consulting)
- [AI Consulting](https://wunderlandmedia.com/services/ai-consulting)

On WordPress you don't have to write it by hand. Yoast SEO has a built-in generator that refreshes the file weekly and pulls in your most recently updated content, cornerstone pieces first. It won't overwrite a file that's already on your server, so delete yours before you turn it on.

Version 2 of the spec also asks for a Markdown copy of each page at the same URL with .md added. That's where the cost hides. The file itself is small. A second version of every page is a second thing to keep in sync, so I'd skip it unless your CMS generates it for you.

Who Actually Reads llms.txt

The 3% of domains that did get requests in Ahrefs' study received about 22,000 of them in total. These are the groups that matter here:

Who requested llms.txt	Share of requests
SEO audit tools	21.7%
General web crawlers	13.1%
AI agents and agentic infrastructure	10.5%
AI training crawlers	5.3%
AI assistants	2.5%
AI retrieval bots	1.1%

The biggest group reading llms.txt is SEO tools checking whether you have one. Ahrefs' own crawlers alone made 10.6% of the requests. AI retrieval bots, the ones that fetch a page because a user just asked something, made 233 requests across all those sites in a month. Ahrefs calls every figure "a ceiling on actual llms.txt consumption", because a fetch doesn't prove anything read the file.

EZY.ai watched 83 sites with the file live for 12 weeks, from April 27 to July 19, 2026. Same sites, same window:

Crawler	robots.txt fetches	llms.txt fetches
OpenAI (GPTBot family)	3,990	7
Anthropic (ClaudeBot)	3,120	9
PerplexityBot	775	0
Googlebot	5,125	67
Meta-ExternalAgent	172	193

So "no AI bot ever reads it" isn't accurate either. Seven is more than zero. It's also a rounding error next to 3,990. Meta's crawler is the odd one out, asking for llms.txt more often than robots.txt, and I don't know why. That panel leans toward small-business sites whose owners were already looking for an AI visibility tool, so it isn't the whole web.

Getting fetched and getting cited are separate questions. SE Ranking looked at nearly 300,000 domains, 10.13% of them with a file, and concluded that "LLMs.txt doesn't impact how AI systems see or cite your content today." I went into that study in GEO Is the New Snake Oil.

Adoption keeps climbing anyway. Originality.ai counted 36,120 llms.txt files in May 2026 across its 3+ million sites, up from 4,088 in June 2025. People are publishing these files much faster than anything is reading them.

What Google and the AI Companies Say

Google has the clearest statement on record. Its AI optimization guide lists llms.txt under things you can ignore: "It's completely fine if you decide to create and maintain LLMS.txt files (or other similar files) for other services or systems that use these files. Doing so will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." I wrote up the rest of that guide here.

John Mueller compared llms.txt to the keywords meta tag back in April 2025. In June 2026 Mueller allowed one narrow use: "If someone is already on your website, maybe some kind of automated system is helpful." That's close to what the spec says the file is for.

Chrome's Lighthouse audit does look at llms.txt, but it only flags a server error. A missing file gets marked Not Applicable, because "providing the file is optional at the moment."

On the AI side, Ahrefs reports that no major LLM provider has formally adopted llms.txt in its crawler protocol. Anthropic publishes a file for its docs and doesn't say its crawlers use anyone else's. The first version of this post listed Anthropic and Cloudflare as early adopters "actively testing it", which blurs that line. Publishing a file for your own docs tells you nothing about what your bots read.

Should You Add an llms.txt File?

Ahrefs' verdict is that "the cons outweigh the pros right now." I'd split it by what you run. If you have developer docs, an API, or a product people work with through coding tools, add one. It's cheap and it has an audience: "The closest thing to an intended audience in our data is coding agents," says the same study.

If you run a local business or a normal company site, I wouldn't pay for one. No study above shows a benefit, and that hour is better spent on the pages AI tools actually open when they search.

If your plugin generates the file for free, leave it on. Google says it does no harm, and the cost is one toggle. And yes, it's awkward to write all this with a live file on my own site. I can't show you it has done anything.

If a vendor bills you for llms.txt as "AI visibility", ask for the log lines showing something fetched it.

Should you publish one, do two things. Link to it, from your footer or your docs. Ahrefs found that "Agents fetch llms.txt when directed, not speculatively", so an orphan file at your root is a good way to join the 97%. And treat it like code. The same study warns that "potential bad actors are already probing llms.txt for prompt injection." If the file is built from anything users can edit, check what ends up in it.

Check Your Own Logs First

Every number above comes from someone else's sample. Ahrefs' own first piece of advice is "Check your own logs before investing further." If you have raw access logs, one line does it:

grep "llms.txt" access.log | grep -iE "gptbot|oai-searchbot|chatgpt-user|claudebot|perplexitybot"

Zero lines over a month means you're in the 97%, and you've lost nothing by finding out. While you're in there, look at how much these bots hit the rest of your site. If that number scares you, I wrote about stopping AI bots from crawling you to death.

Agents are the part I'd keep watching. llms.txt is about being read. WebMCP is about being used, and that's a different bet.

Where That Leaves llms.txt

I don't know if this stays a rounding error. The spec got a v2 this year and the file count keeps climbing, so plenty of people are building for a reader that may show up. In the logs, that reader made 233 requests in a month. As of November 2026, a text file isn't what stands between you and an AI answer.

I can't promise you citations. What I can do is look at your logs and your pages and tell you what's actually in the way, which is part of my SEO consulting. Let's talk if that sounds right for you.