llms.txt is a plain Markdown file at the root of a website that gives AI tools a short, curated map of the site's most useful pages. On its own, it does not help you appear in AI answers: large 2025 and 2026 studies found no link between having the file and being cited, and most AI crawlers rarely request it. It takes about 10 minutes to add, so it is worth doing for a few specific reasons, but it is not an AI visibility strategy.
That answer disappoints people who were told llms.txt is "robots.txt for AI" or "the new sitemap". This guide explains what the file actually is, what the evidence says about who reads it, when it is still worth adding, and what really gets a site cited by ChatGPT, Perplexity and Google's AI features.
What is llms.txt?
llms.txt is a proposal published by Jeremy Howard on 3 September 2024 at llmstxt.org. The idea is simple: web pages are built for people, with menus, scripts and ads, and they are hard for a language model to read cleanly. A short Markdown file can point an AI tool straight to the pages that matter.
The format has only a few parts:
| Part | Required? | What it holds |
|---|---|---|
| H1 title | Yes | The name of the site or project |
| Blockquote | No | A one-paragraph summary of what the site is |
| Plain text | No | Extra context, such as who the site is for |
| H2 link lists | No | Sections of links in the form [Page name](URL): short note |
| "Optional" section | No | Secondary links an AI tool can skip if it needs less context |
The file lives at yoursite.com/llms.txt. The 2026 version of the proposal also allows clean Markdown copies of pages (for example page.md) and link tags that point AI tools to them.
A real example: grovant.co/llms.txt
This is how our own file is built, shortened:
| Section | Example line from grovant.co/llms.txt |
|---|---|
| Title (H1) | # Grovant |
| Summary (blockquote) | > Grovant is a senior-only team for marketing, design and development. Every engagement is run by one named senior specialist, priced by custom quote, month to month. |
| Link list (H2) | ## Services |
| Link with a note | - [SEO](https://grovant.co/services/seo): One senior specialist owns the whole account: audit, technical fixes, content and links. |
| Optional section | ## Optional, then - [About](https://grovant.co/about): Why Grovant exists. |
It is not robots.txt. llms.txt cannot block or allow any crawler. Access rules for AI bots such as GPTBot and ClaudeBot still live in robots.txt.
Does anyone actually read llms.txt? What the evidence says
Four independent sources, using different methods, point the same way: plenty of sites publish the file, few AI systems fetch it, and none shows an effect on citations.
| Source | What was measured | Key finding |
|---|---|---|
| SE Ranking, Nov 2025 | About 300,000 domains, citation frequency in major LLM answers | 10.13% of domains had llms.txt. No link between the file and how often a domain is cited; their model got more accurate when the llms.txt factor was removed |
| Ahrefs, Jun 2026 | 137,210 domains, requests to llms.txt in May 2026 | 28% published the file, but 97% of those files got zero requests. SEO audit tools were the biggest group of requests (21.7%); AI retrieval bots were 1.1% |
| EZY Research, Jul 2026 | 83 sites over 12 weeks, server logs | OpenAI's crawlers fetched llms.txt 7 times against 3,990 robots.txt fetches. ClaudeBot: 9 vs 3,120. PerplexityBot: 0 vs 775. Only Meta's crawler fetched it more often than robots.txt |
| Google, 2025 to 2026 | Official statements and guidance | Google's AI optimization guidance lists llms.txt as something you do not need. John Mueller compared it to the keywords meta tag. Chrome's Lighthouse 13.3 added an experimental llms.txt check, for browser agents, not search rankings |
Two cautions when reading these numbers. A crawler fetching the file does not prove the file changes any answer. And the studies measure different things, so none can prove that llms.txt never helps. What they do show is that there is no evidence it helps today, and that the main AI search crawlers rarely ask for it.
Where llms.txt is genuinely used
The clearest real use is in developer tools. AI coding assistants and documentation platforms can pull a docs site's llms.txt (or a longer llms-full.txt) into context, so a model can answer questions about an API or library. That is useful for software and SaaS companies with public documentation, and much less relevant for a typical service or ecommerce site.
Should you add an llms.txt file?
Add one if any of these apply. Skip it, without worry, if none do.
| Add llms.txt if... | Skip it if... |
|---|---|
| You publish developer docs, an API or a software library | You expect it to raise rankings or AI citations |
| You want a clean, curated page list ready for AI agents and browser agents | Your team has more valuable work waiting, such as content or technical fixes |
| Your CMS or docs platform can generate it in one click | Keeping it accurate would become another forgotten task |
| You want to pass Lighthouse's experimental agent-readiness check | You would publish it once and never update it |
How to add llms.txt in 10 minutes
- List your 10 to 20 most important pages: services, key guides, pricing or contact, about.
- Write the file in Markdown: an H1 with your brand name, a one-paragraph summary in a blockquote, then H2 sections of links with one-line notes.
- Use plain characters. Save the file as UTF-8. Broken characters in place of dashes or quotes are a common mistake and make the file harder to read.
- Upload it to your root folder so it loads at yoursite.com/llms.txt with a text/plain content type.
- Keep it in sync. Ideally your CMS rebuilds it every time you publish, so new pages are never missing.
The file must never contain anything your public pages do not. It is a map, not a place for extra claims.
What actually gets you cited by AI tools
If llms.txt is not the lever, what is? AI search tools mostly find sources the same way search engines do, then pick the passages that answer the question best. The work that moves AI visibility looks a lot like good SEO with a few extra habits:
- Let AI crawlers in. Check that robots.txt and your firewall do not block OAI-SearchBot, PerplexityBot, ClaudeBot or Googlebot. This matters far more than llms.txt.
- Answer first. Open each page with a direct 40 to 60 word answer, then go deeper with short, self-contained sections and tables.
- Cover the topic fully. A planned content strategy with topic clusters gives AI tools many connected pages that answer related questions.
- Publish something only you can say. Original data, real pricing ranges and examples from your own work are what AI answers quote and credit.
- Earn mentions elsewhere. AI tools lean heavily on third-party sources, so coverage, reviews and community answers matter. This is where public relations and SEO meet.
- Track it. Test the prompts your buyers ask in ChatGPT, Perplexity and Google AI each month, and note who gets cited. An AI visibility tool can run these checks for you.
For the bigger picture of how this differs from classic SEO, read GEO vs SEO vs AEO. Agencies that want to offer this to clients can start with our guide to white label GEO.
Get found in AI answers, not just on Google
Grovant's SEO practice covers AI visibility as part of the same account: crawler access, answer-first content, original data and the mentions that AI tools rely on. One senior specialist runs it, starting with a free audit that includes a check of how ChatGPT, Perplexity and Google AI describe your brand today. Pricing is a custom quote, month to month.
