Creating an llms.txt file takes about ten minutes. It is a plain Markdown file, saved as llms.txt, placed at the root of your domain so it resolves at yoursite.com/llms.txt. An H1 with your site name, a blockquote summary, then H2 sections holding lists of links with short descriptions.
That is the whole format, and the rest of this page is the part most guides leave out: whether anything reads it.
The short version is that Google has said publicly it does not use llms.txt, one of its own search advocates compared the file to the keywords meta tag, and no major AI company has committed to reading it in production. We publish one anyway. Both of those things can be true, and the reasoning matters more than the file.
The format, in full
The convention is deliberately small. There is no schema to validate against and no registry to submit to.
An optional companion file, llms-full.txt, holds the same idea with far more text: not just the links but meaningful excerpts, so a model reading one file gets substance rather than a directory.
That is it. If you want the structure generated for you rather than hand-built, our llms.txt generator will lay out the sections and formatting, and you fill in the content.
The complete llms.txt format shown as a file: an H1 with the site name, a blockquote summary, then H2 sections holding Markdown links each followed by a short description, with notes on root placement and plain text content type.
What Google has actually said
This is the part worth reading before you spend an afternoon on it, because the public record is unusually direct.
Google Search's AI optimization guidance, published 15 May 2026, states that llms.txt is not needed for AI Overviews, AI Mode, or any other generative AI Search feature. It groups the file with content chunking and AI-specific rewriting as tactics that do not produce AI visibility. The same position runs through Google's AI features docs, which describe ordinary crawlable content as the requirement and name no additional file.
John Mueller has been blunter still, comparing the file to the keywords meta tag, as documented here. No AI service was using it, and the bots were not requesting the file. He described building separate Markdown pages for bots as a poor use of resources. Gary Illyes and Amir Taboul confirmed at a Search Central event that Google was not pursuing it.
The keywords meta tag comparison is doing a lot of work there, and it is worth sitting with. That tag was a self-declared summary of what a page was about. It was gamed within months and search engines stopped reading it. A file where you tell the machines what matters on your site has the same structural weakness, and everyone involved knows it.
Meanwhile the file is clearly being crawled by something. Wix Studio's AI Search Lab found in October 2025 that Google indexes somewhere between 30,000 and 60,000 llms.txt files globally. Indexed is not the same as used for anything, which is precisely the gap this whole topic sits in.
What Google has said about llms.txt: its May 2026 guidance states the file is not needed for any generative AI Search feature, Search Central confirmed Google is not pursuing it, and a search advocate compared it to the keywords meta tag.
So why do we publish one?
Fair question to put to an agency that just told you the evidence is thin. Three honest reasons, none of them "it improves rankings".
It costs very little once it is automated. Ours is generated from the same data that builds the site. If it were a manual file we would have stopped maintaining it a year ago, which is the real lesson further down.
The downside is nil and the option value is real. If any assistant starts using the file, sites that already have one are ready. If none ever does, we spent an afternoon. That is an acceptable bet at that price, and a bad one at the price of a consultant's retainer.
It forces a useful exercise. Writing a one-line description of every important page on your site, in one place, surfaces how much of your site has no clear purpose. We found pages we could not describe. That was worth more than the file.
What we would not do is pay anyone to build one, or treat it as an AEO strategy. The work that actually gets a business cited in AI answers is structural: answer-first pages, quotable evidence, third-party corroboration. That is covered properly in how AEO differs from traditional SEO, and it is where our AEO service puts its hours.
A small white robot figure standing on a plain surface.
When it is genuinely worth doing properly
The blanket case is weak. The specific cases are not.
Developer documentation and API references. This is the use the convention was designed around. If your users point coding assistants at your docs, a clean index of Markdown versions is a real convenience with a real audience.
Sites where agent traffic already shows up in your logs. If you can see assistant user agents fetching your pages in volume, you are not speculating about the audience.
Large sites with a genuinely confusing structure. If a human cannot find your important pages in two clicks, a curated index has value regardless of who reads it.
Anywhere you would benefit from the exercise itself. See above. The audit is the product.
If none of those describe you, publishing a basic llms.txt and forgetting about it is a perfectly defensible position. Publishing one and telling clients it improves AI visibility is not.
Four cases where an llms.txt file is genuinely worth building: developer documentation and API references, sites already seeing agent traffic in their logs, large confusingly structured sites, and doing it for the auditing exercise itself.
llms.txt, llms-full.txt and robots.txt do different jobs
These get conflated constantly, including by people selling llms.txt services.
| File | What it is for | Who honors it |
|---|---|---|
| robots.txt | Permission. What crawlers may and may not fetch | Widely honored, a long-standing standard |
| llms.txt | A curated index of your important pages, for models | An informal convention, not committed to by major AI companies |
| llms-full.txt | The same idea with full text or long excerpts | Same, and even less established |
The important distinction is that robots.txt controls access and llms.txt does not. An llms.txt file gives nobody permission to do anything, and blocks nobody. If your goal is to control whether AI companies train on your content, llms.txt is the wrong file entirely and robots.txt directives are where that conversation lives.
Comparison of robots.txt, llms.txt and llms-full.txt showing what each is for and who honors it, with robots.txt the only one that actually controls crawler access.
The maintenance trap, which nobody warns you about
Here is the honest first-hand part, and the thing we got wrong.
Our llms.txt currently runs 560 lines and about 140 KB, with 502 linked entries across eight sections. The companion llms-full.txt is 1,655 lines and roughly 340 KB. That is a large file by the standards of a convention that was pitched as a concise index, and it got that way one blog post at a time.
The failure mode is not that the file is wrong when you publish it. It is that the file is correct on day one and silently wrong by month three. Every new page, every changed URL, every deleted service is a divergence, and nothing breaks when it drifts. No error appears. No tool complains. You simply end up serving a confident, well-formatted description of a site that no longer exists.
We have had to reconcile ours against the live site more than once, and every time the gap was larger than expected. The lesson we took from it is the one worth passing on: if you cannot generate the file automatically, do not create it. A stale llms.txt is worse than no llms.txt, because it is a document whose entire purpose is to be an accurate summary.
KeyGrow's own llms.txt at 560 lines with 502 entries and llms-full.txt at 1,655 lines, with a warning that the file is correct on day one and silently wrong by month three because nothing breaks when it drifts.
Five mistakes worth avoiding
Putting it anywhere but the root. It must resolve at yoursite.com/llms.txt.
Serving it as HTML. It should come back as plain text. A file served with the wrong content type is a file nothing can parse.
Listing every page you have. The value of a curated index is the curation. Ours drifted toward completeness and is the weaker for it.
Descriptions that describe nothing. "Our services page" tells a model nothing it could not infer from the URL. Say what is actually on the page.
Treating it as a ranking tactic. It is not one, on the current public evidence, and a client who was sold it as one will eventually read the same Google documentation you just did.
Five ways an llms.txt file goes wrong: placed outside the root, served as HTML, padded with every page on the site, and sold as a ranking tactic.
Do you need help with this?
No, and this is one of the clearest cases on our whole blog.
An llms.txt file is a text file. Writing one is an afternoon at most, generating one is a script, and the published evidence says the upside is currently speculative. Paying an agency to produce one is paying for a deliverable that is easy to produce and hard to value. If someone quotes you for an llms.txt implementation, ask them which AI systems read it, and compare the answer to Google's own documentation.
Where outside help is worth it is the question underneath: whether your business is actually being cited in AI answers, and what is stopping it. That is measurement and content structure work, not a text file, and it is the thing we would rather be doing for you.
A row of glossy white humanoid figures against a dark blue background.
FAQs
What is an llms.txt file?
It is a plain Markdown file at the root of a domain that gives language models a curated index of a site's important pages, each with a short description. It is an informal convention rather than a standard, with no validator and no registry.
How do I create an llms.txt file?
Write a Markdown file containing an H1 with your site name, a blockquote summarizing the site, then H2 sections holding lists of links with short descriptions. Save it as llms.txt and upload it to your web root so it resolves at yoursite.com/llms.txt.
Does Google use llms.txt?
No. Google's AI optimization guidance published in May 2026 states it is not needed for AI Overviews, AI Mode or any generative AI Search feature, and Google representatives have confirmed they are not pursuing it.
Is llms.txt worth creating in 2026?
For developer documentation and API references, yes. For a typical business site, it is a cheap bet rather than a strategy: low cost, no downside, speculative upside. It should not be sold or bought as an AI visibility service.
What is the difference between llms.txt and robots.txt?
robots.txt controls crawler access and is a long-standing standard that is widely honored. llms.txt controls nothing and grants nothing. It only suggests which pages matter. If you want to restrict AI training, robots.txt is the file that matters.
What is llms-full.txt?
A companion file holding the same index plus substantial text or excerpts from each page, so a model reading one file gets content rather than just links. It is even less established than llms.txt and considerably more work to keep current.
Does llms.txt help with AI Overviews or ChatGPT citations?
There is no public evidence that it does, and Google has explicitly said it is not needed for its AI features. Citations are earned through answer-first content structure and outside corroboration rather than through a file that declares your own importance.
How often should I update llms.txt?
Every time the site changes, which is why it should be generated rather than hand-written. A file that quietly falls out of sync is worse than no file, because its only purpose is to be an accurate description of the site.
Our actual recommendation
Generate one if you can automate it. Skip it if you cannot.
That is a less satisfying answer than most pages on this keyword will give you, and it is the one supported by what Google has published and what the people who work on its crawlers have said out loud. The file is cheap, harmless, and currently unproven, which makes it a reasonable thing to have and an unreasonable thing to pay for.
If the underlying goal is getting your business named when someone asks an assistant for a recommendation, that is a real problem with real methods, and none of them are a text file. Start with how AEO differs from traditional SEO instead.
