What is llms.txt?
llms.txt is a plain-text file at the root of a website that gives AI assistants a curated, markdown-formatted map of its most important pages, so a model can find the substance without crawling and parsing the whole site.
Also called: llms.txt file, AI crawler file, llms-full.txt.
Think of robots.txt as the door policy and llms.txt as the guided tour. Robots.txt says what a crawler may not touch; llms.txt says here is what actually matters, described in the words we would use ourselves. Because it is markdown, a model ingests it cheaply and accurately, with no layout or navigation noise in the way.
It is a young convention rather than a ratified standard, and no assistant guarantees it will read yours. The cost of publishing one is close to zero and the upside — being summarised in your own language rather than an inference drawn from your footer — is real, so most serious sites now ship one.
The common mistake is writing it for search engines: a dumped list of every URL. It works better as an editor's summary, with a one-line description per link and only the pages you would genuinely want quoted.
What to actually do about it
- Serve it at /llms.txt as text/plain, with an H1 and a blockquote summary.
- Group links under headings and add one sentence of context each.
- Include pricing, contact and service pages — the things people ask about.
- Generate it from your real content so it can't go stale.
- Keep it short: this is a curated map, not a sitemap.
Koo generates llms.txt and a longer llms-full.txt for every site from its live pages and posts, so both refresh automatically whenever content changes.
Related terms
Answer engine optimisation (AEO)
Answer engine optimisation (AEO) is the practice of structuring a website so AI assistants like ChatGPT, Perplexity, Gemini and Google's AI Overviews can quote it accurately when someone asks a question.
robots.txt
robots.txt is a plain-text file at a site's root that tells crawlers which paths they may or may not request. It controls crawling, not indexing, and it is a public request rather than a security measure.
XML sitemap
An XML sitemap is a file listing the URLs on a site that you want search engines to crawl, along with when each was last changed, submitted through Search Console so new and updated pages are found quickly.