llms.txt is a proposed web standard: a plain Markdown file served at your domain root (yourdomain.com/llms.txt) that gives large language models a curated, token-efficient index of your most important pages. Where robots.txt sets permissions and sitemap.xml lists inventory, llms.txt supplies context — a one-line description of what your site is, followed by an annotated reading list an AI agent can parse in milliseconds. Proposed by Jeremy Howard of Answer.AI in September 2024, it remains a community convention in 2026: widely published, but not yet officially consumed by any major AI provider.
Key takeaways
llms.txtis a curated Markdown index at your site root. The spec at llmstxt.org requires only an H1 with your site name; the summary blockquote and annotated link sections are recommended, not mandatory.- It answers a different question than
robots.txt(may you crawl this?) orsitemap.xml(what exists here?): it tells a model what to read first, and why. - The honest 2026 status: published files grew 8.8x in twelve months, yet 97% of them receive zero requests from AI bots, and Google says it does not use the file at all.
- The realistic upside is agentic retrieval. Assistants and shopping agents that fetch your site live during a conversation benefit from a clean Markdown map instead of heavy HTML.
- It costs about an hour to implement. Treat it as one low-cost line in a broader GEO checklist, not a visibility strategy on its own.
How llms.txt works
The format, defined at llmstxt.org, is deliberately minimal. A valid file contains, in order:
- An H1 with the site or project name — the only required element, e.g.
# GEOly AI. - A blockquote summary directly below it:
> GEO data platform for DTC brands, covering seven AI engines. - Optional paragraphs of free-form context the model should know before reading further.
- H2 sections grouping annotated links, one per line:
- [Quick start](/docs/quick-start): Install and run your first audit. - An optional section literally titled
Optional, for links an agent may skip when its context budget runs short.
Two companion conventions travel with it. llms-full.txt inlines the complete text of your key pages into a single file, so an agent gets everything in one fetch. The spec also suggests serving Markdown twins of important pages by appending to their URLs.
