From Anker SOLIX to xTool — the brands above already see how ChatGPT, Gemini and Perplexity mention, cite and recommend them. Your brand is being talked about in AI right now. See it.
llms.txt in 2026: Does It Work? Evidence & Verdict | GEOly | GEO/AEO Platform for DTC Brands
Does llms.txt Actually Work? What the Data Says in 2026
Summary
The 2026 evidence is in: llms.txt is cheap insurance for docs-heavy sites, but Google ignores it, AI crawlers barely request it, and it is not a ranking lever.
2026/08/24
11 min read
If you have spent any time in GEO or AI SEO communities over the past two years, you have seen the pitch: add an llms.txt file to your site and AI engines like ChatGPT, Perplexity, and Gemini will understand your content better, cite you more, and send you more traffic.
It is a seductive idea — one small Markdown file as a shortcut to AI visibility. By mid-2026, we finally have enough hard data to answer the question honestly: does llms.txt actually work?
The short answer: as a discovery or ranking mechanism, no — the server logs are brutal. As a low-cost convenience layer for documentation-heavy sites and AI coding assistants, it has a modest, real use. This article walks through the evidence on both sides, with sources you can check yourself, and ends with what actually moves AI visibility instead.
Key Takeaways
Google is on the record that it ignores llms.txt. Google Search does not use it for crawling, indexing, or ranking, and John Mueller has compared it to the long-dead keywords meta tag (Search Engine Land).
AI crawlers barely request the file. One analysis of 515 million LLM bot events found just 408 requests to /llms.txt (Limy), and Ahrefs found 97% of llms.txt files received zero requests in a month (Ahrefs).
No study has measured a ranking or citation effect from adding the file. Adoption is real but shallow — roughly 8–10% of sites depending on the sample.
Verdict: cheap insurance, not a lever. Publish one if you run a docs-heavy or developer-facing site (it takes an hour and Anthropic itself publishes one), but spend your real effort on citations, entity consistency, and extractable content.
What llms.txt is — and what it is not
The llms.txt proposal was published in September 2024 by Jeremy Howard, co-founder of Answer.AI and creator of fast.ai. The spec lives at llmstxt.org, and the idea is simple: place a Markdown file at /llms.txt on your root domain that gives large language models a curated, token-efficient map of your most important content — a title, a short summary, and organized lists of links to clean Markdown versions of key pages.
The rationale is sound. LLMs operate with limited context windows, and parsing a modern web page — navigation, cookie banners, JavaScript, ads — is expensive and lossy. A hand-curated Markdown index solves that, in theory, the way a sitemap solves URL discovery.
But three misconceptions have inflated the file into something it was never designed to be:
llms.txt is not robots.txt. Robots.txt is an exclusion protocol with decades of de facto enforcement — it tells crawlers what they may not access, and major crawlers honor it. llms.txt is the opposite: an invitation with no enforcement and no commitment from any major AI lab to read it. If your goal is controlling AI training access, that battle is fought in robots.txt directives for GPTBot, ClaudeBot, and Google-Extended — not here.
It is not an index guarantee. Publishing an llms.txt does not make ChatGPT or Gemini ingest, prefer, or cite your content. No AI engine's documented pipeline treats the file as an input.
It is not a ranking signal. No public study to date has measured a relationship between having the file and appearing more often in AI answers.
With definitions clear, here is what actually happened after two years of evangelism.
The adoption story: real growth, shallow penetration
So this is not a dead-on-arrival spec. Real teams — disproportionately developer tools, SaaS, and documentation platforms — are shipping the file. The question is whether anyone on the consumption side is picking it up. That is where the data turns cold.
The evidence FOR llms.txt
To be fair to the standard, the case for it rests on three legs:
1. The cost is nearly zero. An llms.txt file is a static Markdown document. For a site with organized documentation, generating one takes an hour manually or minutes with tooling — Mintlify, Docusaurus plugins, and various CMS extensions produce them automatically.
2. The docs-and-agents ecosystem genuinely uses it. This is the strongest real-world use case. When a developer working in Cursor or Claude Code wants to load a product's API documentation into context, an llms.txt file is a clean, token-efficient entry point. Anthropic asked Mintlify to implement llms.txt and llms-full.txt for its own documentation, and Anthropic's docs still serve a full Markdown map today. That is a meaningful endorsement of the format for a specific job: feeding docs to coding assistants on demand. FastHTML, Zapier, Cloudflare, and a long tail of developer platforms publish one for the same reason.
3. It future-proofs against a possible shift. If agentic browsing keeps growing — AI agents completing tasks on websites rather than humans clicking — a machine-readable content map could become more valuable. That is speculative, but the option costs almost nothing to hold. (GEOly's own site serves one at geoly.ai/llms.txt, mapping our documentation and blog — for exactly this "cheap option" logic, not because we expect it to move rankings.)
The evidence AGAINST llms.txt
Now the other side of the ledger, which is considerably heavier.
Google ignores it — explicitly and repeatedly. Google's Gary Illyes confirmed at Search Central Live in July 2025 that Google does not support llms.txt and is not planning to, and that ranking in AI Overviews requires only standard SEO. John Mueller has been blunter, comparing llms.txt to the keywords meta tag — a self-declared signal search engines abandoned because it was trivially manipulable — and noting that no AI service has said it uses the file, adding that "you can tell when you look at your server logs that they don't even check for it." Given that Google's ecosystem (Search, AI Overviews, AI Mode, Gemini) drives the largest share of AI-mediated discovery, its refusal alone caps the file's upside.
Server logs confirm the crawlers are not asking for it. This is the decisive evidence, replicated across independent datasets:
Limy analyzed 515,382,577 LLM bot traffic events over a 90-day window, filtering for GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, and Google-Extended. Just 408 requests targeted /llms.txt — statistically negligible.
Ahrefs studied 137,210 domains in May 2026 and found 97% of published llms.txt files received zero requests. Of the traffic that did arrive, most came from SEO audit tools and generic crawlers; AI retrieval bots — the ones that actually fetch content to answer user queries — accounted for just 1.1% of requests. Ahrefs also found that zero AI bots proactively "go looking" for llms.txt files that don't exist.
OtterlyAI's three-month experiment logged more than 60,000 AI bot hits on a site with a correctly implemented llms.txt; the file was used in roughly 0.1% of AI visits. Their conclusion: AI agents overwhelmingly discover content through standard pages and site structure, not llms.txt.
No major AI lab has committed to it in production. As of 2026, neither OpenAI, nor Google, nor Meta, nor Mistral has publicly committed to reading or acting on llms.txt in their production retrieval systems (PPC Land). Even Anthropic's use is about serving its own docs to coding tools — not a statement that ClaudeBot consults your llms.txt when crawling your site.
No measured visibility effect. No public study has demonstrated that adding llms.txt increased AI citations, AI Overview inclusion, or answer-engine traffic. Absence of evidence after two years and multiple large-scale studies is, at this point, evidence worth weighing.
The verdict: cheap insurance, not a ranking lever
Put the two columns side by side and the conclusion writes itself.
Publish an llms.txt if you run a documentation-heavy, developer-facing, or API-driven site. Your real audience for the file is not GPTBot crawling the open web — it is a developer's coding assistant pulling your docs into context on demand. That workflow exists today, costs you an hour, and mildly improves how AI tools consume your documentation.
Do not publish one expecting more AI citations, better ChatGPT visibility, or any Google effect. The engines that decide whether your brand appears in AI answers are not reading the file.
What actually moves AI visibility, based on everything we track across engines:
Citations on sources AI engines already trust. ChatGPT, Perplexity, and Google AI Mode lean heavily on a recurring set of high-authority domains — Reddit threads, review sites, industry publications. Getting your brand mentioned there matters far more than any file on your own server. Our AI citation source breakdowns show exactly which domains each engine pulls from.
Entity consistency. Same brand name, same descriptions, same facts across your site, Wikipedia-adjacent sources, directories, and social profiles. LLMs assemble brand understanding from many documents; contradictions dilute you.
Extractable content. Clear H2/H3 question-shaped headings, direct answers in the first sentences, tables, FAQs, schema markup. This is the on-page work that engine-specific playbooks like our ChatGPT AEO guide cover in depth — and unlike llms.txt, retrieval bots demonstrably consume these pages.
If you want the full framework, the GEO Academy walks through the complete discipline, from entity building to per-engine optimization.
If your site fits the docs-heavy profile, do it properly. The spec at llmstxt.org requires:
A single Markdown file at https://yourdomain.com/llms.txt (root, not a subdirectory)
An H1 with your site or project name (the only required element)
A blockquote summary, then H2-delimited link sections
An optional ## Optional section for links safe to skip when context is short
A minimal example:
# Acme Analytics
> Acme Analytics is a product analytics platform for B2B SaaS. > Docs cover installation, APIs, and integrations.
## Documentation
- [Quickstart](https://acme.com/docs/quickstart.md): Install and send your first event in 10 minutes - [API Reference](https://acme.com/docs/api.md): REST endpoints, auth, and rate limits - [Integrations](https://acme.com/docs/integrations.md): Salesforce, HubSpot, and Segment setup
## Optional
- [Changelog](https://acme.com/changelog.md): Release history - [Blog](https://acme.com/blog.md): Product updates and guides
Practical rules: link to clean Markdown versions of pages where possible (the spec suggests appending .md to URLs); curate rather than dumping your sitemap; keep descriptions to one line; and update the file when your docs change — a stale map is worse than none. Optionally add llms-full.txt, a single file with your full docs content, for tools that ingest everything at once.
Then set your expectations accordingly, and check your server logs in 90 days.
FAQ
Does llms.txt help with Google SEO or AI Overviews?
No. Google has confirmed on the record that Search does not use llms.txt for crawling, indexing, or ranking, and that normal SEO is what gets you into AI Overviews. John Mueller compared the file to the keywords meta tag. Any effect on Google surfaces is zero by design.
What is the difference between llms.txt and robots.txt?
Robots.txt is a widely honored exclusion protocol — it tells crawlers what not to access, and if blocking AI training bots is your goal, that is where you do it (via user-agents like GPTBot or Google-Extended). llms.txt is a voluntary invitation — a curated content map with no enforcement and, per server-log studies, almost no consumption by AI crawlers. They solve different problems, and only one of them is reliably read.
Should I add llms.txt to my website in 2026?
If you run a documentation-heavy, developer, or API-focused site: yes, it costs about an hour and helps AI coding assistants consume your docs. If you run a typical marketing site, blog, or ecommerce store: it will not hurt you, but expect no measurable visibility gain — put the time into extractable content and citations instead. Our free AI prompt generator is a better first step for understanding how AI engines actually see your niche.
Do ChatGPT, Claude, or Perplexity read llms.txt?
Not systematically. No major AI company has committed to reading llms.txt in production retrieval, and log studies show GPTBot, ClaudeBot, and PerplexityBot almost never request the file — 408 requests out of 515 million bot events in one 90-day dataset. Anthropic publishes an llms.txt for its own docs, which endorses the format for docs delivery, not a commitment that its crawler reads yours.
Can llms.txt hurt my site?
Essentially no, with two caveats: a stale file pointing AI tools to outdated docs can cause bad answers about your product, and an auto-generated dump of thousands of URLs defeats the file's curation purpose. Keep it short, accurate, and maintained — or skip it.
If llms.txt doesn't drive AI visibility, what does?
The measurable levers are citations on domains AI engines already trust, consistent entity information across the web, and content structured for extraction (question headings, direct answers, schema). Comparison and alternative queries are a particularly high-leverage surface — see how brands compete for them on our alternatives pages. Then track results per engine instead of assuming: that is precisely what GEOly measures.
