Technical GEO is the work of making sure AI crawlers can reach your pages, parse them cleanly, and quote them accurately inside a generated answer. Traditional technical SEO tuned a site for Googlebot's crawl-and-index loop; technical GEO tunes it for a different pipeline — bots that either train a model on your content or retrieve passages from it in real time to assemble an answer (retrieval-augmented generation, or RAG). When the plumbing is wrong — a blocked user-agent, div-soup markup, missing schema, no visible dates — your brand never enters the conversation, however sharp the copy is.
The checklist below is organized around the audit framework we use at GEOly AI: a 4D × 5L matrix that crosses four performance dimensions with five technical layers. Work through it once as a baseline, then re-run the parts that change. If you're new to the discipline, start with what GEO is and how AI search visibility is measured before you touch the infrastructure.
Key takeaways
- robots.txt is now a strategy decision, not a formality. Providers split training crawlers (GPTBot, Google-Extended, ClaudeBot) from retrieval crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot) — allow the retrieval bots or you are ineligible for citations.
- llms.txt is worth shipping but oversold. Adoption sits near 10% of domains, yet independent 2026 log studies found most llms.txt files receive zero AI-crawler requests; treat it as a cheap hedge, not a ranking lever.
- Clean semantic HTML, JSON-LD entity schema, and visible published and modified dates do more for citeability than any single new file.
- Token efficiency is real: summary-first, high-density content near the top of the DOM is what engines lift into answers.
- Audit on a loop. A 29-point GEO audit plus server-log analysis converts twenty scattered checkpoints into a weekly score you can act on.
The files that gatekeep AI access
Two files decide whether an AI system can use your content at all. Get them right before you touch anything else.
robots.txt: separate training from retrieval
In the SEO era you worried about one line: User-agent: Googlebot. In 2026 the major labs run several bots with different jobs, and lumping them together is the most common technical-GEO mistake we see.
- Training crawlers ingest pages to improve future model versions:
GPTBot(OpenAI),Google-Extended(Google),ClaudeBot(Anthropic),CCBot(Common Crawl).
