LLM SEO is the practice of making your content retrievable, quotable and citable by LLM-powered search: ChatGPT, Perplexity, Google’s AI Overviews and AI Mode, and Claude. Unlike classic search, these systems don’t rank pages. They retrieve passages, assemble an answer from them, and cite the sources those passages came from. That single difference changes what “optimized” means, and this guide covers the full shift: how retrieval actually works, how to write passages a model can lift, which crawlers to let in, and what llms.txt does and doesn’t do.
What is LLM SEO?
LLM SEO means optimizing a website so large language models can find, understand and cite its content when they answer user questions. The label is new, but the discipline overlaps almost completely with what other people call generative engine optimization (GEO) or answer engine optimization (AEO). Same work, three names. We untangle the terminology properly in our guide to what generative engine optimization is.
The name matters less than the mental model. Google’s classic index scores whole pages and returns ten links. An LLM answer engine scores passages, chunks of a few hundred words, and quotes the best ones. You can hold position 4 for a keyword and still never appear in the AI answer above the results, because the engine found a cleaner, more liftable paragraph on a page ranking 14th. The unit of optimization has moved from the page to the passage.
This channel already converts, and not by a little. Vercel reports that around 10% of its new signups now come from ChatGPT, up from 4.8% the month before and roughly 1% six months prior. Buyers are asking chat interfaces the questions they used to type into Google, and the sites that get cited collect both the referral and the brand impression.
How LLM search finds and cites content
Every LLM search product runs a version of the same pipeline: crawl the web, index it, retrieve the chunks relevant to a query, then synthesize an answer with citations. Engineers call this pattern RAG, retrieval-augmented generation. For our purposes only the consequence matters: your content competes at the retrieval step, chunk against chunk, not page against page.
There are two ways into an answer. The first is training data: what the model absorbed about your brand before its knowledge cutoff. This pathway is powerful but slow. You influence it over quarters, through mentions accumulated across the web, not through page edits. The second is live retrieval: the model searches an index at question time and quotes what it finds. This pathway updates in days and rewards page-level work, which is why nearly everything in this guide targets it.
One documented mechanic makes structure unusually valuable here. Google Search Central states that AI Overviews and AI Mode may use a “query fan-out” technique, issuing multiple related searches across subtopics and data sources to build a single response. A user’s question fans out into sub-queries, and each sub-query retrieves independently. A page that covers a topic’s sub-questions in discrete, clearly labeled sections enters several of those retrieval lotteries with one URL. A page written as one continuous essay enters one.
LLM SEO vs traditional SEO: what actually changes
Start with what doesn’t change, because “SEO is dead” is the laziest take in this niche. Google’s own documentation says there are “no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary.” A page simply has to be indexed and eligible to appear in Google Search with a snippet. Crawlable, indexed, snippet-eligible: classic SEO is the entry ticket. If your technical foundation is broken, no amount of AI search optimization fixes it.
What changes is the scoring and the scope:
| Dimension | Traditional SEO | LLM SEO |
|---|---|---|
| Unit scored | The page | The passage |
| Success metric | Rankings, clicks | Citations, mentions in answers |
| Index you optimize for | Google’s | Several: Google’s, OpenAI’s, Perplexity’s, Anthropic’s |
| Feedback loop | Rank tracking | Prompt spot-checks, AI referral traffic |
| Off-page lever | Backlinks | Backlinks plus retrievable brand mentions |
Two practical notes on that table. First, ChatGPT search runs on OpenAI’s own index and Perplexity maintains its own, so ranking well on Google helps but doesn’t transfer automatically. Second, the same Google doc warns that snippet controls cut both ways: nosnippet, data-nosnippet, max-snippet and noindex all limit what can show up in AI features. Plenty of sites still carry max-snippet caps from an old featured-snippet strategy, quietly truncating themselves out of AI answers today.
The Google side of this discipline has its own quirks, from triggering behavior to citation formatting. We cover those separately in how to rank in AI Overviews.
Write self-contained passages an LLM can lift
Chunking usually gets discussed as an engineering topic, something a RAG developer worries about. For LLM visibility it’s a writing discipline, and it’s the highest-impact on-page work you can do. Six rules cover most of it:
- One idea per H2/H3 block. If a section argues two things, split it.
- Open every section with a 40-80 word direct answer. Elaborate after, not before. Retrieval favors passages that resolve the query immediately.
- Make headings questions or claims, not puns. “What does llms.txt actually do?” retrieves; “The .txt files of our lives” doesn’t.
- Never let a passage lean on the previous section. Pronouns like “it” and phrases like “this approach” die the moment a chunk is quoted alone.
- Keep entity names inside the passage. Name the product, the brand, the topic, so the chunk stays attributed out of context.
- Prefer tables, definitions and numbered steps. Structured formats survive extraction intact; flowing prose gets paraphrased and loses your attribution.
Here’s the difference in practice.
Before: “It also helps with speed. Combined with the previous method, most teams see results within a few weeks, which is why we usually recommend doing both together.”
After: “Server-side rendering also shortens the crawl-to-citation delay. Sites that render content on the server get picked up by AI crawlers on the first fetch, with no dependency on JavaScript execution.”
Same intent, but the second version resolves every pronoun, names its entities and makes a claim that stands alone. That’s the version an answer engine can lift.
Specificity does the rest. A model assembling an answer needs a quotable fact: a number with a named source, a dated finding, a concrete threshold. Vague paragraphs offer nothing to cite, which is why generic content earns paraphrases while specific content earns links.
Open the gates: AI crawler access audit
The most common LLM SEO failure we see is self-inflicted: the site blocked the crawler that feeds the answer engine. It happens because “AI bots” is not one thing, and the official docs make distinctions that most blanket-block rules ignore.
OpenAI’s bot documentation describes three separate crawlers with three separate jobs. OAI-SearchBot builds the index behind ChatGPT search, and OpenAI is explicit that “sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers.” GPTBot controls whether your content is used for model training, nothing else. ChatGPT-User handles user-initiated fetches, when someone pastes your URL into a chat. Robots.txt changes take roughly 24 hours to take effect. Anthropic’s crawler docs mirror the split exactly: ClaudeBot collects training data, Claude-SearchBot improves search result relevance, Claude-User fetches pages on a user’s request, and each can be blocked individually.
Google works differently. Its AI features use plain Googlebot, while Google-Extended only controls Gemini training and grounding. Here’s the full matrix:
| Bot | Operator | Job | Block it and… |
|---|---|---|---|
| OAI-SearchBot | OpenAI | ChatGPT search index | You disappear from ChatGPT search answers |
| GPTBot | OpenAI | Model training | Future models learn less about you; search is unaffected |
| ChatGPT-User | OpenAI | User-initiated fetches | Users can’t pull your live pages into a chat |
| ClaudeBot | Anthropic | Model training | Same trade-off as GPTBot |
| Claude-SearchBot | Anthropic | Search relevance | Claude’s search answers skip you |
| Claude-User | Anthropic | User-initiated fetches | Same as ChatGPT-User |
| Googlebot | Search plus AI Overviews/AI Mode | You leave Google entirely | |
| Google-Extended | Gemini training control | No effect on AI Overviews |
The trap is in CDN and WAF dashboards. A “block AI bots” toggle typically blocks every user agent in that table except Googlebot, including the search bots. Sites flip it to opt out of training and accidentally opt out of AI search distribution. Audit both layers: robots.txt and your CDN bot rules, checked against the matrix above.
Access is only half the audit; rendering is the other half. Vercel’s crawler study found that none of the major AI crawlers render JavaScript. OpenAI’s bots, ClaudeBot, PerplexityBot, Meta-ExternalAgent and Bytespider all skip execution, even though they download JS files (ChatGPT’s crawlers spend 11.5% of requests on them, Claude’s 23.8%). If your content only exists after client-side rendering, it is invisible to every AI crawler on that list. We break down the evidence and the fixes in do AI crawlers execute JavaScript.
The same study puts scale and sloppiness in perspective: GPTBot made 569 million monthly fetches and Claude 370 million against Googlebot’s 4.5 billion, and ChatGPT’s crawler wasted 34.8% of its fetches on 404 pages versus Googlebot’s 8.2%. AI crawlers are already heavy traffic but far less efficient, so clean URL hygiene and accurate sitemaps matter more for them than they do for Google. This is standard scope in a proper technical SEO audit; the difference now is that broken URLs waste the crawl budget of four indexes instead of one.
llms.txt: what it is and what to honestly expect
llms.txt is a proposed standard, not an adopted one, and you should treat it accordingly. The spec was published by Jeremy Howard on September 3, 2024: a markdown file served at /llms.txt containing an H1 title, a blockquote summary of the site, and H2 sections of curated links. The rationale is genuinely sound, since “context windows are too small to handle most websites in their entirety.” The proposal also recommends serving a clean markdown version of key pages, reachable by appending .md to the URL, so an agent can read your documentation without fighting your navigation. Many adopters additionally publish an llms-full.txt that concatenates their docs, but that file is a community convention, not part of the spec.
A minimal file looks like this:
# SEOBRO
> SEO agency focused on lead generation for niche businesses.
> Services, methodology and case studies below.
## Services
- [Technical SEO](/services/technical-seo/): crawl,
indexation and rendering audits
- [GEO](/services/generative-engine-optimization/):
visibility in AI answer engines
## Guides
- [LLM SEO](/blog/llm-seo/): getting cited by
ChatGPT, Perplexity and AI OverviewsNow the part most guides skip. Google explicitly states that “you don’t need to create new machine readable files, AI text files, or markup” to appear in its AI features. And no major platform, OpenAI and Anthropic included, has publicly committed to consuming llms.txt for search. Anyone selling llms.txt as a ranking lever is selling you a file the engines haven’t promised to read.
Our verdict: it’s optional, and it changes nothing for ranking. Docs-heavy sites and SaaS products get the most marginal value, because coding agents and research assistants do fetch these files opportunistically even where search products don’t. If it’s trivial to add for your stack, do it and move on; if it isn’t, skip it and lose nothing measurable. If you have one hour for LLM SEO this month, spend it on crawler access and passage rewrites, not here.
Earn the mentions LLMs learn from
Off-page work changes shape in AI search. Models weight brands by how often and how consistently they appear across the retrievable web, which means the surfaces that answer engines actually pull from are your new link targets: industry listicles, comparison pages, Reddit and community threads, review platforms, and digital PR that produces a quotable statistic.
Original data is the strongest citation magnet there is. Everyone cites the source of a number, humans and models alike, because an answer needs evidence and evidence needs attribution. One genuine survey or dataset outperforms a dozen opinion posts in earned citations.
Keep the execution narrow. Ask ChatGPT and Perplexity the ten buying-intent questions your customers actually ask and note which domains get cited; those domains are your target list. Pick the five highest-retrieval surfaces on it and get present on each: a listicle inclusion, an honest entry on a comparison page, a genuinely useful community answer. Consistency matters as much as presence, so keep your positioning and one-line description identical everywhere. Contradictory descriptions across the web dilute what a model can confidently say about you. The ChatGPT-specific version of this work, from prompt research to mention building, is covered in our ChatGPT SEO guide.
Measure it, then focus it on revenue (the FLG lens)
Three signals tell you whether any of this is working. First, segment AI referrals in GA4: traffic from chatgpt.com, perplexity.ai and copilot.microsoft.com, watched as a trend rather than an absolute. Second, branded search lift, because AI answers create brand impressions that come back as branded queries days later. Third, a monthly prompt spot-check: run the 10-20 buying-intent prompts your customers use, in the phrasing they’d use (“best X for Y”, “X vs Y for a small team”), and record who gets cited.
That third signal is the one that matters, and it’s where we apply the FLG lens. SEOBRO runs on Focused Lead Generation: chasing mentions across every possible prompt is vanity, the AI-search equivalent of ranking for keywords nobody buys from. The passages worth engineering are the ones that get cited in commercial prompts where your buyers are actually asking. One citation in “best warehouse software for 3PLs” is a lead channel. Fifty citations in definitional queries are a screenshot for the marketing deck.
Here’s the 30-day starter plan we’d run on any site today:
- Week 1, crawler access audit. Check robots.txt and CDN/WAF bot rules against the crawler matrix above, unblock the search bots you’ve accidentally banned, and confirm your money pages serve full content without JavaScript.
- Weeks 2-3, passage rewrite. Take your five most money-adjacent pages and restructure them: a direct-answer opener per section, self-contained passages, one comparison table each.
- Week 4, off-site and measurement. Pick your five retrieval surfaces and start earning presence; set up the GA4 segment and the monthly prompt spot-check document.
That checklist is the complete map if you want to run it in-house. If you’d rather have it run for you, from the crawler audit through passage engineering to citation tracking against your actual buyer prompts, that’s exactly what our generative engine optimization service does. Either way, start with the crawler audit. Everything else is wasted effort if the bots can’t get in.
