PerplexityBot
Also called: Perplexity crawler, PerplexityBot/1.0, Perplexity-User
PerplexityBot is the crawler Perplexity AI uses to discover and index web pages so it can surface and link to them in its answers. It identifies as PerplexityBot/1.0 and obeys robots.txt, unlike Perplexity-User, the separate agent that fetches pages live when a person asks a question.
Two crawlers, two rulebooks
Perplexity runs two distinct agents, and confusing them is a common mistake. PerplexityBot is the indexing crawler: it builds the pool of pages Perplexity can surface and link to in search results, and per Perplexity’s docs it is not used to crawl content for AI foundation models. Its full user agent is Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot), and it respects robots.txt. Perplexity-User is different: it fetches a page in real time when someone asks a question that page can answer. Because that request is user-initiated, Perplexity says it “generally ignores robots.txt rules.”
To allow indexing, do nothing, or allow the agent explicitly. To block it:
User-agent: PerplexityBot
Disallow: /Perplexity publishes verifiable IP lists (perplexitybot.json and perplexity-user.json) so you can separate real traffic from spoofers in your server logs.
One caveat worth knowing: in August 2025 Cloudflare reported that Perplexity used undeclared crawlers with generic browser user agents and rotating IPs and ASNs to reach pages that had blocked its declared bot. Perplexity disputed the framing. The practical takeaway for site owners: user-agent alone is not a reliable identity check, and a robots.txt block is not always clean. If your goal is visibility rather than blocking, the useful question is whether PerplexityBot can reach and parse your pages at all.
How it affects your traffic
If PerplexityBot cannot reach or parse your pages, you are absent from one of the fastest-growing answer engines, and a Perplexity citation is a direct referral link, not just brand exposure. Getting cited depends on clean crawl access, clear on-page facts, and content structured so a single passage answers a question outright. That is the core of AI SEO work: making your pages the source Perplexity quotes and links, so you capture clicks that never touch a classic Google result.
Get AI SEO that moves the needle
We turn terms like this into ranked pages and qualified pipeline. Start with a free Initial SEO Strategy.