spider trap
Also called: crawler trap, crawl trap
A spider trap (or crawler trap) is a site structure that catches search crawlers in a near-infinite loop of low-value URLs, like a calendar linking to endless dates or filters generating limitless parameter combinations. It wastes crawl budget and can slow indexing of your real pages.
How spider traps form
Google lists a few usual suspects. A calendar that links to an endless run of future dates mints a new URL for each one:
/events/2027/03/ → /events/2027/04/ → /events/2027/05/ → ...Faceted navigation is the big one. When filters live in URL parameters, every combination of color, size, and price is a fresh URL, so Google warns it “can generate infinite URL spaces.” Broken relative links that keep appending path segments (/shop/shop/shop/), pagination that returns a 200 for ?page=99999, and session IDs baked into the URL do the same thing.
The damage is crawl waste. Google calls it overcrawling: bots “access a very large number of faceted navigation URLs before the crawlers’ processes determine the URLs are in fact useless.” That leaves less time for real pages, and on a large site it “wastes a lot of Google crawling time on your site.” Traps can also flood the index with thin, duplicate URLs.
Standard fixes: disallow whole trap patterns in robots.txt, return a genuine 404 when a filter combination has no results (not a soft 200), move filter state into URL fragments like #color=green that Google generally does not crawl, and repair the broken links or calendar loops feeding the trap. Finding and closing these loops is core technical SEO work.
How it affects your traffic
A spider trap quietly bleeds crawl budget: Googlebot spends its fetches on junk URLs, so new products, posts, and landing pages get discovered and refreshed more slowly. On large e-commerce or listing sites the same loops can bury the index under thin duplicate URLs that split ranking signals. A technical SEO audit reads your server logs and crawl data to find these loops, then closes them with robots.txt rules, correct status codes, and fixed link structures so crawl budget flows to the pages that actually earn traffic.
Get Technical SEO that moves the needle
We turn terms like this into ranked pages and qualified pipeline. Start with a free Initial SEO Strategy.