crawl error
Also called: crawl errors, crawling error, crawl error report
A crawl error is a failure a search engine crawler such as Googlebot hits when trying to fetch a URL. It can be site-wide (DNS, server connectivity, or robots.txt failures) or page-level (404, 5xx, or a soft 404), and repeated errors slow crawling and can drop pages from the index.
Two levels: host errors and URL errors
Google splits crawl failures into two buckets. Host-level (availability) problems show up in the Crawl Stats report under host status, which runs three checks: robots.txt fetching, DNS resolution, and server connectivity. If robots.txt does not return a valid file or a clean 404, Google will slow or stop crawling the site until it gets an acceptable robots.txt response. A DNS failure means Googlebot cannot resolve your hostname at all, so nothing gets crawled.
URL-level errors are per-page, and how Google reacts depends on the HTTP status it gets back:
4xx(like404or410): Google treats the content as gone. Previously indexed pages are removed and crawl frequency for that URL gradually decreases.5xxand429: read as server overload. Google decreases the crawl rate in proportion to how many URLs error out, keeps indexed pages at first, then eventually drops them, and ramps crawling back up once the server returns2xx.- Soft
404: the page returns200 (success)but the content reads like an error page. Search Console flags it in the Page Indexing report.
One note on currency: the old standalone “Crawl Errors” report is gone. That data now lives in two places, the Crawl Stats report (host availability) and the Page Indexing report (which URLs indexed and why the rest did not).
How it affects your traffic
Crawl errors are a direct tax on organic traffic: pages Googlebot cannot fetch cannot rank, and repeated 5xx or robots.txt failures throttle crawling site-wide, so even healthy pages get recrawled and updated more slowly. A soft 404 is worse because it quietly pulls a real, working page out of the index with no obvious "broken" signal. Our Technical SEO work reads your Crawl Stats and Page Indexing reports, fixes the server, DNS, and robots.txt issues behind host errors, and clears URL-level errors so crawl budget goes to the pages you actually want indexed.
Get Technical SEO that moves the needle
We turn terms like this into ranked pages and qualified pipeline. Start with a free Initial SEO Strategy.