The Crawl Stats report is a report in Google Search Console that summarizes Googlebot’s requests to a site. It shows how many requests Google made, how large the downloads were, how fast the server answered and which status codes came back.
What does the report show?
It sits under Settings in the property and covers the past 90 days. The headline figures are total crawl requests, total download size and average response time, plus a host status for availability problems. Requests can be broken down by response code, file type, crawl purpose (discovery or refresh) and Googlebot type, for example smartphone, desktop, image or AdsBot.
Limits
Google aims the report at advanced users and says a site with fewer than 1,000 pages usually does not need it. It is available only for root-level properties, meaning domain properties and URL-prefix properties at the root. Google notes that some requests may not be counted, so the figures can differ from server logs. Only Google’s crawlers appear, and the report does not show where a crawl came from (Search Engine Roundtable, 2023).
Example
If the share of 5xx responses grows over several days and the average response time rises with it, the server may be slowing down crawling. The report gives the cue to look at the server log for details. The example is invented.
Sources
Last updated: