# Crawl demand

Category: [Crawling & Indexing](https://seo-geo-glossary.com/en/glossary/category/crawling-indexing/) › [Crawling & crawl budget](https://seo-geo-glossary.com/en/glossary/category/crawling-indexing/crawling-crawl-budget/)

**Crawl demand** is the part of crawl budget that describes how much a crawler wants to crawl a site. The other part is the crawl capacity limit, which describes how much the server can handle. Google defines crawl budget as the set of URLs it can and wants to crawl.

## What does crawl demand depend on?

Every crawler has its own demand. AdsBot, for example, has a higher demand when a site runs dynamic ad targets, and Google Shopping has a higher demand for products in merchant feeds. For Googlebot, demand varies with a site's size, update frequency, page quality and relevance compared to other sites. Google names three factors:

- **Perceived inventory:** Without guidance, Google tries to crawl all or most URLs it knows. Many duplicates or unwanted URLs waste crawling time. Google says this is the factor site owners can influence most.
- **Popularity:** URLs that are more popular on the internet tend to be crawled more often.
- **Staleness:** Google wants to recrawl often enough to pick up changes.

A site move can also raise demand for a while, because Google reprocesses the content under the new URLs. If demand is low, Google crawls less even when the capacity limit has not been reached.

## Example

A shop generates thousands of near-identical filter URLs. Google knows them, so the perceived inventory is large and part of the crawling time goes to the variants. Consolidating or blocking them leaves more time for the pages that matter (illustrative example).

## FAQ

**Does every site need to worry about it?**
Google's guide is aimed at sites with about 1 million or more unique pages that change weekly, or about 10,000 or more pages that change daily. Google calls these numbers a rough estimate.

*Diagram: Crawl budget from capacity and demand.* Google defines crawl budget as the set of URLs that Google can and wants to crawl. According to Google, the perceived URL inventory is the factor a site can control the most.

## Not to be confused with

- [Crawl rate](https://seo-geo-glossary.com/en/glossary/crawl-rate/index.md): is the number of requests a crawler sends to a site within a period of time. For Google it follows from how much the server can handle (the crawl capacity limit) and how much Google wants to crawl.

## Related terms

- [Crawl budget](https://seo-geo-glossary.com/en/glossary/crawl-budget/index.md)
- [Crawl rate](https://seo-geo-glossary.com/en/glossary/crawl-rate/index.md)
- [Crawl depth](https://seo-geo-glossary.com/en/glossary/crawl-depth/index.md)
- [Recrawl](https://seo-geo-glossary.com/en/glossary/recrawl/index.md)

## Sources

- [Optimize your crawl budget – Google Crawling Infrastructure](https://developers.google.com/crawling/docs/crawl-budget)
- [Reduce the Google crawl rate – Google Crawling Infrastructure](https://developers.google.com/crawling/docs/crawlers-fetchers/reduce-crawl-rate)

---

Last updated: October 8, 2026
URL: https://seo-geo-glossary.com/en/glossary/crawl-demand/