# Rate limiting

Category: [Log Files](https://seo-geo-glossary.com/en/glossary/category/log-files/) › [Identifying & managing bots](https://seo-geo-glossary.com/en/glossary/category/log-files/identifying-managing-bots/)

**Rate limiting** controls how many requests a client may send to a server or API within a set time. Clients that exceed the limit are slowed down or rejected. MDN names two purposes, keeping individual clients from using up too many resources and mitigating attacks such as denial of service.

## How does it work?

The server counts requests per sender, commonly per IP address, API key or user account. Wikipedia lists five common algorithms, token bucket, leaky bucket, fixed window counter, sliding window log and sliding window counter. When the limit is reached, a web server typically answers with status 429 Too Many Requests.

RFC 6585 defines 429 as "the user has sent too many requests in a given amount of time". The response may include a `Retry-After` header with the wait time. A 429 response must not be stored by a cache. The RFC leaves open how servers identify senders or count requests.

## What it means for SEO

Rate limits also affect crawlers. Google lists 500, 503 and 429 as codes to return when Googlebot should crawl less in an emergency. Google advises against keeping this up for more than one to two days, because it may have a negative effect on how a site appears in Google products. A rate limit that is set too tight can therefore slow crawling longer than intended.

Your own SEO crawler can run into the same limits. Error responses then distort the audit, and healthy pages appear broken.

## Example

A response that asks the client to wait one hour:

```http
HTTP/1.1 429 Too Many Requests
Retry-After: 3600
```

## Common mistakes

- **Limiting verified crawlers across the board.** If the server answers Googlebot with 429, Google temporarily slows down crawling. A limit that is too tight therefore works like a throttle on crawling.
- **Serving 429 or 503 for a long time.** Google names these codes for emergencies lasting a few hours up to one to two days. Beyond that it may have a negative effect on how a site appears in Google products.
- **Letting 429 responses be cached.** According to RFC 6585, responses with status 429 must not be stored by a cache. Otherwise further clients would still see the block after the limit has expired (a conclusion).

## FAQ

**What does "rate limit exceeded" mean?** The client used up its request allowance for the current time window. Further requests are refused until the window resets.

## Related terms

- [429 Too Many Requests](https://seo-geo-glossary.com/en/glossary/429-too-many-requests/index.md)
- [503 Service Unavailable](https://seo-geo-glossary.com/en/glossary/503-service-unavailable/index.md)
- [Crawl rate](https://seo-geo-glossary.com/en/glossary/crawl-rate/index.md)
- [Crawl budget](https://seo-geo-glossary.com/en/glossary/crawl-budget/index.md)
- [Web application firewall](https://seo-geo-glossary.com/en/glossary/web-application-firewall/index.md)
- [Bot management](https://seo-geo-glossary.com/en/glossary/bot-management/index.md)
- [Fake Googlebot](https://seo-geo-glossary.com/en/glossary/fake-googlebot/index.md)

## Sources

- [RFC 6585, section 4: 429 Too Many Requests – IETF](https://www.rfc-editor.org/rfc/rfc6585.html)
- [Reduce the Googlebot crawl rate – Google Search Central](https://developers.google.com/search/docs/crawling-indexing/reduce-crawl-rate)
- [Rate limit – MDN Web Docs](https://developer.mozilla.org/en-US/docs/Glossary/Rate_limit)
- [Rate limiting – Wikipedia (secondary)](https://en.wikipedia.org/wiki/Rate_limiting)

---

Last updated: October 8, 2026
URL: https://seo-geo-glossary.com/en/glossary/rate-limiting/