The HTTP status code 429 Too Many Requests means a client has sent too many requests in a given time. The mechanism behind it is rate limiting, which a server can apply to the whole site or to single resources.
At a glance
- Status code
- 429
- Class
- 4xx (treated by Google like a server error)
- Duration
- temporary,
Retry-Aftercan state the wait time - In Google
- signal of an overloaded server, crawling slows down temporarily
- Caching
- must not be stored by a cache according to the RFC
- Specification
- RFC 6585, section 4
Last checked:
How does 429 work?
Servers usually identify the client by IP address, though limits can also target users or applications through cookies (MDN). The response may carry a Retry-After header that tells the client how long to wait before the next request. The value is either a number of seconds or an HTTP date.
| Status code | Meaning per RFC | How Googlebot reacts | Effect on the index | Retry-After |
|---|---|---|---|---|
| 429 Too Many Requests (this term) | too many requests in a given amount of time (rate limiting, RFC 6585) | treated as a sign of an overloaded server and as a server error; crawling temporarily slows down | indexed URLs are kept for now; if Googlebot sees the code on the same URL for multiple days, it may be dropped | RFC: may be included; Google: tells crawlers when they can retry |
| 500 Internal Server Error | unexpected condition on the server | the site's crawl rate drops in proportion to the number of URLs returning a server error | indexed URLs are kept for now; persistent errors get them removed, possibly after multiple days on the same URL | Google only describes it for 503 and 429 |
| 502 Bad Gateway | a gateway or proxy got an invalid response from the server behind it | same as 500 | indexed URLs are kept for now and removed if the error persists | Google only describes it for 503 and 429 |
| 503 Service Unavailable | temporarily unavailable, for example due to overload or maintenance | same as 500 | indexed URLs are kept for now; if Googlebot sees the code on the same URL for multiple days, it may be dropped | RFC: may be included and states the expected downtime; Google: tells crawlers when they can retry (seconds or a date) |
| 504 Gateway Timeout | a gateway or proxy didn't get a timely response from the server behind it | not listed individually by Google; for all 5xx codes, crawling temporarily slows down | indexed URLs are kept for now but eventually dropped (statement for all 5xx) | Google only describes it for 503 and 429 |
Google ignores any content it receives with a 5xx response. Once the server responds with 2xx again, Google gradually increases the crawl rate. To deliberately reduce crawling for a short time (a few hours up to 1-2 days), Google names 500, 503 or 429 and advises against doing so for longer. Google treats network timeouts, connection resets and DNS errors similarly to 5xx errors; according to Google, unreachable URLs are removed from the index within days. Sources: Google Crawling Infrastructure: How HTTP status codes affect Google's crawlers (last updated 2026-02-04); Google Crawling Infrastructure: Reduce the Google crawl rate (last updated 2026-10-06); Google Crawling Infrastructure: Debug network and DNS errors (last updated 2025-12-18); RFC 9110: HTTP Semantics, 15.6 Server Error 5xx and 10.2.3 Retry-After; RFC 6585: Additional HTTP Status Codes, 4. 429 Too Many Requests. Last checked: .
What does 429 mean for crawling?
Google treats 429 as a signal that the server is overloaded and handles it as a server error. Together with 5xx errors, it makes Google’s crawlers slow down temporarily and lowers the crawl rate for the site. Google states that URLs which persistently return a server error are removed from the index, without naming a time frame.
429 is the only 4xx code Google reads this way. All other 4xx codes have no effect on crawl rate.
A rate limit that is too strict can therefore throttle crawlers or block them from important pages.
Example
HTTP/1.1 429 Too Many Requests
Content-Type: text/html
Retry-After: 3600
The client should wait an hour before trying again.
Sources
- 429 Too Many Requests – MDN Web Docsdeveloper.mozilla.org
- RFC 6585, section 4: 429 Too Many Requests – IETFrfc-editor.org
- Retry-After header – MDN Web Docsdeveloper.mozilla.org
- How HTTP status codes affect Google's crawlers – Google Crawling Infrastructuredevelopers.google.com
Last updated: