The HTTP status code 401 Unauthorized means a request lacks valid authentication credentials for the requested resource. The server asks the client to authenticate before it will return the content.
At a glance
- Status code
- 401
- Class
- 4xx (client error)
- Duration
- can be repeated with valid credentials
- In Google
- treated like other 4xx except 429, content is not used, indexed URLs are removed
- Specification
- RFC 9110, section 15.5.2
Last checked:
How does 401 differ from 403?
With 401, the credentials are missing or invalid. With 403 Forbidden, the credentials are valid but the client lacks permission for the action (MDN). A 401 response comes with a WWW-Authenticate header that names the authentication scheme the client must use.
Typical triggers are a wrong password, an expired session, a missing Authorization header or a password-protected area such as a login or a Basic Auth prompt.
How does Google treat a 401?
Google handles 401 like the other 4xx codes except 429. Its crawlers tell the next processing system that the content does not exist, Google does not use content from such URLs and does not index them. Previously indexed URLs that return a 4xx are removed from the index.
Google also says not to use 401 or 403 to limit crawl rate, because 4xx codes other than 429 have no effect on it.
For SEO, content behind a login stays out of the index. If a public page is protected by mistake, it can drop out of the index.
Example
GET /admin HTTP/1.1
Host: example.com
HTTP/1.1 401 Unauthorized
WWW-Authenticate: Basic realm="Admin"
Common mistakes
- Missing
WWW-Authenticateheader. According to RFC 9110, the server must send this header when the request lacks valid credentials. - Using 401 to limit the crawl rate. Google advises against using 401 and 403 for this because they have no effect on it. 500, 503 or 429 are the codes Google names for that purpose.
- Protecting a public page by accident. Search Console reports the status “Blocked due to unauthorized request (401)”. The page was blocked to Googlebot by a request for authorization.
Sources
- 401 Unauthorized – MDN Web Docsdeveloper.mozilla.org
- How HTTP status codes affect Google's crawlers – Google Crawling Infrastructuredevelopers.google.com
- RFC 9110: HTTP Semantics, section 15.5.2 (401 Unauthorized) – IETFrfc-editor.org
- Page indexing report – Search Console Help (Blocked due to unauthorized request (401))support.google.com
- Reduce the Google crawl rate – Google Crawling Infrastructuredevelopers.google.com
Last updated: