An ETag (entity tag) is an HTTP response header that identifies a specific version of a resource. When the resource changes, the ETag changes. Clients and crawlers use it to ask whether their stored copy is still current, which saves transferring the content again.
How does an ETag work?
- The server sends the resource with
ETag: "abc123". - The client stores the response together with the ETag.
- On the next request, the client sends
If-None-Match: "abc123". - If the resource is unchanged, the server answers
304 Not Modifiedwithout a body, and the client reuses its copy. Otherwise it sends the new version with a new ETag.
The value is any quoted string. Servers often derive it from a hash of the content, a modification time or a revision number. A weak ETag starts with W/ and signals that two versions are equivalent in meaning but not byte for byte. A strong ETag, without the prefix, means they are identical.
ETags also prevent conflicting edits. A client can send the ETag with If-Match, and the server answers 412 Precondition Failed when the stored version has changed in the meantime.
| Header / status | Request counterpart | What it does | Relevance for Google's crawlers |
|---|---|---|---|
Cache-Control | – (directives can also appear in requests) | lists directives for caches, e.g. max-age (fresh for N seconds), no-cache (reuse only after revalidation), no-store (do not store) | max-age is optional and helps crawlers decide when to recrawl the URL; Google does not support other caching directives |
ETag (this term) | If-None-Match | identifier for the current version of a resource | supported and recommended by Google because it has no date formatting issues; takes precedence when both validators are present |
Last-Modified | If-Modified-Since | date and time at which the server believes the resource was last modified | supported; the date must follow the HTTP format, e.g. Fri, 4 Sep 1998 19:15:56 GMT |
304 Not Modified | response to a conditional GET or HEAD request | signals that the stored copy is still valid; the response has no body | the crawler tells the next system that the content is the same as at the last crawl; Search may recalculate signals, otherwise no effect on indexing |
Google suggests setting both ETag and Last-Modified, since other applications use them too. Whether a Google crawler uses caching depends on its product: Googlebot uses it when recrawling URLs for Google Search. According to Google's December 2024 blog post, only 0.017% of all fetches could be served from cache. Sources: Google Crawling Infrastructure: Overview of Google crawlers and fetchers, HTTP caching (last updated 2026-06-12); Google Search Central Blog: Crawling December: HTTP caching (2024-12-09); Google Crawling Infrastructure: HTTP status codes, network and DNS errors (last updated 2026-02-04); MDN: HTTP caching; RFC 9111: HTTP Caching; RFC 9110: HTTP Semantics (8.8.2 Last-Modified, 8.8.3 ETag, 15.4.5 304). Last checked: .
ETag and Google’s crawlers
Google’s crawling infrastructure supports ETag with If-None-Match as defined by the HTTP caching standard. It also supports Last-Modified with If-Modified-Since, and Google recommends ETag because it avoids date formatting problems. When both headers are present, Google says its crawlers use the ETag value. Individual crawlers may implement caching differently. A correct 304 saves the server from sending the body again.
Example
HTTP/1.1 200 OK
ETag: "33a64df551425fcc55e4d42a148795d9f25f89d4"
GET /page HTTP/1.1
If-None-Match: "33a64df551425fcc55e4d42a148795d9f25f89d4"
HTTP/1.1 304 Not Modified
Sources
- ETag – MDN Web Docsdeveloper.mozilla.org
- Overview of Google crawlers and fetchers (HTTP caching) – Google Search Centraldevelopers.google.com
- HTTP caching – MDN Web Docsdeveloper.mozilla.org
- RFC 9110: HTTP Semantics (sections 8.8.3 ETag, 13.1 preconditions) – IETFrfc-editor.org
Last updated: