Deutsch
Servers, HTTP & caching

HTTP header

Also known as: HTTP headers

An HTTP header is a name-value pair that a client or server sends along with an HTTP request or response to pass extra information, such as the content type, caching rules or indexing directives. Header names are not case-sensitive. With HTTP/2 and later they are written in lowercase.

Which kinds of headers exist?

MDN groups headers by context:

  • Request headers describe the client or what it wants, such as User-Agent or Accept.
  • Response headers describe the answer or the server, such as Location or Set-Cookie.
  • Representation headers describe the body, such as Content-Type and Content-Encoding.
  • Payload headers carry information independent of the representation, such as Content-Length.

The status code, for example 200 or 404, is not a header; it belongs to the status line of the response.

HTTP versions compared
VersionTransportConcurrent requestsHeader fieldsUse by Google's crawlers
HTTP/1.1 (RFC 9112)TCPno multiplexing layer; pipelining possible, responses come back in request order; often several TCP connections for parallel requestswhitespace-delimited text fields; often repetitive and verbose according to RFC 9113supported; Google's default version
HTTP/2 (RFC 9113)TCPyes, many requests interleaved on one connection; TCP-level packet loss still stalls all streamsbinary frames, HPACK compressionsupported since November 2020; no ranking boost; opt out with status code 421
HTTP/3 (RFC 9114)QUIC over UDPyes, over independent QUIC streams; one blocked stream does not hold up the othersQPACK compressionnot stated by Google; the crawler documentation lists only HTTP/1.1 and HTTP/2

All three versions carry the same HTTP semantics (methods, status codes, headers); they differ in how messages are transferred. Google's crawlers pick the version with the best crawling performance and may switch between crawling sessions. Google says HTTP/2 may save computing resources on both sides. Server logs show which version a request used. Sources: RFC 9112: HTTP/1.1; RFC 9113: HTTP/2; RFC 9114: HTTP/3; RFC 9000: QUIC; Google Crawling Infrastructure: Overview of Google crawlers and fetchers (last updated 2026-06-12); Google Search Central Blog: Googlebot will soon speak HTTP/2 (2020-09-17). Last checked: .

Headers that matter for SEO

  • X-Robots-Tag carries robots directives such as noindex in the HTTP response. According to Google, any rule usable in a robots meta tag can also be given this way. It works for files without HTML, such as PDFs and images.
  • Link can declare a canonical URL for non-HTML files.
  • Cache-Control and ETag control caching and conditional requests.
  • Location sends the target of a redirect.
  • Content-Type tells crawlers what kind of file they received.
  • Strict-Transport-Security enforces HTTPS in browsers.

Custom headers used to start with X-. That convention was deprecated in 2012 by RFC 6648, but X-Robots-Tag kept its name.

Example

HTTP
HTTP/1.1 200 OK
Content-Type: application/pdf
X-Robots-Tag: noindex
Cache-Control: max-age=3600

How can I view HTTP headers?

Run curl -I https://example.com/ in a terminal, or open the Network tab of the browser’s developer tools and select a request.

Sources

  1. HTTP headers – MDN Web Docsdeveloper.mozilla.org
  2. Robots meta tag and X-Robots-Tag – Google Search Centraldevelopers.google.com
  3. Consolidate duplicate URLs (Link header for canonicals) – Google Search Centraldevelopers.google.com
  4. RFC 6648: Deprecating the X- Prefix – IETFrfc-editor.org

Last updated: