Deutsch
Log data & formats

Access log

An access log is the file in which a web server records each request it handles, one line per request. A line typically shows when the request came in, which URL was requested, which HTTP status code the server returned, and who sent it.

What does an entry contain?

The operator decides the fields in the server configuration. In the widely used Combined Log Format, an entry holds the client IP address, a timestamp, the request line (method, path, protocol), the status code, the response size in bytes, the referer and the user agent. Apache writes this to logs/access_log by default, nginx to logs/access.log, and nginx uses the combined format unless told otherwise.

IP addresses can be personal data (CJEU, judgment of 19 October 2016, C-582/14, Breyer).

This is not legal advice.

One log line in Combined Log Format, field by fieldSample line from an access log in Combined Log Format, split into IP address, identity and user, timestamp, request line, status code, bytes, referrer and user agent. The user agent claims to be Googlebot, but the IP address comes from a range reserved for documentation. Only a DNS or IP list check shows whether Google made the request.access.log, one line (wrapped)203.0.113.7 - - [08/Oct/2026:10:15:32 +0200] "GET /en/glossary/ HTTP/1.1" 200 5120 "-""Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"IP address203.0.113.7client, or the proxy in betweenIdentity, user- -“-” means not availableTimestamp[08/Oct/2026:10:15:32 +0200]when the request arrivedRequest"GET /en/glossary/ HTTP/1.1"method, path, protocolStatus200status code of the responseBytes5120size without headers, “-” if emptyReferrer"-"referring page, none hereUser agentline 2what the client says about itselfThe user agent claims to be Googlebot, but the IP is in a range reserved for documentation (RFC 5737).Only a DNS or IP list check shows whether Google made the request.
Apache and nginx write the same fields in the same order in the “combined” format. IP addresses can be personal data (CJEU, judgment of 19 October 2016, C-582/14). Not legal advice. Sources: Apache HTTP Server: Log Files (Combined Log Format); nginx: Module ngx_http_log_module (log_format combined); RFC 5737: IPv4 Address Blocks Reserved for Documentation; CJEU: Press release 112/16 on judgment C-582/14.

Why do SEOs care?

The user agent field shows whether a request came from a search engine or AI bot. Together with the URL and status code, that tells you which pages a bot fetched and how the server answered. A user agent can be faked, so Google documents a check of the IP address through reverse and forward DNS lookups and published IP ranges. Apache also warns that log lines can contain client-supplied text without escaping, so write access to the log directory should be restricted.

Example

Text
66.249.66.1 - - [10/Oct/2026:13:55:36 +0000] "GET /products/ HTTP/1.1" 200 2326 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

This invented line shows a request for /products/ that the server answered with status 200.

Sources

  1. Log Files - Apache HTTP Server 2.4httpd.apache.org
  2. Module ngx_http_log_module - nginxnginx.org
  3. Verifying Googlebot and other Google crawlers - Google Search Centraldevelopers.google.com
  4. CJEU, judgment of 19 October 2016, C-582/14 (Breyer) – EUR-Lexeur-lex.europa.eu

Last updated: