# Bot verification

Category: [Log Files](https://seo-geo-glossary.com/en/glossary/category/log-files/) › [Identifying & managing bots](https://seo-geo-glossary.com/en/glossary/category/log-files/identifying-managing-bots/)

**Bot verification** is the check whether a crawler really belongs to the operator named in its user agent. In SEO and server work it usually means verifying Googlebot and other crawlers in your own logs, not the bot checks of chat platforms or CAPTCHAs.

## Why verify?

The user agent is a text value that any client can set. It is not proof of identity. If you grant crawlers special access, exempt them from rate limits or analyze log files, you want to separate real crawler requests from imitations (see fake Googlebot).

## Methods

**Reverse and forward DNS.** Google describes a two-step check. Run a reverse DNS lookup on the IP address from your log. The host name must end in `googlebot.com`, `google.com` or `googleusercontent.com`. Then run a forward lookup on that host name and confirm that it resolves to the original IP address. Both steps belong together.

Google documents which host names to expect per crawler group:

| Crawler group | Reverse DNS pattern |
|---|---|
| Common crawlers (e.g. Googlebot) | `crawl-***-***-***-***.googlebot.com` or `geo-crawl-***-***-***-***.geo.googlebot.com` |
| Special-case crawlers | `rate-limited-proxy-***-***-***-***.google.com` |
| User-triggered fetchers | `***-***-***-***.gae.googleusercontent.com` or `google-proxy-***-***-***-***.google.com` |

**Published IP ranges.** Google publishes crawler IP ranges as JSON files in CIDR format, split by crawler group, for example `common-crawlers.json` for Googlebot. Since March 2026 the files live under `/static/crawling/ipranges/` on developers.google.com. Matching an IP address against these lists can be automated (see crawler IP ranges).

**Signatures.** Cloudflare verifies bots through a cryptographic signature (Web Bot Auth, for which the IETF has set up a working group called webbotauth) or through a published IP list combined with a stable user agent or reverse DNS. Cloudflare also requires that the bot obeys robots.txt and keeps request rates reasonable.

## Example

Checking a suspicious IP address (placeholder values):

```bash
host 192.0.2.10
host crawl-example.googlebot.com
```

Only if the first command returns a host name with a matching ending and the second resolves back to 192.0.2.10 does the request come from Google.

## Common mistakes

- Trusting the user agent string alone.
- Running only the reverse lookup and skipping the forward check.
- Expecting every Google request to follow robots.txt. Google states that user-triggered fetchers generally ignore robots.txt rules because a user requested the fetch.
- Scripts that still point to the old IP file location under `/search/apis/ipranges/`.

## FAQ

**How do I verify Googlebot?** Use the reverse and forward DNS check above, or compare the IP address with Google's published ranges.

*Diagram: Is it really Googlebot?.* Both DNS lookups have to match. For large-scale lookups, Google recommends matching against its published IP lists automatically. Sample values from Google’s documentation.

## Related terms

- [Reverse DNS lookup](https://seo-geo-glossary.com/en/glossary/reverse-dns-lookup/index.md)
- [Bot traffic](https://seo-geo-glossary.com/en/glossary/bot-traffic/index.md)
- [Bot management](https://seo-geo-glossary.com/en/glossary/bot-management/index.md)
- [Web Bot Auth](https://seo-geo-glossary.com/en/glossary/web-bot-auth/index.md)
- [Crawler IP ranges](https://seo-geo-glossary.com/en/glossary/crawler-ip-ranges/index.md)
- [Fake Googlebot](https://seo-geo-glossary.com/en/glossary/fake-googlebot/index.md)

## Sources

- [Verifying Google crawlers and fetchers – Google Crawling Infrastructure](https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests)
- [New location for the Google crawlers' IP range files – Google Search Central Blog (Mar 31, 2026)](https://developers.google.com/search/blog/2026/03/crawler-ip-ranges)
- [Google user-triggered fetchers – Google Search Central](https://developers.google.com/crawling/docs/crawlers-fetchers/google-user-triggered-fetchers)
- [Verified bots – Cloudflare Docs](https://developers.cloudflare.com/bots/concepts/bot/verified-bots/)
- [Web Bot Auth Working Group – IETF Datatracker](https://datatracker.ietf.org/wg/webbotauth/about/)

---

Last updated: October 8, 2026
URL: https://seo-geo-glossary.com/en/glossary/bot-verification/