# Crawl-to-refer ratio

Category: [AI Bots & Crawlers](https://seo-geo-glossary.com/en/glossary/category/ai-bots-crawlers/) › [Managing & measuring AI bots](https://seo-geo-glossary.com/en/glossary/category/ai-bots-crawlers/managing-measuring-ai-bots/) · Also known as: crawl-to-referral ratio

The **crawl-to-refer ratio** (also written crawl-to-referral ratio) compares how many pages a platform crawls with how often it sends users to websites. Cloudflare defines it as how many pages a platform crawls compared with how often it drives users to a website. A ratio of 1,000:1 means 1,000 crawled pages per referred visit.

## Which figures exist?

Cloudflare published these ratios for January and July 2025:

- Anthropic: about 286,930:1, then about 38,066:1
- OpenAI: about 1,217:1, then about 1,091:1
- Perplexity: about 55:1, then about 195:1
- Google: about 3.8:1, then about 5.4:1

A lower number means more visitors come back per crawled page. The values show orders of magnitude for one measurement period and one provider's network. They change from month to month.

## What are the limits?

The ratio depends on the referral side being measurable. Cloudflare's Radar methodology notes that traffic from Claude's native app carries no Referer header and that the same likely holds for other native apps. The referral counts then only cover web-based tools, so the ratios may be overstated by an unknown amount. Visits without a referrer are not counted as referrals.

## Example

A service fetches 10,000 pages of a site in a month and sends 10 visitors back. The ratio is 1,000:1. (Constructed example.)

## At a glance

- **Metrics:** HTML requests from a platform's bots divided by HTML requests with a Referer from that platform
- **Measures:** how many pages a platform crawls compared with how often it sends users to websites
- **Unit:** x:1 (crawls per referred visit)
- **Assessment:** lower value = more visits per crawl; apps without a Referer can inflate the value
- **Data source:** Cloudflare network data (Radar)
- **Docs:** [Cloudflare Blog: The crawl-to-click gap](https://blog.cloudflare.com/crawlers-click-ai-bots-training/)

Last checked: October 8, 2026

## Related terms

- [CCBot](https://seo-geo-glossary.com/en/glossary/ccbot/index.md)
- [Pay per crawl](https://seo-geo-glossary.com/en/glossary/pay-per-crawl/index.md)

## Sources

- [The crawl-to-click gap: Cloudflare data on AI bots, training, and referrals – Cloudflare Blog](https://blog.cloudflare.com/crawlers-click-ai-bots-training/)
- [AI Search crawl-to-refer ratio on Cloudflare Radar – Cloudflare Blog](https://blog.cloudflare.com/ai-search-crawl-refer-ratio-on-radar/)

---

Last updated: October 8, 2026
URL: https://seo-geo-glossary.com/en/glossary/crawl-to-refer-ratio/