- Agent analytics
Agent analytics, in an SEO context, is the analysis of visits by AI bots and AI agents to a website, usually based on server or CDN logs.
- Blocking AI bots
Blocking AI bots means denying specific AI crawlers and fetchers access to a website.
- Crawl-to-refer ratio
The crawl-to-refer ratio (also written crawl-to-referral ratio) compares how many pages a platform crawls with how often it sends users to …
- Pay per crawl
Pay per crawl is a model in which a crawler pays for each page it fetches.
- Web Bot Auth
Web Bot Auth is a method for automated clients such as crawlers and AI agents to identify themselves to websites with a cryptographic …