- anthropic-aiHistorical
anthropic-ai is a user agent name that appears in many robots.txt files to keep Anthropic's crawlers out.
- ChatGPT agent
ChatGPT agent is the OpenAI feature that lets ChatGPT carry out multi-step tasks on the web for a user.
- ChatGPT-User
ChatGPT-User is an OpenAI user agent that fetches web pages when someone using ChatGPT or a Custom GPT triggers an action.
- ClaudeBot
ClaudeBot is the web crawler of Anthropic, the company behind the AI assistant Claude.
- Claude-SearchBot
Claude-SearchBot is an Anthropic crawler that navigates the web to improve the quality of search results for Claude users.
- Claude-User
Claude-User is the user agent Claude, Anthropic's AI assistant, uses to access websites when a person asks Claude a question.
- GPTBot
GPTBot is OpenAI's web crawler for collecting publicly available content that may be used to train its generative AI models.
- OAI-SearchBot
OAI-SearchBot is a crawler run by OpenAI.
- PerplexityBot
PerplexityBot is the crawler of the AI search engine Perplexity.
- Perplexity-User
Perplexity-User is a Perplexity user agent that visits web pages when a person asks Perplexity a question and the answer needs a page.