ChatGPT-User is an OpenAI user agent that fetches web pages when someone using ChatGPT or a Custom GPT triggers an action. It is not a crawler that sweeps the web on a schedule. Each request goes back to a person’s action.
At a glance
- Operator
- OpenAI
- Purpose
- fetching pages on user actions in ChatGPT and Custom GPTs
- robots.txt token
ChatGPT-User- Respects robots.txt
- robots.txt rules may not apply, says OpenAI
- IP list
- yes (JSON)
Last checked:
What is ChatGPT-User used for?
OpenAI runs several bots with separate jobs. GPTBot collects content for training generative models. OAI-SearchBot surfaces websites in ChatGPT search results. ChatGPT-User visits pages when a user action in ChatGPT or a Custom GPT calls for it.
OpenAI publishes the user agent string and a JSON file with the IP ranges:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot
| Bot | Operator | Purpose | robots.txt token | robots.txt according to the operator | IP list published? |
|---|---|---|---|---|---|
| Google-Extended | Controls Gemini training and grounding (no crawler of its own) | Google-Extended | robots.txt token only; fetching is done by existing Google crawlers | n/a (no user agent of its own) | |
| GPTBot | OpenAI | Training | GPTBot | Disallowing it signals content should not be used to train foundation models | yes (openai.) |
| OAI-SearchBot | OpenAI | Search (ChatGPT) | OAI-SearchBot | opted-out sites are not shown in ChatGPT search answers | yes (openai.) |
| ChatGPT-User (this term) | OpenAI | User-triggered fetch | ChatGPT-User | robots.txt rules may not apply, says OpenAI (user-initiated) | yes (openai.) |
| ClaudeBot | Anthropic | Training | ClaudeBot | yes, Anthropic says its bots honor robots.txt | yes (claude.) |
| Claude-SearchBot | Anthropic | Search (Claude) | Claude-SearchBot | yes, Anthropic says its bots honor robots.txt | yes (claude.) |
| PerplexityBot | Perplexity | Search (Perplexity), not used for model training | PerplexityBot | controllable via robots.txt; the docs contain no explicit compliance statement | yes (perplexity.) |
| Applebot-Extended | Apple | Controls AI training (does not crawl, says Apple) | Applebot-Extended | robots.txt token only, applies to content Applebot already crawled | n/a (no crawler of its own) |
| CCBot | Common Crawl | Open web crawl archive | CCBot | can be blocked via robots.txt, says Common Crawl | yes (index.) |
| Meta-ExternalAgent | Meta | Training foundation AI models, improving products | meta-externalagent | can be blocked via robots.txt, says Meta | not stated by the operator |
A selection of ten AI bots from major operators, not a complete list. All values come from each operator's own documentation (as of 8 October 2026). The table does not show whether a bot actually complies with robots.txt; it only reflects what operators state. User agents can be spoofed; published IP lists or reverse DNS identify genuine requests. Sources: OpenAI: Overview of OpenAI Crawlers; Anthropic: Does Anthropic crawl data from the web, and how can site owners block the crawler? (updated 2026-04-07); Perplexity: Perplexity Crawlers; Google: Google's common crawlers (last updated 2026-07-14); Google: Overview of Google crawlers and fetchers (last updated 2026-06-12); Google: Verifying Google crawlers and fetchers (IP range files); Apple: About Applebot (2026-09-04); Meta: Meta Web Crawlers; Common Crawl: CCBot. Last checked: .
Does robots.txt apply to ChatGPT-User?
OpenAI’s documentation says that because these actions are initiated by users, robots.txt rules may not apply. A rule for ChatGPT-User is therefore not a dependable way to keep the bot out. Anyone who needs to restrict access can additionally filter on the server, for example by the published IP ranges (an assessment, not an OpenAI recommendation).
Example
A request in a server log:
203.0.113.7 - - [06/Oct/2026:09:12:44 +0000] "GET /pricing/ HTTP/1.1" 200 18342 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot"
The IP address comes from a documentation range and is a placeholder.
Common mistakes
- Treating robots.txt as protection. According to OpenAI, robots.txt rules may not apply to ChatGPT-User because the actions are initiated by users.
- Blocking ChatGPT-User to leave ChatGPT search. OpenAI writes that ChatGPT-User is not used to determine whether content may appear in search. OAI-SearchBot covers that.
FAQ
Is ChatGPT-User a crawler? It does not browse the web on its own. It fetches a page when a person has asked something that needs it.
How does ChatGPT-User differ from GPTBot? GPTBot collects training data. ChatGPT-User serves a single user request.
Sources
- OpenAI Crawlers and fetchers – OpenAI Developersdevelopers.openai.com
Last updated: