# Amazonbot

Category: [AI Bots & Crawlers](https://seo-geo-glossary.com/en/glossary/category/ai-bots-crawlers/) › [Other AI bots](https://seo-geo-glossary.com/en/glossary/category/ai-bots-crawlers/other-ai-bots/)

**Amazonbot** is Amazon's web crawler. According to Amazon it is used to improve Amazon products and services, and the data may be used to train Amazon AI models. The robots.txt token is `Amazonbot`.

## What is Amazonbot for?

Amazon describes two more bots on the same page. **Amzn-SearchBot** improves search in Amazon products and services and makes content eligible for features such as Alexa. **Amzn-User** supports user actions, for example when Alexa needs current information to answer a question. Amazon says that Amzn-SearchBot does not crawl for generative AI training.

## How do you control Amazonbot?

Amazon states that Amazonbot respects the Robots Exclusion Protocol and honors user agent directives. It also respects the page-level `noarchive` robots meta tag, which Amazon says keeps a page out of model training. If a fresh robots.txt cannot be fetched, a cached copy from the last 30 days is used.

`Crawl-delay` is not supported by Amazonbot, Amzn-SearchBot or Amzn-User. For Amzn-User, Amazon notes that it may not follow all robots.txt directives, because the requests are user-initiated.

## Example

```robots
User-agent: Amazonbot
Disallow: /internal/
```

And as a robots meta tag on a single page:

```html
<meta name="robots" content="noarchive">
```

## Common mistakes

- **Setting Crawl-delay.** Amazon lists the directive as not supported for Amazonbot, Amzn-SearchBot and Amzn-User.
- **Blocking only Amazonbot.** According to Amazon, each user agent setting is independent of the others. Amzn-SearchBot and Amzn-User need their own entries.
- **Expecting an immediate effect.** According to Amazon, changes to robots.txt may take about 24 hours to be reflected in its systems.

## FAQ

**Can I limit Amazonbot's request rate with Crawl-delay?**
No. Amazon lists the directive as not supported.

**Does robots.txt control every Amazon bot?**
Not fully. Amazon says Amzn-User may not follow all directives.

## At a glance

- **Operator:** Amazon
- **Purpose:** Improving Amazon products and services; per Amazon also AI model training
- **robots.txt token:** Amazonbot
- **Respects robots.txt:** yes, Robots Exclusion Protocol; no Crawl-delay
- **IP list:** [yes, separate Amazon page](https://developer.amazon.com/amazonbot/ip-addresses/)
- **Docs:** [Amazon Developer: Amazonbot](https://developer.amazon.com/amazonbot)

Last checked: October 8, 2026

## AI bots compared

| Bot | Operator | Purpose | robots.txt token | robots.txt according to the operator | IP list published? |
|---|---|---|---|---|---|
| Google-Extended | Google | Controls Gemini training and grounding (no crawler of its own) | Google-Extended | robots.txt token only; fetching is done by existing Google crawlers | n/a (no user agent of its own) |
| GPTBot | OpenAI | Training | GPTBot | Disallowing it signals content should not be used to train foundation models | yes (openai.com/gptbot.json) |
| OAI-SearchBot | OpenAI | Search (ChatGPT) | OAI-SearchBot | opted-out sites are not shown in ChatGPT search answers | yes (openai.com/searchbot.json) |
| ChatGPT-User | OpenAI | User-triggered fetch | ChatGPT-User | robots.txt rules may not apply, says OpenAI (user-initiated) | yes (openai.com/chatgpt-user.json) |
| ClaudeBot | Anthropic | Training | ClaudeBot | yes, Anthropic says its bots honor robots.txt | yes (claude.com/crawling/bots.json) |
| Claude-SearchBot | Anthropic | Search (Claude) | Claude-SearchBot | yes, Anthropic says its bots honor robots.txt | yes (claude.com/crawling/bots.json) |
| PerplexityBot | Perplexity | Search (Perplexity), not used for model training | PerplexityBot | controllable via robots.txt; the docs contain no explicit compliance statement | yes (perplexity.com/perplexitybot.json) |
| Applebot-Extended | Apple | Controls AI training (does not crawl, says Apple) | Applebot-Extended | robots.txt token only, applies to content Applebot already crawled | n/a (no crawler of its own) |
| CCBot | Common Crawl | Open web crawl archive | CCBot | can be blocked via robots.txt, says Common Crawl | yes (index.commoncrawl.org/ccbot.json) |
| Meta-ExternalAgent | Meta | Training foundation AI models, improving products | meta-externalagent | can be blocked via robots.txt, says Meta | not stated by the operator |

Sources: [OpenAI: Overview of OpenAI Crawlers](https://developers.openai.com/api/docs/bots); [Anthropic: Does Anthropic crawl data from the web, and how can site owners block the crawler? (updated 2026-04-07)](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler); [Perplexity: Perplexity Crawlers](https://docs.perplexity.ai/guides/bots); [Google: Google's common crawlers (last updated 2026-07-14)](https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers); [Google: Overview of Google crawlers and fetchers (last updated 2026-06-12)](https://developers.google.com/crawling/docs/crawlers-fetchers/overview-google-crawlers); [Google: Verifying Google crawlers and fetchers (IP range files)](https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests); [Apple: About Applebot (2026-09-04)](https://support.apple.com/en-us/119829); [Meta: Meta Web Crawlers](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/); [Common Crawl: CCBot](https://commoncrawl.org/ccbot). Last checked: October 8, 2026

## Related terms

- [Amzn-SearchBot](https://seo-geo-glossary.com/en/glossary/amzn-searchbot/index.md)
- [Amzn-User](https://seo-geo-glossary.com/en/glossary/amzn-user/index.md)
- [AI crawler](https://seo-geo-glossary.com/en/glossary/ai-crawler/index.md)
- [robots.txt](https://seo-geo-glossary.com/en/glossary/robots-txt/index.md)
- [Crawl-delay](https://seo-geo-glossary.com/en/glossary/crawl-delay/index.md)
- [Meta robots tag](https://seo-geo-glossary.com/en/glossary/meta-robots-tag/index.md)
- [User agent](https://seo-geo-glossary.com/en/glossary/user-agent/index.md)

## Sources

- [Amazonbot – Amazon Developer](https://developer.amazon.com/amazonbot)

---

Last updated: October 8, 2026
URL: https://seo-geo-glossary.com/en/glossary/amazonbot/