# Duplicate content Category: [Crawling & Indexing](https://seo-geo-glossary.com/en/glossary/category/crawling-indexing/) › [Canonicals & duplicate content](https://seo-geo-glossary.com/en/glossary/category/crawling-indexing/canonicals-duplicate-content/) · Also known as: duplicate pages, duplicated content **Duplicate content** is identical or very similar content that can be reached at more than one URL, either within one website or across several domains. Google groups such URLs, picks one as the canonical URL and usually shows only that one in search results. ## How does Google handle duplicate content? Google clusters pages with the same main content and selects the one it considers most complete and useful as canonical. Google calls this process canonicalization. The other URLs in the cluster count as duplicates and, according to Google, are crawled less often to reduce load on the site. There is no penalty attached. Google writes that some duplicate content on a site is normal and not a violation of its spam policies. The practical issues are different. Google may pick a URL you did not intend, it gets harder to track how your content performs in search results, and on large sites crawling duplicates takes time away from pages that matter. The exception is copying other sites' content to manipulate rankings. Google's spam policies treat republishing scraped content without adding value as spam. ## What causes duplicate content? Most duplicates are a side effect of how a site is built. Google names several typical sources: - protocol and host variants (HTTP and HTTPS, with and without `www`) - URL parameters for filters, sorting or tracking - regional variants with the same text, such as US and UK pages - separate mobile and desktop URLs - demo or test environments that are accidentally crawlable CMS archives, tag pages, print versions and manufacturer descriptions reused by many online stores add to the list. ## Example The same product page is reachable at three URLs: ```txt https://www.example.com/shoe-123 https://www.example.com/shoe-123?color=blue http://example.com/shoe-123 ``` The HTTP and non-`www` versions redirect with a 301 to the first URL. The parameter URL stays reachable and carries a canonical tag: ```html ``` ## How do you fix duplicate content? Google lists signals of different strength. Redirects and `rel="canonical"` are strong signals, sitemap inclusion is a weak one. All of them are hints. The most reliable setup is one where redirects, canonical tags, sitemaps and internal links all point to the same URL. ## Common mistakes - **Blocking duplicates in robots.txt.** Google advises against using robots.txt for canonicalization. Google may still index disallowed URLs, just without their content. - **Using the URL removal tool.** The removal tool hides all versions of a URL from Search. Google says it is not meant for canonicalization. - **Conflicting signals.** If the sitemap lists one URL and `rel="canonical"` on the same page names another, the hints contradict each other. Google recommends pointing every method to the same URL. - **Declaring a URL fragment canonical.** Google generally does not support fragments such as `#color-blue`, so a URL with a fragment does not work as a canonical URL. ## FAQ **Does Google penalize duplicate content?** Not for technical duplicates within a site. Content copied from other sites to manipulate rankings falls under Google's spam policies. **How do I find duplicate content on my site?** The Search Console page indexing report lists URLs with statuses such as "Duplicate without user-selected canonical." A crawl of the site reveals identical titles, identical body text and parameter variants. ## At a glance - **Type:** identical or very similar content at more than one URL - **Affects:** choice of the canonical URL, crawling - **Penalty?:** none for technical duplicates (according to Google) - **Canonicalization signals:** 301 and rel="canonical" strong, sitemap weak - **Docs:** [Google: What is URL canonicalization](https://developers.google.com/search/docs/crawling-indexing/canonicalization) Last checked: October 8, 2026 *Diagram: Duplicate cluster and canonical URL.* Google groups identical or very similar pages into a cluster and picks one canonical URL. According to Google, redirects and rel="canonical" are strong signals and the sitemap is a weak one. Your own preference remains a hint. ## Not to be confused with - [Keyword cannibalization](https://seo-geo-glossary.com/en/glossary/keyword-cannibalization/index.md): describes several different pages on one site that target the same query with their own content. Duplicate content means the same content at more than one URL. ## Related terms - [Canonical tag](https://seo-geo-glossary.com/en/glossary/canonical-tag/index.md) - [Near-duplicate content](https://seo-geo-glossary.com/en/glossary/near-duplicate-content/index.md) - [Thin content](https://seo-geo-glossary.com/en/glossary/thin-content/index.md) ## Sources - [What is URL canonicalization – Google Search Central](https://developers.google.com/search/docs/crawling-indexing/canonicalization) - [How to specify a canonical URL with rel="canonical" and other methods – Google Search Central](https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls) - [Spam policies for Google web search (scraping) – Google Search Central](https://developers.google.com/search/docs/essentials/spam-policies) - [Page indexing report – Search Console Help](https://support.google.com/webmasters/answer/7440203?hl=en) --- Last updated: October 8, 2026 URL: https://seo-geo-glossary.com/en/glossary/duplicate-content/