Deutsch
Canonicals & duplicate content

Near-duplicate content

Also known as: near duplicates

Near-duplicate content is the SEO term (general usage, not a Google definition) for pages whose content is very similar to other pages without being identical. Often only a detail changes, such as a color, a city name or a text block. Google names no similarity threshold.

What does near-duplicate content mean for SEO?

Google clusters pages whose main content is the same or very similar and picks one of them as the canonical, based on the signals it collected. It treats a canonical preference as a hint, not a rule. Some duplication on a site is normal, and Google says it is not a violation of its spam policies.

Crawlers measure similarity with adjustable thresholds. Screaming Frog uses 90 percent by default, which is a setting of that tool and not a Google value.

Duplicate cluster and canonical URLFour URL variants of the same category page form a cluster: an HTTP version, a version without www, one with the tracking parameter gclid and one with a sort parameter. Google picks https://www.example.com/dresses/ as the canonical URL. Below are the signals in order of strength according to Google: redirect strong, rel="canonical" strong, sitemap weak.DUPLICATE CLUSTERCANONICAL URLhttp://www.example.com/dresses/protocol variant (HTTP)https://example.com/dresses/host variant (no www)https://www.example.com/dresses/?gclid=ABCDtracking parameterhttps://www.example.com/dresses/?sort=pricesort function (parameter)Google pickshttps://www.example.com/dresses/usually shown in resultsand crawled mostregularlyWHAT INFLUENCES THE CHOICE, BY STRENGTH ACCORDING TO GOOGLE1redirect to the preferred URLstrong2rel="canonical" (HTML link or HTTP header)strong3URL listed in the sitemapweakGoogle also prefers HTTPS over HTTP.Schematic. The preference is a hint. Google may choose a different URL as canonical.Several signals pointing to the same URL are stronger together.
Google groups identical or very similar pages into a cluster and picks one canonical URL. According to Google, redirects and rel="canonical" are strong signals and the sitemap is a weak one. Your own preference remains a hint. Sources: Google Search Central: What is URL canonicalization; Google Search Central: How to specify a canonical URL with rel="canonical" and other methods.

Example

A service company runs 200 location pages with the same text, and only the city name changes. To a crawler and to Google, these are near duplicates.

What helps?

  • Merge pages that serve the same purpose.
  • Set a canonical on the preferred version.
  • Give each page content that is useful on its own.
  • Review crawler hits by hand, because some similarity is legitimate.

Sources

  1. What is URL canonicalization – Google Search Centraldevelopers.google.com
  2. How to specify a canonical URL with rel="canonical" and other methods – Google Search Centraldevelopers.google.com

Last updated: