Deutsch
CMS, operations & special files

Staging environment

Also known as: dev environment, test environment, staging site, staging server

A staging environment (also called a dev or test environment) is a copy of a website or application where changes are tested before they go live. It is meant to resemble production as closely as practical. For SEO, the copy must not appear in search results.

What does a staging environment mean for SEO?

Teams use staging to try new templates, redirects or content before visitors see them, including an SEO check before launch. If the staging domain is reachable by Google and unprotected, it can get indexed and create duplicates of the live site.

Google names two reliable ways to keep a page out of search: noindex or password protection. A robots.txt file does not do this. Google states that for noindex to work, the page must not be blocked by robots.txt, otherwise the crawler never sees the rule and the page can still appear in results. Password protection avoids the problem, because crawlers get no content at all.

One hosting provider points out that a staging site may not mirror production perfectly, for example in caching behavior (secondary source).

A staging environment is often blocked completely in robots.txt with Disallow: /. If that file is carried over at launch, crawlers are locked out of the new site. Putting the production robots.txt on staging and protecting the environment with a password or an IP allowlist keeps it out of search and lets a crawler test realistic behavior before launch (practical experience).

Example

A staging response that blocks indexing through a header:

HTTP
HTTP/1.1 200 OK
X-Robots-Tag: noindex, nofollow

With HTTP password protection, crawlers receive a 401 status instead of the page.

Common mistakes

  • Blocking staging only with robots.txt. Google can still index the URLs.
  • Using noindex on a page that robots.txt blocks. The rule only works if the crawler can fetch the page.
  • Carrying noindex or the password over to production. The live site then drops out of search or becomes unreachable. This follows from the two effects above and is practical experience.

More on the searchVIU blog

Sources

  1. Block Search indexing with noindex – Google Search Centraldevelopers.google.com
  2. Introduction to robots.txt – Google Search Centraldevelopers.google.com
  3. What Is a Staging Site? – WP Engine (secondary)wpengine.com
  4. RFC 9309: Robots Exclusion Protocol – IETFrfc-editor.org

Last updated: