Site search (also on-site search or internal search) is the search function inside a website or online store. In SEO this entry is about the store’s own search box and the result pages it generates. The site: operator in Google is a different thing.
Why is site search an SEO topic?
Result pages usually have URLs such as /search?q=running+shoes. If any link points to such a URL, Google can follow it. Users can type any term, so the number of possible URLs has no natural limit. That matches the pattern of a crawl trap. Google describes related duplicate-URL problems for parameters such as session IDs and filters in its e-commerce URL guidance, which does not mention internal search result pages specifically.
Two controls are common:
- robots.txt. A
Disallowrule for the search path stops crawling. Google states that robots.txt is not a mechanism for keeping a page out of Google, so a blocked URL can still be indexed without a description if other sites link to it. - noindex. A robots meta tag with
noindexkeeps a result page out of the index. It works only if the page stays crawlable, so combining both on the same URL cancels the effect.
Site search is also a source of data. The terms visitors type show wishes and assortment gaps (assessment).
Example
User-agent: *
Disallow: /search
Disallow: /*?q=
This stops crawling of search result URLs. If specific results should rank, build dedicated category or topic pages for them rather than indexing free-form search queries.
Common mistakes
- Disallow and noindex on the same URL. A
noindexworks only if Google can fetch the page, so a robots.txt block cancels its effect. - Treating robots.txt as index protection. Google states that robots.txt is not a mechanism for keeping a page out of Google. A blocked URL can still be indexed without a description if other sites link to it.
- Leaving the search path unmanaged. Users can type any term, so the number of possible result URLs has no limit once a link points to one of them.
FAQ
Should internal search result pages be indexed? In most cases no. Curated category pages cover the topics worth ranking for.
How do you do a site search on Google?
Type site:example.com plus a term into Google. That operator filters Google’s results to one domain, and Google notes it does not necessarily return all indexed URLs.
Sources
- Introduction to robots.txt – Google Search Centraldevelopers.google.com
- Designing a URL structure for ecommerce sites – Google Search Centraldevelopers.google.com
- The site: search operator – Google Search Centraldevelopers.google.com
Last updated: