On-site search results pages: the page shoppers need and crawlers should not index
A customer typed three words into your search box and got a useful page. Google's crawler found the same page from a linked query, and then found ten thousand of them, each thin, each nearly identical, each competing with your real category pages. The page that helped a shopper is the same page that is eating your crawl budget.
Why search pages are a crawl problem, not a content problem
Google's own guidance on faceted navigation and crawl efficiency flags the same mechanism: a site that generates a URL for every filter combination or search query can present an effectively infinite crawl space, most of it duplicate. Baymard Institute's research on site search makes the user side clear too: shoppers use site search when navigation failed, so the results page is a recovery tool, and its value is in the moment, not as a landing page.
The cost is crawl budget and index quality. Google spends its crawl allowance on generated URLs that will never rank and never convert, and the real pages wait. Worse, when generated pages do get indexed, they dilute the site's topical focus: thousands of thin pages on queries nobody searched twice, sitting beside category pages a merchandiser actually built.
The fix: noindex, not a robots block
Noindex the search results pages. The distinction matters and Google's documentation explains it: a robots.txt block stops crawling, but a blocked URL can still appear in results when linked elsewhere, and the crawler never reads the instruction on the page itself. A noindex tag lets the page be crawled, read, and dropped from the index, while shoppers keep using it exactly as before.
Implement it at the template level so every generated search URL carries the tag, including paginated and sorted variants. Test it on one URL first and watch it in the search console coverage report before declaring the whole site fixed. Search pages generated by apps and plugins need the same treatment, and they are the ones teams forget because the template lives in the app, not the theme.
Make the page worth landing on, for the human who is already there
Noindexing is not an excuse for a dead end. A good search results page handles the no results case with alternatives, not an empty grid: the closest category, the bestsellers, and a prompt to rephrase. It shows the query it understood, because a shopper who typed a brand name and got unrelated products needs to know the search heard them.
Track search as a merchandising signal, not a vanity metric. Queries with no results are a product gap or a naming gap, and the fix is often a synonym mapping or a missing product, not a better search box. Queries that end in a purchase tell you what customers call things, which belongs in product titles and collection copy. The search log is the cheapest customer research most stores own and ignore.
Exceptions: the rare search page that earns its index slot
A few search pages deserve indexing, and they are the ones a merchandiser built on purpose: a curated landing page for a high volume query with real copy, real products, and a reason to exist as a destination. The test is whether a person would link to it and whether it answers a query better than the category page it would replace. If the answer is yes, it is a landing page and should be built as one, with a stable URL and copy of its own.
Everything else, every query a visitor typed once, stays noindexed. The rule for the team is simple: generated pages do not rank, curated pages do, and the difference is decided by a human with a reason, not by a URL parameter.
Keep reading: Out of stock pages: keep the URL, fix the signals, sell the wait, Faceted navigation: let shoppers filter everything, let Google index almost none of it.
Frequently asked questions
Why should site search results pages be noindexed?
Because every query generates a thin, near duplicate URL, and at scale that is thousands of pages competing with your real categories for crawl budget and index quality. Shoppers need the pages in the moment; search engines do not need them as landing pages. Noindex keeps the human use and removes the index problem.
Should I block search pages in robots.txt instead?
Usually no. A robots block stops crawling, but a blocked URL can still be indexed if it is linked elsewhere, and the crawler never sees the noindex instruction on the page. Noindex lets the page be crawled and dropped from the index while shoppers keep using it normally. Reserve robots blocks for URL spaces you never want crawled at all.
What should a no results search page show?
Alternatives rather than an empty grid: the closest category, bestsellers, and a prompt to rephrase, plus the query as the search understood it. A shopper who typed a brand name and got unrelated products needs to see what the search heard. Log the query too: a no results search is a product or naming gap worth fixing.
Are there search pages that should stay indexed?
A few, and only ones a merchandiser built deliberately: a curated landing page for a high volume query, with real copy, a stable URL, and products chosen for it. The test is whether a person would link to it and whether it answers the query better than the category page. Generated query URLs never meet that test.