Ecommerce index bloat is what happens when search engines index far more URLs from a store than the store has genuinely useful pages. Filter combinations, sort orders, session parameters, empty categories and duplicate product paths accumulate until the valuable pages are a minority of what is indexed.
It is a structural problem rather than a content one. The individual URLs are not spam, they are simply near-identical variants that nobody decided should exist as separate pages.
Bloat and crawl budget are related but distinct. Bloat describes what ends up in the index; crawl budget describes the crawling capacity consumed getting there. Bloat is usually the visible symptom of the second problem.
How to tell whether you have it
- Indexed page count substantially exceeds the number of products and categories you actually have.
- Search Console shows large numbers of pages under excluded reasons such as duplicate or crawled-not-indexed.
- A
site:search surfaces filtered, sorted or parameter URLs rather than category pages. - The indexed count keeps rising while the catalogue stays roughly the same size.
- New products take a long time to appear in search results.
That last symptom is the commercially expensive one. Where crawling is consumed by low-value variants, genuinely new pages wait.
Where the URLs come from
Faceted navigation
The dominant source. Each filter combination can produce a distinct crawlable URL, and combinations multiply rather than add. Four filters with five options each generate hundreds of variants of one category.
Sorting and view parameters
Sort order, items-per-page and grid or list toggles frequently generate separate URLs holding the same products in a different arrangement.
Duplicate product paths
Products reachable through multiple category paths, or through both a bare and a category-scoped URL, produce several URLs for one product.
Empty and near-empty pages
Categories with no products, tag archives holding a single item, and attribute archives generated automatically by the platform.
Internal search results
Search result pages are occasionally crawlable and can be generated endlessly by any crawler that follows a search form.
Fixing it in the right order
The sequence matters, because removing URLs from the index takes time and doing it in the wrong order removes pages you needed.
- Stop the source first. There is no point pruning while the mechanism keeps generating new variants.
- Decide what should be indexable. Usually categories, products, and a small number of filter combinations with genuine demand.
- Prevent crawling of the rest at source rather than relying on noindex alone, since a noindexed page still consumes crawling.
- Correct canonicals on variants that must remain reachable for users.
- Remove internal links to URLs you do not want crawled. Crawlers follow links regardless of directives.
- Let it settle. De-indexing is gradual, and repeated intervention makes it harder to tell what worked.
One caution: blocking a URL in robots.txt prevents crawling but does not remove an already-indexed page, because the crawler can no longer see the noindex. Where a page is already indexed, allow crawling until it drops, then block.
What good looks like
- Indexed count approximates products plus categories plus supporting pages, not a multiple of it.
- Excluded reasons in Search Console are dominated by deliberate exclusions rather than duplicates.
- New products are indexed within days rather than weeks.
- Category pages rather than filtered variants appear for category queries.
Frequently asked questions
How many indexed pages should a store have?
Roughly your products plus categories plus supporting content. If the indexed count is several times that, the surplus is almost certainly filter, sort or duplicate URLs rather than pages anyone searches for.
Is index bloat a penalty?
No. It is an efficiency problem, not a sanction. The damage is indirect: crawling is spent on low-value URLs, signals split across duplicates, and genuinely valuable pages compete with variants of themselves.
Should I use noindex or robots.txt?
Noindex to remove something already indexed, since the crawler must be able to see the directive. Robots.txt to prevent crawling of URLs that were never indexed. Using robots.txt on an already-indexed page can leave it indexed indefinitely.
How long does it take to clear?
Weeks to months, depending on how often the URLs are crawled. Large stores clear slowly because low-value URLs are crawled infrequently by definition. Treat it as a trend rather than an event.
Consulting CTA
If your indexed page count is many times your catalogue size, or new products are slow to appear in search, book an SEO consultation to identify the sources of bloat and agree a safe sequence for clearing it.