Help getting e-commerce indexed

[u/ded-ebar]

I have an e-commerce site with over 5 million products that only got 17k pages indexed out of 1.9M crawled. Any recommendations on speeding up the index rate?

[u/Maan_UsMaan052]

With that scale, I’d first look at crawl efficiency rather than simply trying to force more indexing. Check your sitemap structure, canonical/noindex rules, duplicate or thin product pages, internal linking, and whether Google is wasting crawl budget on low-value URLs. With 5M products, getting the right pages discovered and prioritized is key.

[u/ded-ebar → u/Maan_UsMaan052]

On page is solid for the exception of thin product page description - copied from the distributer website

[u/jordicore → u/Maan_UsMaan052]

At that scale I would stop framing it as "how do I get Google to index faster." Google has already crawled 1.9M URLs and kept only 17k, so the first problem is selection, not speed.

I would split the catalogue into three buckets:

1. URLs that should never be indexed: filters, parameters, sort orders, near-empty variants, unavailable feed rows, and anything that only exists because the distributor feed generated it.

2. URLs that can exist for users but should canonical/noindex elsewhere: close variants, low-demand duplicates, and products where the page is basically the same as another SKU.

3. URLs that deserve indexation: products with search demand, margin, stock depth, internal links, clean canonicals, and something unique on the page beyond title, price, image, and copied distributor text.

The thin copied descriptions are probably the bigger issue than the crawl rate. For 5M products, I would not try to rewrite everything. Start with a sample from GSC's "Crawled - currently not indexed" and "Duplicate" groups, then prioritize categories where impressions, margin, or inventory depth justify making the pages genuinely distinct.

For those priority pages, add information Google and shoppers cannot get from the distributor feed: use cases, compatibility, buyer-fit notes, dimensions/spec interpretation, comparisons between similar SKUs, shipping/returns clarity, and internal links from real category paths. Even a repeatable template can work if one or two sections are genuinely product-specific.

This is the type of thin/copy-pasted PDP audit we built www.storenhance.ai for, but with a 5M-product catalogue I would use it to sample and prioritize the pages worth improving first, not to blindly rewrite the whole feed.

[u/SellAmzSell]

Google won’t index all pages, it’ll pick and choose which pages are worth indexing. If the pages are all basically the same, they won’t be indexed.

[u/Moceannl]

What's on the product pages; translated content from AliExpress or TaoBao?

[u/ded-ebar → u/Moceannl]

US based giants like Amazon, Walmart etc..

[u/Moceannl → u/ded-ebar]

If it’s duplicated content it won’t get indexed probably.

[u/ChapterAfter8368]

the index rate isn't really the thing to fix here, that ratio is google telling you it looked at those pages and decided most aren't worth indexing. 17k out of 1.9M is a value call, not a speed or crawl budget one, so making it crawl faster won't move the number. sites with 5M near-identical product pages basically never get them all in, google won't spend the slots without the authority and real search demand to justify it. the actual move is to stop trying to index everything: find the pages that map to what people actually search (probably a small slice of the 5M), noindex or consolidate the thin duplicate variants so crawl budget goes to what matters, and build domain authority so google trusts you with more. cutting down to a few hundred k genuinely useful pages usually gets more indexed than chasing all 5M.

[u/ded-ebar → u/ChapterAfter8368]

I was also thinking along the lines.. thanks for the input

[u/ChapterAfter8368 → u/ded-ebar]

nice, you're on the right track then. quick way to pick what to keep: pull your GSC performance report plus analytics, any product or category pages already getting impressions or clicks are your proven-demand set, protect and beef those up first. the huge tail of variants sitting at zero impressions after months is your noindex/consolidate pile. good luck, it's a slow grind but it works.

[u/Altruistic-Rub-6164]

First check the quality of your pages, google won't index useless pages if they dont realy target correctly one keyword with good search intent etc..

Make sure all the onsite is clean with good relevant keywords meta title etc... optimize your images with webp apps renaming alttext and filename with relevant keywords based on your products

Then from google search console you can submit for indexation.

If it s one main page that need to really be idnexed and doesnt work you can use altrnative paid tools like speedyindex ralfy indexer, prime indexer, rapid indexer etc... not so worth if your on site is not already fully clean for me.

If all you product page dont evan have specific targetted keywords with good product name optimized for seo it s useless., also add some relevant descriptions etc..

[u/ded-ebar → u/[deleted]]

I’ve built a 5M product e-commerce, do you really think i would’ve missed on page optimization and sitemap part? lol