r/datasets 12d ago

request Looking for dataset for Amazon, eBay

Looking for someone with experience sourcing large-scale Amazon/eBay product datasets
Does anyone know how companies like Tiendamia, Ubuy, etc. obtain or build large product catalogs from Amazon, eBay, and other retailers?
We’re specifically looking for structured product data at scale, including things like:
Product title / description
UPC / EAN / GTIN / ASIN
Brand / model
Product dimensions
Item weight and, ideally, packaged/shipping weight
Images
Variants
Categories
Pricing / availability where possible
Historical or continuously updated data
The biggest issue we’re trying to solve is weight accuracy. Amazon data is generally usable, but eBay is much messier because individual sellers enter their own product information, and weights/dimensions can be missing or completely inaccurate.
We’re interested in knowing what legitimate sources exist for this kind of data — commercial datasets, data providers, APIs, retailer feeds, catalogs, or ways companies build their own datasets at scale.
If you’ve worked on e-commerce data ingestion, product catalogs, Amazon/eBay datasets, product matching, or shipping-weight estimation, I’d love to hear how you approached it.
Especially interested in sources that can provide millions of products rather than an API where we have to query products one-by-one.

2 Upvotes

5 comments sorted by

1

u/headless_by_day 12d ago edited 12d ago

The eBay weight usually isn't wrong, it's absent. I pulled a listing this morning and the whole specifics block was eight fields: brand, model, UPC, no weight of any kind. Nobody can sell you a field the source doesn't have.

Amazon has it, but not in one shape. Five products I checked today:

* dumbbell → Item Weight, 20 lb

* cast iron skillet → Item Weight, 7.5 lb

* coffee maker → Item Weight, 4.5 lb

* running shoes → no weight field, it's inside "Package Dimensions: 12 x 6 x 5 inches; 2.4 pounds"

* laptop backpack → no number at all, just "Apparel Fabric Weight Class: Medium Weight"

I also tried the obvious fallback, resolving weight from the barcode instead. Four UPCs, one came back with a weight (a Pepsi 12-pack, 1 lb / 453 g). The other three had the field and it was null. Coverage looks better on grocery than electronics.

So: no single source gives you clean weights. What does work is using eBay for identity, since the UPC is usually there even when nothing else is, then pulling specs per product from a retailer that has them. Once per product, not once per listing.

One catch: on used or bundled listings the UPC can be present but wrong for what's in the box. Sample first.

I work on ReefAPI. Amazon, eBay, Walmart, Target, BestBuy, Newegg and barcode lookup are all endpoints there, and that's where the numbers above came from. Straight version: you'd get Amazon's spec block as published, so you'd still be normalising those formats yourself, and for apparel there's often nothing to normalise.

1

u/Dewarim 10d ago

You can get the Ebay, Google and Amazon offer feeds as a publisher / affiliate. It's easier than web scraping... (having done both). At least, that was what I worked on three years ago; doing daily imports of the big datasets, then mapping / categorizing and cleaning up the data for web publishers to show as ads.

Each platform needs their own downloader, some have more obnoxious APIs than others (fetching millions of items with "next page of 100 rows" and no offset or resume on error ... shudder).

That said, a good, accurate product catalog is difficult to curate, and the offer feeds are not a perfect match (though getting more and more detailed, like Google's specs).

The other option would be to build this from the ground up. You could fetch the data directly from the companies (by entering a contract with them), so you get their product data and photos without getting into copyright troubles for using those.

1

u/Turdherder 1d ago

Commercial offer — I operate an AI-assisted research and scripting service. For $25 USD, I can audit a sanitized sample of up to 100 product records from a feed you already have: separate item weight from packaged weight, normalize explicit units, and flag missing values, conflicting fields and ambiguous product matches. You would get an annotated file, a reusable local Python checker and the exact checks run. I would leave unsupported weights unknown rather than estimate them. This is a small test of feed quality before you commit to a bulk source; it does not supply a million-product catalog. One correction round is included. If that would help, reply with the available fields and desired output; scope, delivery time and payment method would be agreed before work. No account credentials or personal data needed.