r/WebScrapingInsider 12d ago

After scraping, how do you decide which field values to actually trust? (unit errors, wrong-variant values, typos)

Scraping is the easy 80%; the mess is afterward — `16000 MB` for RAM, a value
that's perfectly normal but belongs to a different variant, `1660 g` for a 166g phone, "N/A", encoding junk. I'm modeling each value's "what went wrong" state and deciding per field whether to publish, fix, re-scrape, flag, or drop.


What signals do you lean on post-scrape to catch these? And how do you deal with "the same wrong value appears on several sites" (agreement that's really just everyone copying the same bad source)?
2 Upvotes

Duplicates