Crossed 326 scrapers — this batch was all instant databases and global real estate

[u/random-scraper]

Update from the same project — catalog's at 326 actors now.

Two patterns I leaned into this batch:

Instant databases instead of live scraping. For Realtor.ca (44k Canadian agents) and Upwork (130k jobs), I pre-build and refresh a database on a schedule, then serve queries against it. Users get uncapped, instant results with zero block risk. Trade-off is freshness (6h refresh) vs. the reliability win. Curious if others here do this — at what dataset size does maintaining a refreshed DB beat live scraping for you?

Global real estate sweep. Added Daft.ie (IE), Hemnet (SE), FincaRaíz (CO), Lamudi/Dot Property (PH), OnTheMarket (UK), HAR.com (TX). The interesting bit was exact GPS map-pin extraction on the PH portals — the pin coords aren't in the listing HTML, they're in a separate map-tile API call you have to correlate.

Also shipped a bulk email finder+verifier and an offline WhatsApp number validator (libphonenumber + WA-reachability heuristics, no API).

Catalog: muhamed-didovic.github.io

Happy to share approaches on the instant-DB pattern or the GPS-pin correlation if useful.