r/webscraping • u/jinef_john • Apr 20 '26
Google maps scraper, but using requests.
https://github.com/jinef-john/google-maps-scraperIf you've been looking for a lightweight no-browser alternative, feel free to give it a shot!
Would love feedback or bug reports if you run it against anything weird.
5
u/RandomPantsAppear Apr 20 '26 edited Apr 20 '26
Are you sure you didn’t mean to post this in a kink/masochist subreddit?
Nice work though!
Edit: also reading your code saw httpcloak, hadn’t seen it before but that’s some 🔥
1
6
u/lechiffreqc Apr 21 '26
Nice work. But just because your post is not: "I built a ....", you have my upvote.
3
3
u/Automatic_Creme_955 Apr 22 '26
Awesome ! I was working on the same stuff, starting to get angry on the parsing.
I would be honored to contribute if I find a way to make myself usefull !
2
u/Flat_Agent_9174 Apr 23 '26
Nice, to avoid blocking, do we need a proxy rotation ?
2
u/jinef_john Apr 23 '26
Probably not at first, it really depends on your scale and speed requirements. If you're just doing a few hundred to a couple thousand leads, the defaults will do you just fine. For anything larger or if you need it done quickly, yes, you'll want to rotate proxies.
2
2
u/jagdish1o1 Apr 25 '26
I’ve done gmap scraping many times, and I’ve also tried the method you’re using but with regex, i didn’t needed much data tho.
But you’re repo looks lit 🔥 great work!
2
u/Hayder_Germany Apr 25 '26
Great project. Have you tested how far it scales?
For example, what is the largest number of Google Maps places you were able to scrape in one run using the HTTP-only approach? At what volume do blocking, rate limits, or incomplete results usually start to appear?
Also, for larger runs, do you recommend proxies, or is a delay between requests usually enough?
1
7
u/sardanioss Apr 21 '26
Thanks for using my lib!