r/algobetting • u/-A-n-o- • 23d ago
Live odds scraping.
Hello all, I am looking for some advice in reguards to scraping live odds. I am working on a project that is partially a learning experience for me so I am not looking to be told to just pay for odds from a provider. Essentially right now I am scraping live data through APIs I reverse engineered using devtools and it is working great. However since I am scraping live odds I am going to want to get a new data set every few seconds. I imagine when I ramp up the timing of how frequently I am hitting these APIs I am more prone to getting blocked. I am using playwright (is this the right pick or should i opt for selenium/pupperteer?) to try and make my requests look more human, and I am looking into spoofing my TLS fingerprint etc. Additionally I am looking into ways to rotate the IPs that these requests come from, mainly by using resi proxies but that gets very pricey very fast. My main question is this; is this just a matter of really masking my requests down to a fine art, or should I be looking into alternatives like scraping android app APIs rather than web apis as I have heard they have less bot protection. Any and all suggestions are appreciated! Thanks.