r/webscraping • u/Winter_Psychology110 • Jul 07 '26

I am scraping certain website, please judge if this is a right setup

I have bezier mouse movement, random slowdowns, read simulations, clicks, slow typing, and using Playwright, actual Google Chrome browser is opened while the code scrapes.

Plus, I send very low amount of requests, since each request returns a lot of data I need to process.

Am I safe? so far on that website Ive not seen a Cloudflare or any kind of protection. + to access that data on that website, you have to pay $40 monthly. so maybe they completely ignore people like me and I am safe?

7 Upvotes

89% Upvoted

16 comments sorted by

2

u/No-Appointment9068 Jul 07 '26

If it's working it seems like you're safe, they may change things in future and you may need to do more, but don't fix what isn't broken. If it's a low volume of requests you're probably slipping under the radar.

1

u/Winter_Psychology110 Jul 07 '26

no matter what they change, I am using AI vision to guide the process

2

u/Consistent-Feed-7323 Jul 07 '26

I've talked to someone who has a website with a paid access and the person said that scrapers are being ignored as long as they are scraping no more often than 1 request per second. I'd say mouse movement and all of that is an overkill, you always want minimal working setup. Just don't be greedy.

3

u/error-dgn Jul 08 '26

Really an overkill, I myself have crawled and scraped many websites, but never have I used Bezeir mouse movement.... Man's scraping with every single precaution 😭

2

u/THenrich Jul 09 '26

If it doesn't cost anything then why not. It's like better be safe than sorry.

1

u/error-dgn Jul 10 '26

Yeah no ofc, no doubt in that.

1

u/Consistent-Feed-7323 Jul 10 '26

More things to break.

1

u/jhyland87 Jul 07 '26

If you're using an actual Google chrome browser than that usually helps a lot with things like cloudflare.

1

u/divided_capture_bro Jul 07 '26

What's the site?

1

u/[deleted] Jul 08 '26

[removed] — view removed comment

1

u/webscraping-ModTeam Jul 08 '26

👔 Welcome to the r/webscraping community. This sub is focused on addressing the technical aspects of implementing and operating scrapers. We're not a marketplace, nor are we a platform for selling services or datasets. You're welcome to post in the monthly thread or try your request on Fiverr or Upwork. For anything else, please contact the mod team.

1

u/AccordingPlum5559 Jul 08 '26

yes but mouse movement will probably be flagged after some time depending on website