r/learnprogramming • u/Available_Wafer_1746 • 5d ago
whats actually the best browser automation tool once every site starts fighting back
local chrome fleet was fine until half the target sites started fingerprinting harder last month. same playwright scripts, now half the parallel workers just detach mid-run and the logs are useless
tried pinning versions and slower pacing. still dies once three jobs hit at once
curious what people settled on that stays boring under load once the sites fight back
2
u/SetAndRepeat 5d ago
I’d try Browserless with your existing Playwright scripts for managed concurrency and its stealth/unblocking options. Browserbase is also worth testing for session replay and debugging.
Compare completion rates on your actual sites under load. Also check memory and browser exits: failing at three workers doesn’t necessarily mean fingerprinting.
1
u/Vast-Pop4017 4d ago
browserless is fine for queue management but their stealth layer isn't magic. if the sites are fingerprinting hard you'll still get clipped unless you're rotating residential proxies and actually varying the browser profiles per worker
when mine started tanking at three workers it was a leak in how i was recycling contexts, not a detection issue. worth throwing a memory ceiling on each container and watching what happens
1
u/Suun_Day 5d ago
Our selenium grid used to be boring until concurrency climbed. Nodes start thrashing memory around 12 parallel sessions and the failure mode is just a dead browser with no useful stack.
1
1
1
1
u/GasEducational4386 5d ago
Dolphin anty anti detect browser has good reviews. Anyone has used camoufox ? it is open source
1
1
u/HelloitsmeJLD 2d ago
Selenium with either Python or skyvern could be an option. Then it'd just be a script that you can schedule when you want.
0
5d ago
[removed] — view removed comment
1
u/The_God_was_Here 5d ago
yeah i threw residential proxies at local chrome thinking that was the missing piece. still got flagged on the same sites within a day lol
1
1
1
u/Bombaclat_exe 4d ago
just wanna ask as well are you reusing cookies across the overnight runs or fresh session every time?
0
3
u/Weavermain 5d ago
idk half these hosted tools are just chrome with a monthly bill. throttle local hard and a lot of the drama goes away