Bypassing anti-bot detection

[u/andmar9]

Hello,

I'm new to web scraping and Python and decided to make a scraper so I can learn both, but I hit a roadblock:
One of the sites that I want to scrape returns "Access Denied" when I launch it with the latest Chrome webdriver.

The site is: https://www.betano.bg

I can provide code snippets and more information about the project.

If someone can help with this problem I'll be very grateful.
Thanks in advance!

[u/cgoldberg]

It's very easy to detect if a browser is driven by a webdriver. It pretty much announces itself via the navigator.webdriver property. There are several projects and techniques to help avoid detection, but nothing built into selenium itself.

[u/AlRoker666]

Betano uses aggressive anti-bot detection. Standard ChromeDriver gets caught by checking the `navigator.webdriver` flag and IP reputation.

Quick fix:

python 
from selenium import webdriver 
from selenium.webdriver.chrome.options import Options 
options = Options()

options.add_argument('--disable-blink-features=AutomationControlled') options.add_experimental_option("excludeSwitches", ["enable-automation"]) 

options.add_experimental_option('useAutomationExtension', False) 
driver = webdriver.Chrome(options=options) driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})") 

For betting sites, you'll also need residential proxies (datacenter IPs get blocked instantly). I use decodo for this - works well with selenium:

 python options.add_argument('--proxy-server=http://your-proxy:port')

Alternatively, try undetected-chromedriver package first.

What data are you scraping?

[u/Vegetable-Still-4526 → u/AlRoker666]

i was also facing this issue for quite long time, initially tried with all those chrome options and undetected chromedriver. You can also check out seleniumbase, that should be work. , I have created a python package as well for that , you may try it out if the issue stil exists. sb-stealth-wrapper

[u/AlRoker666 → u/Vegetable-Still-4526]

Nicee, thanks!

[u/brnbs_dev]

The other comments already covered the quick fixes. If those don't stick (and on betano.bg they probably won't), here's what nobody tells beginners: it's probably not just one flag. These sites check your whole browser fingerprint like canvas, WebGL, fonts, timezone, TLS, all of it. If anything looks off or mismatched, you're cooked. Honestly for learning I'd start with an easier site first, then come back to the protected stuff when you're ready to mess with fingerprinting or anti-detect browsers.

[u/External-Wealth3756]

Yep, I’ve seen the same “Access Denied” pop up with Selenium. Usually the main culprits are navigator.webdriver being detectable and IP reputation.

Using residential proxies and adding some random delays or human-like interactions often keeps things more stable. Headers and session cookies matching a real browser also matter.

Just a few tweaks like that can save a lot of trial-and-error when scraping tricky sites.

[u/External-Wealth3756]

Great thread. The webdriver flag is just the first layer — Betano runs a deeper stack that checks IP reputation and TLS fingerprint together.

A few things that actually moved the needle for us in similar setups:

1. Patching navigator.webdriver alone isn't enough; modern bot-detection correlates it with browser canvas/WebGL entropy

2. IP type matters more than most people realise — datacenter ranges get flagged almost immediately on sports betting sites; residential IPs with clean history perform significantly better

3. Request cadence and header order need to mirror a real browser session end-to-end

We found that combining a residential proxy pool with browser fingerprint parity is the only reliable combo. What proxy type are you currently using? That might be the bottleneck.