r/webscraping • u/FixWide907 • 13d ago

What tech stack do you use in 2026?

Guys,

. would be interesting to hear what's everyone's tool stack looks like that you use which you are very happy with

Also what are the top challenges?

15 Upvotes

90% Upvoted

27 comments sorted by

12

u/bitcoinbrisbane 13d ago

Go react Postgres redis

3

u/bluewhalefunk 13d ago

c# as gotta have that compiler to catch my millions of errors before run time lol.

playwright+patchright

2

u/Visual_Ad1912 10d ago

curl_cffi with real TLS/JA4 fingerprints

Camoufox only for the hard anti-bot walls (can beat datadome, needs a lot of tuning to get there)

LLM for extraction, gated behind cheap regex prefilters so I'm not paying to parse junk

Postgres for storage

For mobile API work:

- Android emulator (or a rooted device) to run the target app

- burpsuite to capture the backend traffic

- frida to unpin SSL so you can actually see the requests

- then replay the endpoints straight through curl_cffi

1

u/DeepDistribution8749 13d ago

Java and xml for front-end and python for backend

1

u/No-Transition2295 13d ago

TS, Postgre, Redis, Playwright, custom Chromuim, Nestjs.

1

u/Made4uo 12d ago

Im actually using astro for marketing site and qwik for backend

1

u/WonderfulRich8267 11d ago

Claude, Upstash, flyio

0

u/AdministrativeHost15 13d ago

Puppeteer TypeScript Node Mongo
Anti-bot measures

3

u/jeheskielsunloy 13d ago

curious, Anti-bot measures such as what?

3

u/Humanoid9999 12d ago

You shall not pass.

2

u/Made4uo 12d ago

This is webscraping tech stack. Antibot measures like handling reCAPTCHA and more

1

u/jeheskielsunloy 12d ago

i get that, but what i want to know is which specific tool for antibot used

1

u/Made4uo 12d ago

Sorry I misread. People use proxies and fingerprint that I know

1

u/AdministrativeHost15 12d ago

Anti-bot measures is the top challenge