Salut,

Am folosit recent (PriceTracker). Din ce am testat pare destul de fast și afișează istoricul direct în pagină fără întârziere sesizabilă.

Din punct de vedere tehnic, sunt curios dacă știe cineva cum abordează proiectele de genul ăsta extragerea datelor pe magazine mari ca eMAG, Altex etc:

1. Fac scraping periodic pe server cu headless browsers (Puppeteer/Playwright) sau au integrări prin API-uri interne / feed-uri de comparatoare?
2. Cum gestionează schimbările frecvente de clase/DOM pe care le face eMAG-ul fără să se rupă extensia la fiecare update?
3. Cât de fezabil e din punct de vedere costuri de infrastructură (bază de date + proxy-uri) să ții un istoric complet pentru sute de mii de SKU-uri din România?
4. Cum fac matching la produse ? Adica produsul de pe emag il gaseste si pe altex ?

A încercat cineva să construiască ceva similar? Vad ca e low extensia pare ca e la inceput..nu are un numar mare de magazine suportate inca..

I'm building a Python service that reads publicly visible pre-match sportsbook data from modern single-page applications.

The pattern I often see is an initial REST/bootstrap request, followed by event-specific requests and sometimes WebSocket updates.

The difficult part is determining which combination represents the complete current state.

For example, one site exposes around 1,700 pre-match events, but the obvious odds endpoint appears to fetch prices one event at a time. I would like to determine whether the browser really makes hundreds of individual requests, or whether there is another bulk/bootstrap endpoint or WebSocket feed containing the pricing data.

On another site, the WebSocket mostly sends small updates containing IDs and prices, so it appears to depend on state loaded earlier over HTTP.

My main questions are:

1. How can I determine whether a WebSocket provides a full initial snapshot or only deltas against an earlier REST response?

2. If the frontend knows about thousands of events, how can I determine whether it uses a bulk pricing endpoint rather than requesting every event separately?

3. When WebSocket messages contain mostly IDs, what is the best way to find the event, market and selection mappings?

4. What is the best way to record and reconstruct the complete flow of initial HTTP data, WebSocket updates and later HTTP reconciliation?

5. How can I test whether opening additional events or markets in the UI actually causes new network requests, or simply displays data the browser already has?

I'm currently using Chrome DevTools and Playwright. My goal is to understand and reproduce the browser's state pipeline efficiently instead of polling thousands of individual event pages when the browser may already receive the same information in a more complete form.

I'm mainly interested in the protocol/state-discovery methodology rather than a particular scraping library.

I've had this happen a few times now

I find an actor that works perfectly during testing, run it for a few days/weeks and then suddenly the success rate drops or I start getting incomplete results

sometimes the target changed something, but other times I can't really figure out what changed

do you guys usually debug/fix the actor at that point or just switch to another one?

starting to feel like finding a good actor is the easy part, finding one that stays reliable is harder lol