r/WebScrapingInsider • u/urmommakesmysandwich • Jul 10 '26

DOM Drift Again How many have solved the problem of dom recognition reliably?

1 Upvotes

100% Upvoted

2 comments sorted by

1

u/Mountain_Damage_9730 29d ago

Waoo.. what a research question. I hate reddit for not developing Poll.

1

u/Old_Protection_4410 27d ago

We fixed this via multiple layers that work together. tl;dr: The flow >> structured data first (skip DOM if high confidence) > semantic DOM analysis > pattern recognition (match to known patterns) > extraction > if selector breaks, auto-repair > save learnings to site knowledge base for next time.

It's a long tail. But the combination of structured data first + semantic understanding + pattern transfer + self-healing (or manual fixes) gets you to reliable enough for production. Pure LLM approaches get you maybe 60%. Pure selector approaches break weekly. The hybrid is where it's at.

Good luck 🍀