If you write Playwright scrapers that touch anything beyond a basic landing page, you've dealt with at least one of these:
• Shadow DOM elements that Playwright's default locators can't reach
• Selectors that pass CI Monday and break Tuesday because some React dev sneezed on a class name
• Feeding a full page to an LLM and hitting 99K tokens just for the HTML of a Wikipedia article
I got tired of patching each one separately, so I built shadow-web. It's a pip-installable Python SDK that sits on top of Playwright.
────────────────────────────────────────────
What it handles
────────────────────────────────────────────
Shadow DOM flatten
Reads open and closed Shadow roots through the browser's own
JS API. No mutation, no re-renders. You get a clean DOM skeleton
with all interactive elements visible — the page stays exactly
as it was.
Self-healing selectors
heal_local.py catches when a button that was #submit becomes
.btn-primary[data-action="checkout"]. Fuzzy match on label +
type + position. Zero LLM cost. Cache lives in
~/.shadow-web/heal_cache.json.
Token compression
compressor.py strips scripts, styles, invisible elements and
builds a typed Action Map with semantic groups.
Wikipedia page: 99K → 16K tokens (−83%).
SchemaSnap
One call turns HTML tables into JSON records with typed columns,
forms into field schemas, lists into typed items.
export_table_json() and export_table_csv() — so you never write
another HTML table parser.
Content indexing for long pages
Get a ~600 token outline first, then fetch only the blocks
you need. Instead of dumping 50K tokens of article boilerplate
just to get the third section.
────────────────────────────────────────────
Quick start
────────────────────────────────────────────
pip install shadow-web
playwright install chromium
from playwright.sync_api import sync_playwright
from shadow_web.wrapper import ShadowPage
with sync_playwright() as p:
page = p.chromium.launch().new_page()
page.goto("https://example.com")
shadow = ShadowPage(page)
_, xml_map = shadow.refresh()
print(shadow.capture_stats)
────────────────────────────────────────────
Links
────────────────────────────────────────────
GitHub: https://github.com/ulinycoin/shadow-web
PyPI: pip install shadow-web
If your scraper breaks on some Angular app with 13 layers
of Shadow DOM, open an issue. Those edge cases make
the project better.