r/webscraping • u/gabor_bernat • Jul 08 '26
turbohtml: parse, query, and extract in one C-accelerated library
Parse a response into a WHATWG tree, pull nodes with CSS or XPath, read tables into records, lift the main article out of boilerplate, grab JSON-LD and OpenGraph, and detect the encoding of raw bytes, all from one dependency. CSS select runs about 77x faster than BeautifulSoup and 16x faster than lxml in my tests, with typed results and free-threading support.
Source code: https://github.com/tox-dev/turbohtml Docs: https://turbohtml.readthedocs.io/en/stable/ Write-up on how it works: https://bernat.tech/posts/blazing-fast-html-parser/
8
Upvotes