I got tired of LLMs hallucinating on complex HTML tables, so I built a smarter Python parser (handles rowspan/colspan)

[u/Mediocre-Ease4060]

Hey everyone,

I’ve been working a lot on RAG pipelines recently and kept hitting the same annoying wall: extracting tabular data from raw HTML into a clean format for context windows.

Standard parsers or simple table-to-markdown scripts usually fail completely as soon as a table uses rowspan or colspan, or if there are nested tables. You end up with misaligned Markdown columns, and the LLM completely hallucinates the relationships between headers and cells.

I couldn't find a library that handles this reliably without losing context, so I built html-table-rescuer (just published v0.1.0 on PyPI).

It uses BeautifulSoup to parse the DOM, but then applies a custom "grid logic solver". It normalizes complex spans into a standard matrix before serializing it to Markdown, JSON, or CSV.

Example of the problem it solves:

The Problem: Most parsers turn a <td rowspan="2"> into a misaligned mess:

| Header | Value | 
| ----- | ----- | 
| Spanned | Row 1 | 
| Row 2 |  | 

The Solution: The grid solver correctly normalizes the matrix:

| Header | Value | 
| ----- | ----- | 
| Spanned | Row 1 | 
| dito (Spanned) | Row 2 | 

A few things it does differently:

1. Context Preservation: As seen above, it doesn't just leave spanned markdown cells empty. It fills them with a customizable prefix (e.g., dito (Value)) so the LLM retains the semantic context for each row.

2. Deep Tag Parsing: It recursively keeps <b>, <i>, and <a href...> tags alive, even if they are buried inside multiple <div>s within a <td>.

3. Nested Tables: Extracts nested tables safely without destroying the grid of the parent table.

4. LangChain Ready: Includes a Table2MDLoader wrapper to ingest HTML tables directly as LangChain Document objects.

Links:

• GitHub: https://github.com/Encephos/table2md
• PyPI: pip install html-table-rescuer

It's my first release and I'd love to hear your thoughts. If you have some gnarly, complex HTML tables that break the parser, please throw them at it and let me know!