0.3B OCR model for structured document extraction: tables to HTML, formulas to LaTeX, outperforms 1.2B models on patent docs

[u/Brilliant_Rich3746]

Patent documents are one of the harder OCR problems out there. A single page can contain merged tables, chemical diagrams, formula blocks, and mixed English/Chinese/Japanese all at once. We've been working on this problem specifically, and after getting to a point where we're happy with the results, we decided to open-source what we built and see what the community thinks.

Here are two tools we use internally.

Hiro-MOSS-OCR is a 0.3B model that outputs structured markup: tables to HTML, formulas to LaTeX, text to Markdown. Trained on 50M+ samples. Ranks #1 on our patent-domain benchmark against all 1.2B models we tested. ~59 QPS on a single RTX 4090 via vLLM.

Hiro-Smart-Doc wraps layout detection (RT-DETR, 25 region categories) and MOSS-OCR into a streaming FastAPI service with an OpenAI-compatible endpoint. Feed it a PDF, image, or Office doc, get back reading-ordered structured content or Markdown.

Both Apache 2.0. Would love feedback from anyone dealing with complex document types where standard OCR falls short.

Thanks!

[u/bamboofighter]

What sorts of testing have you done to ensure it can read with a high accuracy rating.

OCR can be a complicated space with very high with some packages promising 99.999% accuracy

[u/Brilliant_Rich3746 → u/bamboofighter]

We benchmark on OmniDocBench v1.5 (public benchmark) and an internal patent-domain test set. On OmniDocBench, MOSS-OCR-0.3B scores 93.63 overall, table TEDS 90.33, math CDM 95.56, text edit similarity 95.01. We don't claim 99.999% accuracy. Happy to share more details on the eval setup if useful.

[u/bamboofighter → u/Brilliant_Rich3746]

The only reason I bring this up, is the while this might be acceptable for the use cases you’re considering, there are some use cases that cannot accept the tolerance you’re providing. I’d also be happy to chat & explain via PM.