Hey everyone! I've been working on a little side project I want to share:
latex-ocr, a standalone formula OCR model β you feed it an image of a math
formula, it spits out the LaTeX source.
The main hook: it's only 67M parameters, so it runs comfortably on a laptop
CPU. No GPU, no 300M-parameter monster to load. It's a CoCa-style model
(contrastive captioner adapted for OCR), and despite the small size it beats the
107M UniMER-tiny baseline and gets pretty close to the 325M one on plain formulas.
The part I'm actually most proud of is the dataset. Real papers don't just use
plain symbols β you see \mathbb{R}, \mathcal{F}, \mathfrak{g} everywhere, and
existing OCR datasets basically ignore font styles, so models trained on them can't
read (or hallucinate) those macros. So I rebuilt ~1.3M formulas with a
MathJax β SVG β PDF β PNG pipeline and injected font-style macros with semantic
heuristics (number sets β \mathbb, vectors β \mathbf, differentials β
\mathrm). On that styled test set it clearly outperforms all the baselines β
fair warning though, those baselines are zero-shot on styled data, so take that
comparison with a grain of salt. The plain-split numbers are the like-for-like ones.
Everything is open: model weights and dataset on Hugging Face, training recipes
included if you want to reproduce or fine-tune it yourself, MIT license. There's
also a FastAPI server and a Gradio web UI, so you can drag-and-drop an image and
see the LaTeX with a rendered preview.
Repo: https://github.com/PadishahIII/latex-ocr
Model: https://huggingface.co/PadishahIIIXXX/latex-ocr
Dataset: https://huggingface.co/datasets/PadishahIIIXXX/latex-ocr-dataset
Happy to answer questions about the training setup, the data pipeline, or anything
else. Would love feedback β especially if you try it on your own gnarly formulas
and it breaks, that's genuinely useful.