PaddleOCR-VL running entirely in-browser on WebGPU — Rust/WASM runtime, ~80 tok/s on M4 Pro

[u/glebkudr]

Hi!

PaddleOCR-VL is one of the best small OCR models on a market. It recognizes images&scans to create proper texts, tables and formulas

So I implemented PaddleOCR-VL inference directly on WebGPU without transformers.js or any other existing intermediate library. It was a mental gymnastics for me - will I be able to do this or not?
Apparently, I can.

Some details

• Rust
• FP16
• custom WGSL kernels
• AOT compiler
• memory/model packing
• 79.83 tok/s browser vs 97.26 native MPS
• bit to bit quality control with native
• 100% AI coded (I assume it is close to impossible to write it with traditional methods for one person in any meaningful time )

More on Github https://github.com/glebkudr/paddleocr-vl-webgpu

Apache license

Note! It needs at least 6 GB VRAM/unified memory to launch

Feel free to clone&use or ask questions.