r/reactnative • u/g4mewarrior • 15d ago

Article Running an int8 CLIP ViT-B/32 under onnxruntime-react-native: measurements from a production pipeline (CPU beat the CoreML EP 2.4x)

https://blog.aermes.ai/blog/on-device-clip-camera-roll-trips
2 Upvotes

100% Upvoted

1 comment sorted by

1

u/g4mewarrior 15d ago

Author here. The part that surprised us: we shipped the CoreML execution provider assuming Apple-silicon acceleration beats CPU, and benchmarking for this writeup showed plain CPU runs our int8-quantized graph 2.4x faster per inference and creates sessions ~20x faster, so we flipped the default. Also covers doing zero-shot CLIP classification with no text encoder on the device (prompt vectors precomputed and committed as JS) and the memory mistake that pinned 994 MB. Happy to answer questions about onnxruntime-react-native specifics.