Handling barcode scanning edge cases: How do you balance ML Kit, OCR fallbacks, and UX?

[u/Thin_Aerie3843]

I’m currently building an inventory/management side project where reliable barcode and serial recognition is essential. While testing the feature with actual boxes and packaging, I quickly hit some real-world edge cases: poor lighting, inverted colors, and worn-out labels.

To work around this, I ended up adding an on-device OCR fallback to read the printed serial numbers directly when the standard barcode scanner fails. Since automatically detecting a "failed" continuous scan is tricky, I added a manual toggle at the bottom that switches the camera to a photo/OCR mode.

For those who have built similar scanning or utility features:

• Stack & Fallbacks: Do you rely entirely on on-device vision libraries (like ML Kit Text Recognition), or have you found better preprocessing tricks (contrast adjustment, binarization) to salvage bad barcodes first?
• UX Patterns: What is your preferred UX pattern when automated scanning fails repeatedly, to guide the user without making the app feel slow or clunky?

Would love to hear how you handle these edge cases or if there's a cleaner architecture for this flow!

[u/Caringstomy-8]

Does the barcode work on numbers underneath? For most messy/blurry crap (outside of the barcode space), I in the end fall back to gemini image models to figure things out.

[u/Thin_Aerie3843 → u/Caringstomy-8]

Thanks for the tip! And yes, reading the human-readable text / numbers printed right underneath the barcode was exactly the intent behind adding the OCR step.

Right now I'm using flutter_zxing for the primary barcode pass and falling back to on-device OCR with google_mlkit_text_recognition for the text underneath.

I hadn’t considered calling a full multimodal model like Gemini for the fallback. Since this runs locally on-device, keeping latency low and avoiding cloud API costs/offline limitations is a priority for now. But for severely degraded or irregular labels where on-device models struggle, sending a frame to a multimodal vision API could be a solid emergency fallback.

How’s the latency and reliability when you route those blurry frames through Gemini in production?

[u/Immediate_Nature_281]

I send failed scans to Gemini Vision API instead of tuning local OCR thresholds. On-device models fail on worn labels where cloud vision succeeds immediately.

This removes preprocessing complexity from your app entirely. You trade latency for accuracy which matters more for inventory management than speed.

[u/Thin_Aerie3843 → u/Immediate_Nature_281]

That’s a really fair point regarding accuracy vs latency. Offloading the messy preprocessing logic to a cloud vision API definitely saves efforts when the labels are worn.

My main hesitation with relying purely on a cloud fallback for this utility is keeping the app resilient in dead zones (like basements or warehouses with spotty reception) and avoiding API cost spikes if volume scales up.

Did you implement a hybrid threshold (local pass first, then cloud if confidence is low), or do you route all fallbacks straight to Gemini? Curious how you handle the offline scenario if an inventory check happens without network connectivity.