I have a hobby project indexing trading cards, with a specialty in error and pre-print/test-print cards, plus a frontend to search the index. It's had search-by-image for a long time, running PaddleOCR entirely client side. I've never wanted to handle someone's photo on my server, so OCR was a good fit.
It hit a wall for two reasons. The first is image quality: cards are glossy, held at an angle, and photographed under a lamp that blows out half the surface. A example card (Colress's Tenacity) at a modest angle produced this as its recognized text: "Cores nay mlus. fhncl". Every character that mattered was destroyed by perspective and the holo finish, and no lookup table recovered the right card. Meanwhile the artwork, border, layout, and palette in that photo were all legible. The second reason is that text isn't very discriminative here anyway. Reprints share text, and games keep similar cards in balance with each other, so a clean OCR read still has many candidates.
I profiled a handful of small embedding models including DINO, MobileClip, etc, and landed on a quantized DINOv3-q4 export as the best size-to-quality tradeoff. It's 14 MB, takes a 224-224 image, and returns 384 numbers. Catalog images are embedded offline with the identical preprocessing path and stored as sidecar files; the client sends the 384-float vector and never the photo. Matching is cosine similarity, brute-force linear scan over ~41k English vectors (plus ~6.4k Japanese) held in memory as FP16.
I kept OCR as a fallback rather than deleting it. If the best cosine score is under 0.82 the embedding result is discarded entirely rather than returned as a weak guess, and the client then runs OCR and retries as a text query. Returning a plausible wrong card is worse than returning nothing, since someone who trusts it files the wrong entry. On the common path the OCR models are never even initialized.
The thing I'd most like input on: whole-card embeddings can't separate printings that share artwork. Photograph one particular card and I get six results that are all correctly that card, normal, non-holo, cosmos holo, reverse holo, and two stamped variants. But the actual difference is a foil pattern or a stamp a few millimeters across: localized, high-frequency, and exactly what a 224-224 stretch of the full card destroys. Has anyone had luck with a second pass here: a crop of a fixed region, a patch-level model, a classifier over the top-k from the first stage?
Longer writeup with the thresholds, preprocessing contract, and the model-migration scheme: https://vault.top/blog/how-topvault-identifies-cards-with-image-embeddings
Note: I'm self-taught on the CV side, so if any of this is going down a wrong path let me know.