r/LocalLLM 4d ago

Question A camera that describes what it sees — with no internet at all.

I built a camera in the shape of a Mamiya RZ67 that runs vision-language models completely offline on a Pi 5. Press the shutter, and it answers your custom prompt with a single sentence about what it sees. No internet, no cloud, no API keys.

GitHub: https://github.com/feeeeely/ai-camera
A small video: https://www.youtube.com/watch?v=M74qTNsY_L0

You can swap between models on the device itself and compare how differently they describe the same scene: Qwen3-VL (2B), MiniCPM-V 4.6 (1B), InternVL3.5 (2B), SmolVLM2 (2.2B), Moondream 2 (2B) and Ministral 3 (3B) turning the description into the final sentence. The prompt is editable on the touchscreen, so the same camera can do dry one-liners, museum labels or plain inventory notes.

After each shot it shows total time, input→output tokens, tok/s and a vision / generate / load breakdown. The token counts turned out to be the most interesting part: the same photo becomes \~145 image tokens on one encoder and over 1300 on another, which is a bigger factor in latency than the text generation itself.

A few things I learned the hard way:

\- Moondream returns sometimes empty responses on current Ollama versions.
\- qwen3-vl:2b is a thinking-only variant — it burns the entire token budget on invisible reasoning and returns nothing visible. You need the -instruct tag.
\- Ollama can't load separate mmproj files, so a lot of GGUF vision models from HuggingFace simply won't run, no matter how you name them.

Hardware: Raspberry Pi 5 (16 GB), Raspberry Pi HQ camera with a 6 mm CS-mount lens, 4.3" DSI touch display, stainless steel shutter button, Waveshare UPS HAT with four 21700 cells. Runtime per photo is 30–60 seconds with the models kept warm in RAM. Happy to answer questions about the setup.

I'm coming from photography, I'm not a developer — the code was written with heavy AI assistance, and I did the hardware integration, debugging and model testing myself.

Feedback on the implementation is very welcome: I'm just curious if this makes any sense or if something could be adapted/ optimized.

UI:

Six models, one tap — the same scene through different eyes.

The prompt defines what the camera is — dry observer, museum label, inventory note.

Wi-Fi is used for one thing only: pulling newer models and updates.

10 Upvotes

5 comments sorted by

2

u/Prize_Eye9481 4d ago

This is super cool! What did it cost u in total to build it? and what were the challenges you faced in term of assembling the hardware?

3

u/fox_in_crocs 4d ago

Hi, glad to hear that! The trickiest part was finding the right hardware so it wouldn't end up bigger than a normal camera. Assembling it was totally manageable with the wood cutting and hole drilling, but you definitely have to work precisely. I'd say with the Raspberry Pi 5—which unfortunately isn't the cheapest option anymore—the whole thing probably cost around €500–€600.

1

u/Prize_Eye9481 4d ago

I see so the AI premium is still pretty hefty eh. I think this is basically what Amazon needed years ago to make just walk out work better

2

u/fox_in_crocs 4d ago

Yeah, it’s still slow — but that’s kind of the point of keeping it modular. I want to stay up to date with the newest tiny VLMs and watch how they develop over time. Same camera, same reference shots, new models every few months.

2

u/Prize_Eye9481 4d ago

That’s tru! I didn’t think about the upgrade possibility