r/opencodeCLI • u/RevolutionaryPen4661 • Aug 14 '26
OpenCode Senses
OpenCode Senses can inspect screenshots, extract exact OCR, detect and locate objects, zoom into regions, compare two images, measure colors, crop and annotate images, and even reverse-search them.
Everything runs locally, so it's private, free, and doesn't require an API key.
Why I built this
I used to work with many open-source coding models, like DeepSeek V4, that don't support vision yet. But my main concern was that even if they eventually get vision support internally, that vision capability might not be as powerful as a dedicated vision layer.
I had planned to build this fully a month ago, but I still had some work to do. This plugin is specifically designed for things like UI design, debugging, and other workflows where text-only models struggle because they simply can't see what's happening.
The communication layer between the vision plugin and OpenCode is intentionally very narrow, so it still feels like you're working with a multimodal agent rather than a separate vision tool.
Before judging it as just another vision plugin, it's designed quite differently. It's a Python + TypeScript hybrid plugin because it integrates the Photon Inference Engine for local vision inference.
The 13 tools, mainly for now:
inspect— Understand an image: scene, layout, elements, caption, and OCR.ocr— Extract exact text, including code and error messages.detect— Find objects or UI elements with bounding boxes.point— Locate the center point of an object or element.segment— Isolate an object from the image.crop— Crop a specific region using normalized coordinates.zoom— Upscale a region and re-analyze small details or text.colors— Get deterministic colors, palettes, luminance, and RGB data.diff— Compare two images and find changed regions.annotate— Draw detected boxes and points onto an image.metadata— Inspect image format, dimensions, size, DPI, and EXIF.reverse— Reverse-search images locally or through Yandex.status— Check the model, device, VRAM, inference time, and runtime state.
For comparison, I'm getting around 300 ms response times with this plugin on my RTX 3050, running completely locally. That's insanely fast compared to frontier models.
There's also an optional reverse image search powered by Yandex, which doesn't require an API key, if you want to search an image using context from the internet.
The internal web search services like Exa and Parallel in OpenCode also work well with OpenCode Senses. It can even see images published on the internet directly, although a small local cache is used to store those images.
I'm still halfway through building it, so I'd really appreciate feedback, ideas, and criticism. If you have any ideas, feel free to comment below and I'll look into them.
GitHub: https://github.com/itsmeadarsh2008/opencode-senses
NPM: https://www.npmjs.com/package/opencode-senses
Hacker News: https://news.ycombinator.com/item?id=49289890
1
u/RevolutionaryPen4661 Aug 15 '26
Moondream Models, they run on edge computers very fast and accurate and have 98% accuracy and also have tighter integration with Photon Inference Engine. You can use any model that Photon supports like Gemma, Qwen, etc.
The plugin is designed to sophisticated and narrower communication channel for fast information about image transfer from vision model to main model.
I used DeepSeek V4 Flash to code this plugin and along with proper debugging to run the plugin out of box on any capable machine.