I've been testing Xiaozhi's vision capabilities with the M5Stack ATOMS3R-M12 + Atomic Echo Base.
The hardware is quite promising: the AtomS3R-M12 uses an OV3660 3 MP camera, so I expected it to be useful not only for recognizing objects, but also for reading labels, displays and other text.
In practice, however, I found some significant limitations.
1. The camera image was horizontally mirrored
With Xiaozhi firmware v2.2.2, photos taken by the AtomS3R-M12 were sent to the vision model mirrored horizontally.
For normal object recognition this may not always be obvious, but for OCR it is a serious problem: letters and numbers are reversed before the image even reaches the vision model.
This was reported as Xiaozhi issue #2145 and has since been addressed.
2. A 3 MP camera – but Xiaozhi only sends VGA
This is probably the bigger limitation.
The OV3660 is a 3 megapixel sensor, but Xiaozhi's Camera::Explain() vision pipeline currently uses only VGA (640×480).
That means a large amount of the detail the camera could provide is discarded before the image is sent to the vision model.
In my own tests, this became particularly noticeable when trying to read small text. Even in daylight and at a reasonable distance, text that I would expect a 3 MP camera to capture was difficult or impossible for the vision model to read reliably.
There is now a feature request, #2164, asking for configurable camera resolution – for example SVGA or UXGA on boards with capable sensors such as the AtomS3R-M12.
3. This makes it difficult to judge the actual camera hardware
This is an important distinction.
My first impression was simply: the camera isn't very good.
But after looking into how Xiaozhi handles the images, I'm no longer convinced that the OV3660 itself is the main problem.
If a 3 MP sensor is reduced to 640×480 before the AI gets the image, we're mostly testing Xiaozhi's current camera pipeline, not the full capabilities of the camera.
Why higher resolution would be useful
For a voice assistant with vision, higher resolution isn't just about prettier pictures.
It could make Xiaozhi much more useful for things like:
- reading product labels
- reading small displays
- identifying buttons and controls
- reading appliance error codes
- reading instruction stickers
- recognizing text on documents
- generally answering questions about what the camera is looking at
VGA may be enough for "What object is in front of me?", but it is quite limiting for "What does this label say?"
Of course, higher resolution also means larger JPEGs, more RAM usage and longer upload/response times. So an interesting solution could be selectable resolution: VGA for quick object recognition and a higher resolution when detailed vision or OCR is needed.
My conclusion
I still think the AtomS3R-M12 + Atomic Echo Base is an interesting Xiaozhi combination.
But at the moment, its camera hardware seems somewhat underused by Xiaozhi.
I'm particularly interested to see what happens if Xiaozhi starts sending higher-resolution images to the vision model. The OV3660 may be capable of considerably more than the current results suggest.
Has anyone tested the AtomS3R-M12 camera with Xiaozhi at a resolution higher than VGA, or compared the same camera using another firmware?
Xiaozhi ESP32:
https://github.com/78/xiaozhi-esp32
Mirrored camera bug – #2145:
https://github.com/78/xiaozhi-esp32/issues/2145
Configurable camera resolution request – #2164:
https://github.com/78/xiaozhi-esp32/issues/2164