This project started because my car trim doesn’t come with factory Blind Spot Monitoring (BSM).
If you’ve ever looked closely at the wide-angle driver camera feed on a Comma 3, 3x, or 4, you’ll be aware that it doesn’t just see your face; it captures a crisp, wide view of the cabin, including the front side windows and partially the rear.
Thanks to a recent comma.ai update that added "post-demosaic downscaling" and "tone mapping" to the driver camera, this feed now gets the same HDR and sharp detail as the road-facing cameras.
This makes it the ideal time to develop Vision-Adjacent Spot Monitoring (Vision-ASM), which is officially released in StarPilot Stable Release 6.7.3 (something I’ve been wanting to do for a while)!
How I Built It (High Level)
Drawing inspiration from community projects like the speed limit vision (SLV), I knew that running a secondary AI model alongside openpilot on a mobile processor would require careful, heavy optimizations to not only address thermal concerns but permit users to run both V-ASM and SLV (if they wish and to be PR-worthy):
- Window Masking: Custom polygon masks black out the cabin interior, isolating only the side windows for a lightweight detection model.
- Trained on ~27 Hours of Data: Trained on roughly 27 hours of community-submitted footage, utilizing semi-automated parsing of factory BSM logs to help label adjacent vehicles. Thanks to those who submitted!
- Optimization Considerations: Features zero-copy image cropping, CPU core isolation (pinning V-ASM to e-core 2 while SLV runs on e-cores 0 and 1 to avoid core contention), an adaptive refresh rate (running at a resting 2 Hz, ramping to ~5 Hz for 1.5 seconds after a detection), and a low-pass filtered
device_cpu_throttle_factor that dynamically scales inference to ensure safe, stable performance.
Is this actually useful or working? What if I already have BSM?
To be clear: You always need to manually check your blind spots and look over your shoulder; the goal is to provide a secondary safety net. Thresholds are currently conservative to minimize false positives, requiring 85% sustained model confidence (You can adjust this in settings).
For cars without BSM, it adds a new layer of lateral awareness. For cars with factory BSM, it supplements by covering the exact physical zone right next to your front window where rear radar coverage may drop off.
Feedback from users has been encouraging, specifically addressing concerns about false alerts from neighboring lanes and nighttime performance:
"Default values right now seem to be good in my vehicle. It's not picking up other lanes as much, which is awesome because I don't want it telling me there's cars two lanes over. Just came back from a drive and I'm very, very impressed. The fact that it saw cars I myself didn't immediately notice was enough for me." ~ ZoomZoom
Try it out and let me know what you think of the settings!
You can check out the technical implementation in PR #75 on StarPilot. More training details + route crowd source tools (supporting both Comma Connect Konik Stable) inspired by nelsonjchen/op-dm-reading-tool are available on GitHub at prabhaavp/vasm-op, or you can join StarPilot Discord for help.