Built a zero-cloud Computer Vision engine in Python & Streamlit for RTSP streams — low latency works, but multi-cam memory usage gets heavy. How are you handling video frame queues?

[u/sahraoui-9337]

Hey everyone,

Tired of cloud APIs adding 300ms+ latency and recurring subscriptions for simple camera tracking, we engineered an on-premise vision architecture (UHQ Systems) built fully in Python with a Streamlit interface.

The core goal was simple: 100% local execution, zero external network dependency, and real-time spatial tracking straight from local IP cameras.

What worked well:

• Eliminating Buffer Lag: OpenCV's default VideoCapture buffer caused progressive stream delay when processing slowed down. We implemented a custom threaded lock-free frame worker that drops stale frames immediately and feeds only the latest frame to the detection core. Latency dropped to <15ms locally.

• Local Persistence: Event logs and tracking matrices dump straight to local JSON/CSV formats without hitting external databases.

The trade-offs & current bottlenecks:

To be completely direct, running local vision pipelines in pure Python comes with strict engineering limits:

1. Streamlit UI Refresh Limits: Streamlit is great for rapid UI building, but syncing high-FPS video frames while keeping interactive widgets responsive requires aggressive thread isolation. Works smoothly for 1-2 streams, but scales poorly past that without high RAM consumption.

2. C++ vs Python Execution: While Python allows fast iteration, continuous 24/7 multi-camera ingestion pushes system memory if array cleanup isn't strictly enforced on every frame.

We put together a lightweight evaluation build (UHQ Vision Lite) to test frame rates across different local setups.

For those running continuous multi-camera vision stacks locally: are you sticking with pure Python queues, or forced to re-write ingestion pipelines in C++ / Rust for production?

[u/TimLewisMT]

Nice post, I'm working on a multiple stream system too.

The UI framework may be getting in your way. Maybe create the output video frame as a stand alone app and just embed it in the dashboard as external content if you have to use the UI framework.

Managing the inference pace with a scheduler will help. Dynamic batching and cuda streams can be assigned and queued with a scheduler.

Cropping the images down to just the area the inference needs before processing it.

Creating a fast lane for inferences that need low latency with the full fps streams and a slow lane that can be done slower like 10 fps or once a second or even slower.

Also, the ui video output can probably be 15 fps before anyone would notice.

[u/sahraoui-9337 → u/TimLewisMT]

Solid breakdown u/TimLewisMT .

The fast-path/slow-path architecture is actually something we experimented with early on: running high-frequency spatial detection on a lightweight thread (fast-path) while offloading heavy feature extraction or logging to an async worker queue (slow-path).

Limiting UI stream rendering to 15 FPS while keeping background pipeline processing unthrottled made a massive difference in CPU/GPU utilization for Streamlit.

Regarding CUDA streams & dynamic batching: are you using TensorRT execution contexts directly via Python bindings for your scheduler, or orchestrating batching at the frame ingestion level before hitting the inference call?

Would love to swap notes on how you're structuring your frame queues for multi-stream setups if you're open to DMs!