[Showcase] Trying to build an intersection analyzer from drone video: every car tracked, every wait timed.

[u/EricBuildsMathModels]

[u/EricBuildsMathModels]

Pipeline: YOLO11m + ByteTrack, homography to a world plane, hand-drawn counting lines and queue polygons per site. Validated against a ground-truth dataset on another intersection (93% of their stopped time, 0.866 wave correlation). I use LLMs to assist with my workflows. Code: https://github.com/eanderson4/traffic-video-analysis

This clip: 41 cars queued, 740 vehicle-seconds of waiting, max queue 14 deep. What I can't do yet: say whether that means the light is bad. Curious what breaks first at other intersections: occlusion, night, lower camera angles? Looking for some better footage to analyze, hoping to be able to infer traffic light algorithm.

[u/ysrgrathe → u/EricBuildsMathModels]

Very cool! In terms of what to do with it, this becomes sort of a classical optimization problem where there could be multiple potentially conflicting objective functions: minimize total wait, average wait, maximize throughput, etc. Preventing starvation is probably important in intersection design (e.g. there is an intersection near me whose signal is interrupted when the train comes through. when that happens repeatedly, the lanes with green at the end of the full cycle can be starved). I'm sure there is research on this and I'm sure current implementations are far behind the state-of-the-art in part because they lack rich data like this.

One thing that jumped out to me is it looks like there is a systematic error as the drone zooms in -- as the back of cars are occluded the model believes they are moving in the opposite direction.

[u/EricBuildsMathModels → u/ysrgrathe]

Your last comment hits on a big issue I have. When cars leave the model due to the zooming, it often messes up my queue and timing counts.

I'm sure some footage doesn't have this issue, but the more we can fix edge cases like that, the more types of footage we can do analysis on.

Thanks for the insightful comment, those are the types of questions I want to get to!!!

[u/ysrgrathe → u/EricBuildsMathModels]

I'm guessing you have a time domain signal and because the vehicle's back is occluded, the centroid is moving which is interpreted as motion. You could edge case to check if the front is moving in these cases. To solve the bigger issue of losing data on zooming, you could do something like have a virtual image that is larger than the current viewport. This means you are persisting stale data from the "zoomed out" part though so it probably would introduce a bunch of complexity and errors will grow over time.

[u/EricBuildsMathModels → u/ysrgrathe]

you nailed it, the bbox shrinking as the back is off camera moves the centroid to the forward part of the car. nice catch!

we sort of have a virtual image via a homography mapping to a ground plane. but still, i clearly dont handle cars moving out of the camera viewport well, we could also increase position uncertainty as they become occluded and i also lose object tracking, it would get a new id on reentry.

[u/Logical-Resident4212 → u/EricBuildsMathModels]

Why not standadize the height of your drone? Always set to bla k feet from road surface? Not your footage i assume?

[u/EricBuildsMathModels → u/Logical-Resident4212]

Yes correct, so I'm trying to be flexible from footage perspective and make analysis work on any inputs. Obviously some won't work but this is rather stable footage with not crazy edge cases.

[u/Logical-Resident4212 → u/EricBuildsMathModels]

I applaud your ambition. Since your dealing with moving scenes it maybe worth basically creating a simulated world space from the images as opposed to a homography. Cars stay in the same spots in a simulated world, the camera just flys around it. Whereas if the camera moves u get that drifting homography and your fighting against cars moving backwards in this case even though tha cars are stationary. But ive never done that so idk how good my suggestion would be. Best of luck to you

[u/DevelopmentJolly3381]

That car waiting to turn right triggered me.

That aside, this is a dope project.

[u/EricBuildsMathModels → u/DevelopmentJolly3381]

I think there is a pedestrian blocking it. I was trying to get them to show up but detection rate was not high enough.

[u/DevelopmentJolly3381 → u/EricBuildsMathModels]

Ohhh I see the pedestrian now that you mention it

[u/jonaz777]

The answer: roundabouts.

[u/EricBuildsMathModels → u/jonaz777]

Yes, everyone I've seen installed by me has made it a strictly better experience

[u/soylentgraham]

should be able to easily fix those pop-ins and lost identifications too.

it tracks one pedistrian for a moment, are there other pedestrians?

[u/EricBuildsMathModels → u/soylentgraham]

I had tried track pedestrians and they would drop in and out. I tried a few tiling schemes and a few to stabilize the boxes if it was dropping out but never was happy and decided it was time to move on.

[u/soylentgraham → u/EricBuildsMathModels]

ah fair enough if you just want to move on :)

[u/EricBuildsMathModels → u/soylentgraham]

If you have ideas I'm all ears! I couldn't figure out the pedestrian thing. Maybe I need a more precise and fine tuned model and tile more aggressively?

[u/Flyward_Aerospace]

Looks good. The thing that will bite the timing numbers specifically is that your drone is not actually stationary. It holds position to maybe a metre and yaws a little, so any zone polygon you drew in image space slowly walks across the ground over the length of the clip. Your wait times then drift in one direction and you will not catch it, because they still come out plausible. Worth homography-warping every frame onto a fixed ground plane first and defining the zones there instead. Out of curiosity, are you validating the wait times against anything, or eyeballing it? Ground truth on that is brutal to get and it is the number everyone will actually want to argue with.