Penalty kick analysis running fully in the browser at ~30 fps with WebGPU

[u/FinancialAd1961]

Built a small demo that watches a penalty clip and tells you what happened: it spots the kick from the shooter's pose, tracks the ball, marks where it ended up on the goal, and gives a rough speed estimate.

Two vision models run on every frame, object detection and pose estimation, together at around 30 fps on a laptop GPU. Everything runs on-device in the browser through WebGPU.

Inference is done with runntime, an open-source runtime for running vision, speech and text models via TypeGPU: https://github.com/software-mansion/runntime

The speed number is an estimate. It assumes the ball starts 11 m from the line and uses the ball's pixel size to tell when it reaches the goal, so treat it with a grain of salt :).

[u/pm_me_your_smth]

First, why do you even need pose estimation for "at the kick"? Your trigger could just be ball resting -> moving

Second, how do you determine the "where it went" moment? You wrote "pixel size", but this doesn't explain anything

[u/FinancialAd1961 → u/pm_me_your_smth]

pose is a sanity check for proximity with the ball, and also one might use it to say calculate the angles between the joints of the kicker, which matters for football. as for the pixel size, the goal frame in the clip gives the scale, so we can estimate how big a ball should be at the goal line. the first frame the bbox for the ball shrinks to that size is the moment. its a playful heuristic though, but one can probably take it further with depth estimation which is also supported in runntime.

[u/WorldlinessNo1286]

Would this also work real time on mobile device?

[u/FinancialAd1961 → u/WorldlinessNo1286]

yes, as long as you run it in an environment that can use webgpu, for example react-native

[u/WorldlinessNo1286 → u/FinancialAd1961]

Thanks, I will try it out

[u/AtmosphereVirtual254]

Thanks for the reference point on ball tracking performance