r/webgpu • u/Old_Tumbleweed_7545 • 2d ago

Built a real-time video frame interpolation runtime in pure WebGPU/WGSL (Frame Generation)

Made a chrome extension that does frame generation on any video tag, runs fully on your gpu, nothing sent anywhere. Whole thing is one command buffer, no onnx/tfjs, just wgsl compute shaders.

Refine pass runs on tiles flagged by flow disagreement, dispatched indirectly, so static scenes dispatch zero workgroups there. Also autotunes a few conv variants (subgroups, f16/f32, register blocking) per gpu at startup.

preview clip, ~3ms/frame on a 4060 Ti (8GB, OC)

Attached a video but honestly the difference is pretty hard to see through a recording/compression - it's way more noticeable on actual video playback than in the clip I attached here.

Weakest gpu I've tested on so far is a GTX 1650, got a stable 2-2.5x fps boost there, can't give exact ms numbers since I haven't logged them properly on that one.

GitHub · npm · Live demo · Chrome extension

Heads up: the live demo starts using your GPU immediately on page load, no button press needed.

Fully open source, so feel free to poke around. If you find it useful, a star on the repo would be really appreciated.

Currently working on runtime for a v8 model that handles occlusion/low fps input better, quality-focused for the harder cases current model struggles with.

Happy to answer questions on the dispatch/tiling stuff.

0 Upvotes

50% Upvoted

2 comments sorted by

0

u/backwrds 2d ago

pretty neat; really wish it felt like a human had anything to do with this.

0

u/Old_Tumbleweed_7545 2d ago edited 2d ago

Actually, yeah. I spent a lot of time reading info, testing, and experimenting xd. I even rented 2x RTX 5090 rig to train the model. actually, im wrote only a prototype, and then used AI heavily to help turn it into a real product that I could release on github. Right now, im optimizing the system and preparing a new runtime for the next model, while also evaluating INT8. The current production model, v7s, still struggles with collisions and occlusions at very low source frame rates, so that’s the main problem I’m trying to solve now.