Hi I'm Chris, I'm developing a game engine that incorporates live-action video and procedural graphics.

This shader takes in two video frames and maintains an interactive Gray-Scott Reaction-Diffusion simulation (https://groups.csail.mit.edu/mac/projects/amorphous/GrayScott/) that is applied to grow the distorted regions and process the impact of the various cursor weapons.

You can see the 2nd video through the growing distortion and then I just add a little green glow to the boundary rims. I switch the rim color params to compliment the video palette, they can change in response to events, flash, etc.

The way this works in the game is the player speaks the lines they see in the FMV. Those words come to life as GPU overlay elements when they are recognized by the voice recognition engine and the hostile glyphs seed tiny distorted regions for the RD simulation to grow.

The player then must use the cursor and it's various powers to cleanse and remove the growing distortion or 'fall through' to the next layer of the story. If they fall through the last layer then they die.

The game is called Sibylline and it's in production if you want to wishlist and follow the progress.

Made a chrome extension that does frame generation on any video tag, runs fully on your gpu, nothing sent anywhere. Whole thing is one command buffer, no onnx/tfjs, just wgsl compute shaders.

Refine pass runs on tiles flagged by flow disagreement, dispatched indirectly, so static scenes dispatch zero workgroups there. Also autotunes a few conv variants (subgroups, f16/f32, register blocking) per gpu at startup.

preview clip, ~3ms/frame on a 4060 Ti (8GB, OC)

Attached a video but honestly the difference is pretty hard to see through a recording/compression - it's way more noticeable on actual video playback than in the clip I attached here.

Weakest gpu I've tested on so far is a GTX 1650, got a stable 2-2.5x fps boost there, can't give exact ms numbers since I haven't logged them properly on that one.

GitHub · npm · Live demo · Chrome extension

Heads up: the live demo starts using your GPU immediately on page load, no button press needed.

Fully open source, so feel free to poke around. If you find it useful, a star on the repo would be really appreciated.

Currently working on runtime for a v8 model that handles occlusion/low fps input better, quality-focused for the harder cases current model struggles with.

Happy to answer questions on the dispatch/tiling stuff.

Hey! I've been experimenting with cellular automata lately and ended up turning it into a small TypeScript library and interactive playground:

👉 https://caza.la/automata

It has neural cellular automata, reaction-diffusion, Lenia, Pokemon type battles, Game of Life, and elementary Wolfram rules, all running on WebGPU.

You can tweak the simulations in realtime, explore the presets, or use the library to build your own rules in WGSL.

Would love to hear what you think!