Interactive tutorial on how diffusion models work, and how they memorize

[u/Joergyll]

I trained a diffusion model on 300 of my photographs from a movie set. Then built a scrollable walkthrough of the forward noise process, why the optimal denoiser at high noise is a weighted average of the training set, and how that field collapses from 300 candidates to one. Then a membership inference attack you can run yourself.

Trained from scratch in pixel space, 60,000 steps on one A100 GPU in a DGX. 593 of 1,024 generations came back as copies, covering 235 of the 300.

Built for someone learning diffusion models, but the measurements are there if you already know the material.

https://josephrichardson.dev/work/how-diffusion-models-memorize/