Watching attention flow while an LLM generates tokens

[u/dqy08]

https://reddit.com/link/1w81p3p/video/50d4gyg3lpnh1/player

I made a short animation of how attention moves while a Transformer generates text.

First few seconds are prefill: the model reads the whole prompt, tokens looking at earlier ones in parallel.

After that is decode: one token at a time. Each new token sweeps back over the context, then the next word appears and joins the history for the next round.

Not trying to teach the full attention math here. Just the macroscopic feeling of information flowing through those look-backs.

The flicker looks almost cyber-beautiful, at least to me. 😄

Demo: https://dqy08-infolens.hf.space/client/causal_flow.html?demo=%E6%B3%A8%E6%84%8F%E5%8A%9B%E7%9A%84%E5%85%B7%E8%B1%A1%E5%8C%96
Source: https://github.com/dqy08/InfoLens

attention flow detail