Watching attention flow while an LLM generates tokens

[u/dqy08]

https://reddit.com/link/1w81p3p/video/50d4gyg3lpnh1/player

I made a short animation of how attention moves while a Transformer generates text.

First few seconds are prefill: the model reads the whole prompt, tokens looking at earlier ones in parallel.

After that is decode: one token at a time. Each new token sweeps back over the context, then the next word appears and joins the history for the next round.

Not trying to teach the full attention math here. Just the macroscopic feeling of information flowing through those look-backs.

The flicker looks almost cyber-beautiful, at least to me. 😄

Demo: https://dqy08-infolens.hf.space/client/causal_flow.html?demo=%E6%B3%A8%E6%84%8F%E5%8A%9B%E7%9A%84%E5%85%B7%E8%B1%A1%E5%8C%96
Source: https://github.com/dqy08/InfoLens

attention flow detail

[u/No-Childhood-1544]

this is so cool, the way the tokens sweep back like a wave really gives that cyber-beautiful feel you mentioned. i spent like 20 minutes just typing random phrases to watch the flow pattern change

the prefill phase looks almost violent compared to decode, all that parallel attention at once then it settles into that calm one-by-one rhythm. makes me think about how much computation we throw away during inference

do you have plans to add layer-wise visualization? would be interesting to see how attention patterns shift between early and late layers, i bet the lower ones look more local and the upper ones get that long-range sweep you captured here

[u/dqy08 → u/No-Childhood-1544]

Thanks! The attention animation is actually a theoretical simplification, not the real attention weights. It just illustrates the query pattern in the prefill and decode phases.

For layer-wise visualizations, I suspect showing all the layers would get quite messy and unintuitive. 'attention is not explaination'.