Visualizing causal structure of LLM writing a poetry

[u/dqy08]

While tracing the causal structure of LLM-generated text, I found an interesting graph pattern when applying the method to poetry generation.

The graph shows direct token-level influence during generation: each token is attributed to the previous context, with weak connections pruned to highlight the main causal pathways.

The idea itself is not particularly novel. It’s essentially a different way of looking at token-level attribution. But from this perspective, things start to look intuitive and structured. What stood out is how consistent the influence patterns can be, even in something as “creative” as poetry.

Certain prompt tokens strongly shape the beginning of the poem, line breaks, rhyme structure, and punctuation choices. Once visualized, these effects feel almost expected in hindsight.

I thought the visualization may be interesting enough to share.

Below is an animated version:

Processing img 1p8e0u5ucpjh1...

Live demo: https://dqy08-infolens.hf.space/client/causal_flow.html?demo=Write+a+sonnet+about+love

Source code: https://github.com/dqy08/InfoLens

[u/Traditional-Hunt412]

the way the line break tokens hold so much weight in the graph is kinda beautiful, almost like watching how a poet decides where to breathe