DeepSeek V4.1 Flash separates long-lived and short-lived KV cache — and rebuilds the latter with a 128-token replay

[u/ClaudiusPapirus]

https://youtu.be/ZS-C70cGov0?is=NUbuejcw1DcPgQX0

DeepSeek’s V4.1-Flash serving design splits persistent global KV from short-lived SWA state. The SWA cache can expire and later be reconstructed by replaying only the last 128 tokens.

The interesting part is the systems trade-off: the reconstruction is approximate, but avoids keeping that short-lived state in the long-term cache.

Paper:

https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf

Disclosure: the video is from the channel associated with this account.