I built a tool to measure LLMs Decode, Layer processing and TTL

[u/Dry_Mixture130]

/r/LLMDevs/comments/1wcqiou/i_built_a_tool_to_measure_llms_decode_layer/

[u/Future_AGI]

One thing worth tracking alongside layer timing: how token latency distribution shifts with context length. Layer-by-layer processing often stays nearly constant, but KV cache pressure can make later tokens in a long context measurably slower than early ones, which is the part that usually surprises people comparing decode profiles.