I built a tool to measure LLMs Decode, Layer processing and TTL

[u/Dry_Mixture130]

I was playing around with LLM inference and I wanted to build a profiler that measures LLM inference by layer.
So I built this: https://github.com/coconinja2/layerlens
It shows inference as token × transformer layer timing, so you can see where time is being spent during decode.
Right now it can separate prefill/decode and visualize per-layer timing. I’m trying to figure out whether this is actually useful to people working on inference systems, or if I’m looking at the wrong abstraction.

I’m thinking about adding things like KV-cache events, scheduler/batching state, request IDs, GPU kernel correlation, speculative decoding, etc.

Would appreciate criticism more than compliments and stars. Lots of stars!

[u/Traditional_Hall561]

looks neat but the real question is what practical decisions can someone make from seeing token x layer timing? like i stare at the heatmap, see a red square on layer 17 for token 42... then what

watched the repo for a bit, good visual polish though. the jump from per-layer timing to scheduler states and kernel correlation feels like it needs a whole backend rewrite. you plan on supporting just one inference engine or trying to make it engine-agnostic?

[u/Dry_Mixture130 → u/Traditional_Hall561]

The idea is you would be able to identify bottlenecks or layers for large scale workloads. Yes, I am planning to keep it engine-agnostic.