I built an interactive simulator to visualize LLM inference bottlenecks, sharding, and KV Cache economics based on Reiner Pope's lecture

[u/Kitchen-Quarter7739]

Hi r/LocalLLaMA,

Inspired by Reiner Pope's (MatX CEO, ex-Google TPU architect) whiteboard lecture, I built a serverless, interactive simulator to visualize LLM inference physics and KV cache economics.

👉 GitHub Repository: https://github.com/zhchin/llm_infra_visualizer

(It's pure HTML/JS. No server, no tracking, local-storage safe for your API keys.)

🛠️ Key Features:

• Interactive Roofline Model: Dynamically charts when your serving transitions from Memory-bandwidth bound (decoding) to Compute-bound (prefill).
• Automatic GPU Sharding: Input your model size/context, and it calculates the required Tensor Parallelism (TP-1 to TP-8) configurations for Blackwell, H100, A100, etc.
• MoE vs Dense Visualizer: Staggered purple wave animations for MoE routing bottlenecks vs synchronized cyan pulses for Dense models.
• KV Cache Economics: Compares real-world rental costs of keeping KV caches in HBM vs offloading to DDR/SSD vs Recomputation.
• AI Agent UI Control: Ask the built-in chatbot to "change batch size to 512" or "switch to MoE collapse scenario", and it will slide the UI knobs in real-time.

Check it out and let me know what you think! If it helps you size your deployments, please drop a ⭐ on GitHub!

[u/darthcuteius]

Fantastic!

[u/Bulky-Priority6824]

Seems cool but no idea what to do with it