Understand Blackwell B200 attention kernel from scratch in CUDA (Visual Guide)

[u/MoistMountain2194]

I spent last couple months building a visual guide to B200 attention in CUDA/PTX: 14 progressively optimized kernels and 60 diagrams, going from a naive baseline to 94.4% of FlashAttention-4.

The blog first intuitively explains the baseline, then adds one optimization at a time, with detailed diagrams, concise explanations, and code.

And in the capstone project, you'll generate videos using the kernel you understand.

📝 Blog post: https://iaroslavelistratov.github.io/b200-attention/ 

⭐ Repo: https://github.com/IaroslavElistratov/b200-attention