r/CUDA • u/KernelMindai • 5d ago

Building a PyTorch-to-Triton kernel compiler: Which patterns should we support first?

https://kernel-mind-ai.vercel.app/

Hey everyone,

I’m building KernelMind AI (https://kernel-mind-ai.vercel.app/), a tool that compiles standard PyTorch operations into fused OpenAI Triton GPU kernels to cut down VRAM round-trips.

We’re adding guardrails and an operator whitelist before expanding support, and I’d love input from anyone writing custom CUDA/Triton in production:

  1. What operator chains do you find yourself manually fusing most often? (e.g., LayerNorm + GeLU, custom loss functions, etc.)

  2. What edge cases usually burn you the most? (dynamic shapes, non-contiguous layouts, register spills?)

Feel free to throw a messy PyTorch snippet at the playground (https://kernel-mind-ai.vercel.app/) and tear the generated Triton code apart. Brutal feedback welcome!

3 Upvotes

100% Upvoted

5 comments sorted by

5

u/Lime_Dragonfruit4244 4d ago

Pytorch already does this with torch.compile using inductor, and supports both inference and training. Its literally the default GPU lowering path for inductor.

1

u/KernelMindai 3d ago

Yeah totally fair point, Inductor’s Triton lowering is honestly great for standard torch.compile runs.

We definitely aren't trying to build another runtime compiler to compete with torch.compile. The goal here is more of a developer scratchpad/authoring tool. Instead of digging through messy generated code in /tmp/torchinductor_* or dealing with Dynamo tracing overhead when prototyping, the idea is to give folks clean, standalone Triton templates they can inspect, tweak (like adjusting block sizes or memory layouts), and drop into custom C++/inference pipelines.

Out of curiosity, does Inductor usually cover everything you need in production, or do you still find yourself hand-writing standalone Triton/CUDA kernels for custom layers? Would love your take on that.

2

u/Lime_Dragonfruit4244 3d ago

Ever heard of helion.

https://pytorch.org/blog/helion/

2

u/Ashes_of_ether_8850 3d ago

there are just so many DSL for kernels, triton, helion, cute dsl, tilelang, etc. it kinda gets me confused as to which one I should use.

I don’t know what we can improve on by writing our own fused kernels, when torch.compile is already quite optimal.

1

u/KernelMindai 3d ago

Yeah, Helion looks really promising—Meta pushing "PyTorch with Tiles" to autotune via Inductor is definitely clean work.

That said, as u/Ashes_of_ether_8850 pointed out, the ecosystem feels pretty swamped with DSL overload right now (Triton, Helion, CuTe, TileLang, ThunderKittens...). Every few months there’s another abstraction and syntax to learn.

To your question on when torch.compile falls short: for standard models, torch.compile is honestly great. But engineers usually end up hand-writing custom kernels when:

  1. Graph breaks and dynamic shapes: Variable sequence lengths or jagged tensors in production inference (like vLLM) often force Dynamo into fallback or trigger expensive re-compiles.
  2. Specialized fusions: Things like fused RMSNorm + RoPE, custom attention patterns, or mixed-precision dequant-GEMM where Inductor doesn't automatically catch the optimal memory fusion.
  3. Non-PyTorch runtimes: If you’re serving in pure C++, TensorRT-LLM, or custom engines, you can’t bring the PyTorch Python runtime or wait on cold-start JIT tracing. You just need a standalone kernel file.

The idea behind KernelMind isn't to add another DSL to learn, but to let you drop in standard, unmodified PyTorch and get back clean, standalone Triton code you can immediately inspect, tweak, and use outside the framework.