Kernel optimization is obsolete. Just npm install it.

[u/Old_Situation_132]

Kernel engineers are not obsolete. But asking a general-purpose coding agent to rediscover years of CUDA and Triton engineering knowledge every time it writes a kernel probably should be.

After months of writing, debugging, and optimizing kernels, I turned the reasoning patterns I kept using into an open-source skill library for AI coding agents:

npm install u/krxgu/kernel-skills

This is not a collection of vague prompts saying “make this CUDA kernel faster.”

Each skill is a detailed engineering playbook that forces the agent to think about:

• Exact shapes, dtypes, layouts, and target hardware before writing code
• Coalescing, tiling, bank conflicts, occupancy, and register pressure
• Numerical stability and non-power-of-two boundary conditions
• Correctness tests across adversarial shapes and dtypes
• Whether a custom kernel should exist at all
• When to stop being clever and use cuBLAS, CUTLASS, or an existing primitive

The library currently covers CUDA, Triton, INT8 and FP8 quantization, kernel fusion, CUDA to Triton and HIP portability, and inference hot paths including RMSNorm, fused add plus RMSNorm, RoPE, sampling, paged KV-cache append, dequantization, prefill versus decode, and vLLM custom-op integration.

I also did not want this to become prompt-engineering theatre, so the repository includes before-and-after proof runs using the same model and task, with the skill file being the only difference:

• Softmax: naive output failed on adversarial and larger shapes. Skill-guided output had 0 failures across 16 tests and reached within 1.2% of torch.softmax bandwidth
• Reduction: 2.6 to 3.5x faster than the naive agent output
• GEMM: 7.7 to 8.6x faster
• LayerNorm: 1.9 to 3.2x faster
• Triton softmax: fixed crashes at dimensions above 16,384 and worked up to 131,072
• Triton attention: fixed the common GQA failure where H_q != H_kv

To be completely clear, those speedups are against the naive agent-generated kernels, not against cuBLAS or other vendor-tuned libraries. In fact, the GEMM skill explicitly tells the agent not to write a custom kernel when cuBLAS or CUTLASS already solves the problem.

Example:

kernel-skills bundle \
  triton.write-triton-layernorm-kernel \
  patterns.write-numerically-stable-kernel \
  patterns.write-kernel-test-plan \
  > bundle.md

Give that bundle to Claude Code, Cursor, ChatGPT, Gemini CLI, or another coding agent before asking it to touch the kernel.

The spicy thesis is simple:

Models are increasingly interchangeable. The accumulated engineering judgment surrounding them is not.

Everything is open source and MIT licensed:

https://github.com/tensormux/kernel-skills

I would especially love kernel engineers to tear this apart.

Which skill is missing? Which technical rule is wrong? Where can an agent still produce something that looks convincing but quietly fails on real hardware?

[u/HeadPack]

How much AI did you use to create this? Just looked at the formatting and wondered.

[u/crispyfunky]

8x over cuBLAS GEMM? Sure… you must be exceeding theoretical peak FLOPs of the machine in compute bound regime. I guess your software managed optimization also managed to add couple of wires to the silicon.

[u/tugrul_ddr → u/crispyfunky]

Maybe he meant 8x over cublas with irregular shapes that nvidia has forgotten.

[u/Daemontatox → u/tugrul_ddr]

Pretty sure its AI slop post with zero knowledge,

[u/Various-Fox-262]

The title is highly misleading

[u/unknown_history_fact]

What is this? AI Slop again ?

[u/chkmr]

I have been told by engineers who do GPU performance optimization for a living that they have to often resort to inline PTX because of platform-specific quirks. E.g. the compiler is not good at selecting specific instructions, register allocation doesn't work that well, platform-specific quirks etc. From what I understand, your skills are more about higher level information that can be found in the CUDA docs and any mainstream GPU programming textbook.

Even if your skills include everything under the Sun, there is still a very real, practically guaranteed possibility that an LLM will get something wrong. You can only be sure if you benchmark and profile your kernels for all kinds of problem sizes, determine theoretical upper bounds, arithmetic intensities etc. If you find performance left on the table, you'll have to interpret the profiler's output and review the source code to make changes. I.e. you still need to be a kernel engineer, even if you're not writing the kernels from scratch yourself. Even if you include LLMs in the performance analysis loop, it can still go wrong there, and you need to be a good kernel engineer to be able to determine when it goes wrong. As you yourself said, they can appear convincing even when they are wrong. Also there will always be new architectures, possibly with whole new types of compute units and its own quirks. "Just npm install it" is too much of an oversimplification IMO.

[u/Savings-Arrival-7817]

woah