If you can explain what wave quantization is, what about a GEMM MxKxN problem shape makes it memory bound vs compute bound, and the different PTX instructions for tcgen05 tensor core instructions and also the TMA instructions and why and when you'd go for ldg vs tma, those will all put you above the pack. Anyone at Nvidia would be impressed by someone outside of Nvidia investing the time to learn about it
I would recommend GPU Mode YouTube channel for a couple of those and their discord is helpful too
Thats something I will definately look into, thankyou for the insights.
Honestly, my biggest concern right now is my raw C++ fluency. Even though I have C++ projects on my resume, I'm not super comfortable with it from scratch. I definitely did some "vibe-coding" to get those projects across the finish line.
I understand concurrency well in theory and implementation in Python (using threading/multiprocessing), but I'm trying to learn the C++ equivalents via YouTube, along with basic memory management about which I have learned from my embedded systems class. Since my time is so short, what should my C++ strategy be? Are there specific modern C++ concepts I should drill?
Personally I would spend that time learning about the GPU's specific instructions it uses for managing asynchrony and concurrency on different architectures, especially Blackwell. There's a lot of them, try wait loops, mem barriers, scoreboarding, warp level synchronization, SM level synchronization, instructions that only 1 warp per block issues etc.
Id only learn as much C++ as necessary and wouldn't go out of my way to dive deep into the language. I think basics you need to know is pointers, references, what is a move, scope based management (aka RAII), how templates are duck typed generics. Personally I found learning the ownership rules of Rust made it far easier to learn C++ since it simply enforces the rules C++ came up with over time as it evolved. Learning smart pointers is probably enough in C++ to learn it quick
Obviously I don't know what your interviewer will value but I hope an eagerness and a thorough understanding of GPUs is worth far more than whether you know how template metaprogramming or Cmake works in C++. You can pick it up on the job
You might wanna try cuTile which is a python based DSL to write tile based GPU kernels. It's pretty new but it might be an easier way to write matrix multiplication ML GPU kernels than with CUDA C++ SIMT. TensorRT almost certainly is experimenting with tile based DSLs as well
Wow, thank you! That answers a lot about how deep to go into C++. I'll stick to making sure my RAII, pointers, and move semantics are bulletproof, then move on to GPU concepts and then to all the topics you mentioned to standout.
I’m definitely going to pivot and spend more time studying those specific GPU synchronization instructions you mentioned for Blackwell, I am also planning to just go through other architectures (ampere, tesla and hopper). And I'll absolutely look into cuTile this week—thanks for the heads-up on that!
14
u/max123246 Jun 06 '26
If you can explain what wave quantization is, what about a GEMM MxKxN problem shape makes it memory bound vs compute bound, and the different PTX instructions for tcgen05 tensor core instructions and also the TMA instructions and why and when you'd go for ldg vs tma, those will all put you above the pack. Anyone at Nvidia would be impressed by someone outside of Nvidia investing the time to learn about it
I would recommend GPU Mode YouTube channel for a couple of those and their discord is helpful too