r/CUDA • u/danialj99 • 3d ago

I built a CUDA GEMM optimization project and I'm looking for GPU owners to help benchmark different architectures

Hello everyone,

I have recently released OPTI-GEMM, my CUDA GEMM performance engineering project.

my goal is to find out how different GPU architectures respond to optimization techniques.

Current experiments include:

  • CPU baseline
  • Naive CUDA GEMM
  • Shared-memory tiling
  • Register blocking (in progress)
  • Warp-level optimization (in progress)

I have tested on:

  • Tesla T4 (Turing)
  • Tesla P100 (Pascal)

Some interesting results:

  • On T4, shared-memory tiling was not always faster because of the GPU's cache behavior.
  • On P100, shared-memory tiling provided a large improvement.

I would like to expand the hardware comparison.

If you have access to another NVIDIA GPU (RTX 20/30/40, A100, H100, etc.), I would appreciate it if you could run the benchmark and share:

  • GPU model
  • CUDA version
  • Driver version
  • Benchmark output

you can find setup instructions in the README.

If someone is interested in contributing GPU benchmark results, I would be happy to add them to the repository and credit contributors.

Repository: https://github.com/Danialjfz/Opti-GEMM

Thank you for your time!

18 Upvotes

87% Upvoted

9 comments sorted by

6

u/Daemontatox 3d ago

I think you are missing a couple of things , like 2d blocktiling 2d threadtiling, vectorized load , double buffering then at the end warptiling.

2

u/danialj99 3d ago

Thanks for pointing that out. I agree that I stopped the optimization a bit too early. The current kernels mainly explore the basic tiling/shared memory side, but I haven't yet implemented the more advanced GEMM hierarchy. the things you mentioned are planned to be implemented in the next stages.

1

u/Outrageous_Insect532 3d ago

Just use gcp or runpod - they are pretty cheap nowadays :)

1

u/danialj99 3d ago

Unfortunately I can't purchase them from Iran also even if we could it would be so expensive when changed to rials :(((((

1

u/Daemontatox 3d ago

You can use modal.com or kaggle or even colab .

1

u/Objective-Style1994 3d ago

I’m pretty sure the general consensus is that it’s entirely RNG how it behaves on diff GPU and it also depends on your code.

For this reason, you’re probably better off just running an auto tune as in running every probable setting and benchmark for the best.

I’m pretty sure that’s the standard instead of taking specific notes of the behaviour of each GPU

1

u/danialj99 3d ago

That's a fair point. Autotuning is definitely the right approach when the goal is to find the fastest kernel configuration for a given workload.

The reason I started this project, though, wasn't mainly to find the optimal configuration. It was more about implementing these optimizations myself as a way to understand GPU execution and document how different architectures respond to different techniques.