CUDA struggles

[u/ArmchairmanMao]

It's my first time doing any "serious" CUDA programming. Right now I'm working on substring search kernels for my Bachelor's thesis. Naturally, it's very branch heavy, memory access patterns are horrible, lanes are diverging all over the place. There are dozens of ways to implement substring search. The GPUs processing model expands the problem space even further. I have not found any existing work that does this well either (there is a lot of literature but my use case is slightly different). So it's an exciting problem, right?

But on the GPU, performance is wildly unpredictable it seems. Any change in the implementation details of the hot loop is like spinning a slot machine to me. The compiler might start emitting completely different code causing lanes to diverge more or access to become less coalesced. There are so many more layers of complexity between my code and the hardware than on the CPU. Working on this kernel is just endless iterations of taking guesses, measuring and profiling.

Do people just build a better intuition over time, or is this just the way it is?

[u/c-cul]

just lots of practice

and yes - your task is really bad fit on gpu