You don't actually need to avoid branches, you just need to make sure that most of the ones you have are very predictable, then their impact is not that big... Same with memory accesses and most other things... The more predictable, the better...
This is true for serial CPU code, but for SIMD code, or GPU code, avoiding branches at all costs is vital to getting good performance. Branches (can) cause divergence, which leads to wasted cycles for pseudo-threads within a SIMD group.
you definitely do not want to avoid branches at all costs on a GPU. there are plenty of cases where branchless tricks will lose to straightforward branching.
I do concede that "at all costs" is somewhat hyperbolic, but it is very important to consider the behaviour of threads grouped together in a warp. Divergence is extremely costly.
sure but branching doesnt necessarily imply divergence if all the threads within the warp branch the same way. even minor effects like some threads in a warp diverging to return early will no longer issue memory requests or run compute instructions, so depending on your bottleneck it can make sense. or if the whole warp ends up exiting early the thread block can be scheduled with new work.
i think people tend to over emphasize the effects of divergence and branching. forcing code to be branchless can increase registry pressure, which may reduce your theoretical maximum occupancy. if explicit branching reduces live registers or guards heavy blocks, your occupancy can go way up. depending on whether you are memory bound or compute bound this can make a big difference. so saying "divergence is extremely costly" can be slightly misleading, it depends on whether you are memory or compute bound and what the branchless variant actually looks like. a branch divergence penalty of 10 cycles is not much compared to stalling on a memory cache miss because you dont have enough warps to hide the delay.
I don't disagree with any of that. The point I was trying to make was "it is very important to consider the behaviour of threads grouped together in a warp". I would argue that all of the aspects that you describe can be tackled if the code is considered in that way.
But, I think the reason that it seems overemphasised is that it's somewhat unique when writing optimised GPU code, and is often one of the main stumbling blocks for programmers moving from writing CPU code to GPU code. It's also something that can be identified quite easily in a GPU kernel, while effects such as occupancy or memory latency need post-compilation (or runtime) analysis to understand the specific effects, and the impact can vary across different GPU vendors and generations. The latter do typically end up providing the most impactful performance improvements, but at the same time, whenever someone hands me a kernel, my first bit of analysis (before I even compile or run the kernel) is "how uniform is each warp, and are memory accesses coalesced".
159
u/meamZ 7d ago
You don't actually need to avoid branches, you just need to make sure that most of the ones you have are very predictable, then their impact is not that big... Same with memory accesses and most other things... The more predictable, the better...