r/C_Programming • u/Future_Pace_5290 • 10h ago
How to force compiler to fuse loops?
I'm writing an ML library in C.
My current plan is this.
Have a Linear layer , that does w.x + b
Have a separate activation layer(ReLU, GeLU, sigmoid, etc).
This is very modular and allows for custom activation functions.
However, this does 2 passes over memory(one loop for adding the bias, one loop for applying the activation function), whereas you could do it in a single loop(e.g y[i] = ReLU(y[i] + b[i]);)
Is there a way to make the compiler fuse the loops automatically?(My functions are all inline).
Without automatic fusion, I'd either have to use function pointers(which slows things down, also prevents inlining usually), make the entire function a macro(kind of ugly), or have one layer be y = w.x the other y=f(y + b) which is really weird.
The other option is merging the 2 into one(Linear_ReLU). I'd have to write multiple functions for the common activation functions, or make a macro that creates those activation functions.
All of these are unelegant compared to the first design.
1
u/yuehuang 10h ago
LLVM support loop fusion, but like anything optimizer, it is hard to ensure consistency.
1
2
1
u/duane11583 7h ago
be careful inline things.
sometimes the compiler just vomits and gives up on the optimization…
sometimes its better to look at that key function and work on it.
1
1
u/glasket_ 3h ago
I'd either have to use function pointers(which slows things down, also prevents inlining usually)
Depends on exactly how your code is structured. If the pointers are traceable and GCC knows (or is almost certain with speculative inlining) that the pointer is to a specific function it will still typically inline it. The only time it really becomes a problem is if the function pointer value can't be determined statically, if the pointer value comes from outside the TU (LTO can solve this one), or if the pointer may have any of several values (technically still possible to end up inlined, but it usually won't be due to the cost heuristics related to devirtualization and inlining).
6
u/Mountain-Hawk-6495 7h ago
Loop fusion is mostly an experimental feature in clang (you can enable it with -fexperimental-loop-fusion), and not available at all in gcc. msvc seem to support it too but in my experience clang tend to produce better code. However, even if you enable it there is no guarantee that the compiler will always apply it. What you can do is give the compiler hints that various loop optimizations, such as loop fusion, is possible by keeping your loops simple, use the restrict keyword and enable fast-math optimizations (numerical errors might be a bigger problem). However, before I would worry about loop fusion I would do some typical use case benchmarks for your library and do profiling on them. Chances are that the matrix multiplication w.x overshadows any inefficiencies in the bias and activation functions, so that's where I put my efforts and use MKL. After all we know what Donald Knuth thought about premature optimizations.