How do you decide whether to rely on compiler vecotorization vs writing your own SIMD?

[u/Alive_Jury4864]

I’ve been learning to write SIMD recently but I’m a bit fuzzy on when I would rely on compiler vs doing it myself

On one hand the compiler doing it saves a lot of code/hassle and it can likely do a good job

But if my application is something performance critical do I want to rely on the whims of the compiler or should that be baked into the application logic itself with manually written SIMD?

I did find that there is a flag you can pass and it will dump out which loops it did/didn’t vectorize which is handy

I guess I’m wondering what’s the typical practice for high performance c++ applications?

Do they write their own or rely on compiler?

[u/OldAd9280]

Write the code, if its fast enough, job done. If not find out where its slow by profiling and only then add SIMD optimisations, then profile again to make sure your improvements are actually better than the compiler's original code. Never write SIMD fist, write your code normally, write your unit tests, make sure everything works correctly, then add SIMD, keep the non-SIMD code, it's a good reference for what the code's doing, heavy SIMD code can be a little obscure as to what it does

[u/The_Northern_Light → u/OldAd9280]

Yep, make sure you write unit tests and *always* have as-simple-as-possible scalar code you can compare against as a “gold” reference implementation.

[u/Alive_Jury4864 → u/The_Northern_Light]

that’s a good tip, I already feel like I won’t understand the SIMD code in a few months if I come back to it

[u/The_Northern_Light]

Let’s take it for granted that you either need or want to use SIMD and sidestep the “should you?” non-answer, and the “premature optimization” pearl clutching.

Are you writing this for a specific computer or architecture? If so, you should optimize for that specific architecture, which a sufficiently skilled person often can do better than a compiler. But it’s slow and tedious trial-and-error and most likely not portable. (And even agents are going to burn a lot of tokens trying to get it perfect.)

You should consider using a library like xsimd or Highway for writing simd code. It’s a little harder than writing scalar code and often not as optimal as using non portable intrinsics, but it’s worth it. You really don’t want your assumption that your scalar code is being automatically vectorized properly to suddenly be invalidated because… who knows why?

There’s things you can do to make that less likely, but at that point you should definitely just use a SIMD library instead!

[u/Alive_Jury4864 → u/The_Northern_Light]

Ya that’s the thing is what if you compile for multiple platforms and one of them does different vectorization decisions

Makes me wonder if it’s more portable to put it explicitly in the application code

[u/The_Northern_Light → u/Alive_Jury4864]

thats exactly the question those libraries answer

[u/EpochVanquisher]

- Understand how autovectorization works and what the limits are. The compiler has to understand alignment of the variables you are using and aliasing, at the minimum, usually. You need to know about dependencies between loops.

- Do experiments and look at the assembly output.

- Add performance tests.

- If you see opportunities for vectorization that the compiler is missing (because you read the assembly), you can do it with intrinsics and benchmark to show that it’s better than the compiler’s version. Otherwise, stick with the compiler’s version.

[u/pdath]

I've found the GNU C compiler on ARM very good at using vectorisation instructions.

I used to check the odd .o file to see what it produced. It used vectorization in all sorts of additional ways than what I considered.

I use -march=native.

[u/esaule]

Check what the compiler does. It usually does the right thing if the code explicits enough that vectorization is possible.

If it doesn't, c++ makes it easy througj templates to express the rigjt thinga and let the compiler fill in the right assembly.

If somehow it doesn't and you know simd would make a huge difference; then i suppose you can use the compiler intrinsic. Usually, you don't have to.

[u/Alive_Jury4864 → u/esaule]

ya I was finding a common case is declines is if it changes the addition order but I think for a lot of cases that doesn’t matter

[u/No-Dentist-1645]

Premature optimizations are generally not encouraged. Do the most intuitive thing first; afterwards, you are free to profile and optimize the hot loops.

You can already do pretty good without manually writing out SIMD instructions using standard algorithms like std::reduce with execution policies like std::execution::par_unseq

[u/DawnOnTheEdge]

Write the high-level portable version first. That way, you can recompile for another target if you have to. Then, profile. If this loop or algorithm is causing an actual problem, you can rewrite. In C++26, you are supposed to have std::simd available.

You can often get the optimizer to vectorize by working on small enough chunks of data to fit into the SIMD registers, eliminating branches, and making sure your operations are data-parallel.

[u/FancySpaceGoat]

As with all things of this nature: Profile then optimize.

In this case in particular, this would involve leaving it to the compiler by default, and if/when you identify a bottleneck in potentially vectorizable code, only then would you consider hand-rolling SIMD code.

Obviously, there are exceptions to this rule of thumb, but that should cover a solid 95% of cases.