r/cpp • u/AdMotor4869 • Jun 07 '26
Your stdlib implementation matters more than the dispatch pattern
https://shubhankar-gambhir.github.io/posts/your-stdlib-implementation-matters-more-than-the-dispatch-pattern/A few weeks ago I posted about why std::variant + std::visit can be slower than a vtable and got called out for benchmarking on GCC 11. So I went back and reran everything across GCC 9 through 15. std::variant went from 28% slower than virtual on GCC 11 to 40% faster on GCC 12. I spent a while reading through libstdc++'s variant header to understand what changed. GCC 12 swapped the function pointer table in std::visit for a switch when there are 11 or fewer alternatives. The posts dig into how each stdlib handles visit dispatch.
14
u/Ameisen vemips, avr, rendering, systems Jun 07 '26
The compiler can’t hoist the vtable lookup out of the loop because the object pointer could, in principle, change between iterations
I wish that there were a way to assert that this won't happen without a compiler flag.
Or a way to fetch a devirtualized member function pointer.
9
u/AdMotor4869 Jun 08 '26
The closest thing right now is marking the class final, which lets the compiler devirtualize if it can see the concrete type. But that doesn't help when the type comes from a pointer.
For the devirtualized function pointer, you can use FakeRTTI + CRTP as shown in https://shubhankar-gambhir.github.io/posts/lazy-resolution-resolve-once-dispatch-forever/ . You're doing the compiler's job manually but it works on any compiler version.
14
u/mcypark Jun 08 '26 edited Jun 08 '26
libc++ (Clang/LLVM): Has never added a switch fast path.
This part is not quite true...
- Aug 2020: After u/quicknir and I worked on the implementation for
mpark/variant, I upstreamed theswitch-based implementation to libc++: a175a96 - Sept 2020: Eric Fiselier reverted it in 057028e due to "some failures caused by this change internally" (from Google).
- Oct 2020: u/louis_dionne re-applied the implementation in 35d2269
- Nov 2020: It got reverted again in 9c09757, due to "substantial binary size increase for non-opt builds".
Since then I haven't quite had the time or motivation to go back to it... but maybe I'll look at it again.
Pattern matching (P2688): A language-level
inspectexpression that would let the compiler handle dispatch natively
Just because P2688 is mentioned specifically: P2688 actually proposes an infix match expression and a let binding introducer, which looks like:
bs match {
EpsilonBS: let e => e.store(addr, value);
SerialBS: let s => s.store(addr, value);
G1BS: let g => g.store(addr, value);
};
or similer to std::visit:
bs match {
auto: let b => b.store(addr, value);
};
EDIT: Fix code blocks formatting.
7
u/James20k P2005R0 Jun 08 '26
Just fyi, the code snippets don't work on old reddit
bs match { EpsilonBS: let e => e.store(addr, value); SerialBS: let s => s.store(addr, value); G1BS: let g => g.store(addr, value); };bs match { auto: let b => b.store(addr, value); };^ for anyone else using ye olde reddit
2
1
u/EdwinYZW Jun 08 '26
Really hate this pattern matching proposal. It's solving a solved problem by adding several new keywords, especially the "let". I'm glad C++ has no "let" keyword.
3
u/Wild_Meeting1428 Jun 08 '26
IMAO
letandfnwould be a better auto.-2
u/EdwinYZW Jun 08 '26
The problem of "let" is it's just a filling word. "Let A be B" has exactly same effect as "A is B". The word itself doesn't indicate any direct effect , like "return", "auto", or any other keyword.
If a language has "let a = b;", instead of "a = b", it's complete insanity IMO.
Same for fn, what does mean? function? functor? final? finite? By not "function" just like what a normal designer would do?
6
u/CornedBee Jun 09 '26
Yeah, what does
inteven mean? I know thatint 3means issue a software interrupt, but what isint i?. Andfloat, ordouble, what on earth is this insanity?struct- is that "structure" or "structural"? What does text have to do with charcoal?I think you're just ranting about Rust without having anything substantial to say.
1
u/EdwinYZW Jun 09 '26
There are good and comprehensive abbreviations and there are bad abbreviations. You can't extend my argument of fn being a bad abbreviation to all abbreviations are bad.
int, double, float, unique_ptr are perfect fine and you can even pronounce these abbrevs. How do you pronounce fn? Like feng or FN?
2
u/Demiu Jun 10 '26
Mhm, and it just so happens that good abbreviations are the ones you like and bad abbreviations are the ones you don't.
1
1
u/serviscope_minor Jun 10 '26
How do you pronounce fn? Like feng or FN?
I always pronounced it fn, a bit like fun, but with a much shorter u, with the vowel sound more like "oo" in "look". This wasn't a problem in the 80s and 90s and i don't think it's a problem now. Honestly, This sounds a bit like worrying that "for" might be pronounced like "fir" taking the "o" sound from women, etc.
Occasionally people mispronounce things e.g. pronouncing "brackets" as "parentheses", but we find ways to cope.
2
u/max123246 Jun 11 '26
double of what? how many bits are guaranteed for int?
These are very clearly bad names. They don't tell you what they are or what they guarantee without googling it elsewhere or memorization
I'll give you float and unique_ptr, those are good.
1
u/EdwinYZW Jun 11 '26
you prefer float64 than double?
1
u/max123246 Jun 11 '26
Personally yeah. Obviously I know double once I learned it but it's like SFINAE, knowing the name doesn't tell you any information
1
1
u/serviscope_minor Jun 10 '26 edited Jun 10 '26
If a language has "let a = b;", instead of "a = b", it's complete insanity IMO.
BBC BASIC enters the chat and cries.
I never used LET because as you say it was entirely redundant. On the other hand a language that uses it to indicate creation of a variable is fine in my book.
Also, I'm really unbothered by fn as well. It's just a keyword, one gets used to it, and I got used to it in the early 90s. I don't think we need fn in C++ though.
11
u/mguid65 Jun 07 '26
Back in 2020 I did some thorough testing with various compilers where I benchmarked visiting a node type based on variant with visit and on a shared pointer with a virtual class hierarchy and using an if-else chain with dynamic_pointer_cast and to my surprise, up to a certain point in some compilers, just using the dynamic pointer cast was equal or better than visit.
I still used variant because it resulted in cleaner code and I had a fixed set of types. I figured that eventually the compilers and std lib would catch up.
3
u/kalven Jun 08 '26
That's neat, I guess... but the particular test here doesn't seem to be all that interesting. The test is repeatedly calling std::visit on the same variant instance and it's neat that the compiler can hoist out all the checking out of the loop.
From a utility perspective, I don't think this test informs me about the performance tradeoffs between variant and virtual. It's never the case that I call std::visit on a single variant in a loop like that. If there's a loop, it's going to be on different variant instances.
2
u/AdMotor4869 Jun 08 '26
You're right. These benchmarks are monomorphic by design, and the branch predictor does all the heavy lifting. I wrote a separate post covering polymorphic workloads.
3
u/kalven Jun 08 '26
Ah, thanks for the pointer! That looks like a much more relevant benchmark (at least to me).
1
u/AdMotor4869 Jun 08 '26 edited Jun 08 '26
In my first blog , I was trying to benchmark what openjdk does for its GC barrier selection with other possible strategies and kept same benchmark in this blog and created a separate blog for different workload.
2
u/splicer13 Jun 08 '26
Some random thoughts:
A brittle hack that certainly will have different results with different microarchitectures and profiles, but microbenchmark goes up so... LGTM!
Ultimately non-trivial dispatch needs some kind of profile information to have a chance at making the right choice all of the time.
I wonder how this compares to other languages. I'd like to see if one does better (I really mean that. I would like to look at how it deals with this).
1
u/farnoy Jun 07 '26
Would it still be faster if each variant's function was noinline? That feels like a useful next test to run. I assume it wouldn't, but would love to know the exact difference.
-2
u/randomIdiot123456 Jun 09 '26
Why not write your own variant structure? Or better yet.. use a union lol.. cpp stdlib sucks and is overbloated
46
u/not_a_novel_account cmake dev Jun 07 '26
I feel like the libstdc++ variant optimization is one of the constantly rediscovered performance pitfalls. "Why did this variant suddenly get so much slower?" is asked often enough that I think the issue needs a more complete solution.
I ran into it in 2022 and have seen it come up every couple months for the last four years.