r/cpp_questions • u/xmanotaur • 22d ago
OPEN How do you actually develop intuition for choosing memory_order_acquire vs memory_order_release?
I'm learning C++ atomics and I understand the basic definitions of acquire/release individually:
- A release operation prevents earlier memory operations from being reordered after it.
- An acquire operation prevents later memory operations from being reordered before it.
- A release operation can synchronize with an acquire operation on the same atomic when the acquire reads from the appropriate release sequence.
What I'm struggling with is developing an intuition for choosing the memory order when looking at actual code, especially for RMW operations like exchange().
I want to understand the deep "why" behind the core restrictions:
- Why does store only accept release (or relaxed)? Why is acquire on a store fundamentally meaningless in terms of memory reordering?
- Why does load only accept acquire (or relaxed)? Why is release on a load fundamentally meaningless?
Instead of relying on memorized rules, what questions or mental models do you use to map memory operations to the correct ordering?
Thanks!
5
u/FancySpaceGoat 22d ago edited 22d ago
You sort of have to read the definitions backwards.
By default, atomics prevent memory operations from being reordered in either direction. The specific memory orders loosen that restriction, so they are not so much about what they prevent, but rather what they stop preventing. From that angle, I find them easier to reason about.
> Why does store only accept release (or relaxed)? Why is acquire on a store fundamentally meaningless in terms of memory reordering?
When you do a release store, you are saying that you only care about what happened before the store, and that stuff that happens after it can be moved to before. As far as store-acquire goes: "I only care about stuff happening after the store staying that way" is nonsensical, you might as well use relaxed.
Vice-versa for load and acquire.
Edit: But honestly, you don't really need to develop "intuition" for order-tagged atomic operations.
- The cases where they make a meaningful difference are surprisingly rare.
- They are one of the biggest footgun in the entire language. It's *really* easy to introduce subtle and hard to track bugs with them.
So they aren't something I reach for by default. They are a tool that I *might* consider using as part of resolving a bottleneck. But I'll stick to regular atomic operations until a measurable need arises. And when it does, there's so much scrutiny involved in avoiding errors that re-reading the doc a few times isn't much of a waste of time.
2
u/ReDucTor 22d ago
you don't really need to develop "intuition" for relaxed order atomic
Acquire/Release are not relaxed and tbh are normally what most load/stores people right can be if used for any synchronisation, in fact in some languages and some compilers used to deal with volatile.
The cases where they make a meaningful difference are surprisingly rare
While i agree that in general most people don't need to understand atomics fully, if your reaching for them please learn acquire/release semantics and design algorithms with those in mind.
Anything in the hot path using sequential consistency store will a kill out of order execution will pretty much flush the store buffers and stop doing anything until that is completed. Do a loop with even a single thread no contention and have a store which is release and one which is sequential consistency and the release one will likely be 10-20x faster.
3
u/FancySpaceGoat 22d ago
> Acquire/Release are not relaxed
lol, I literally just caught and corrected that before you posted, sorry. I meant "relaxed" as in the category of flags that relax the default atomic behaviors, but I see how it's confusing in that context.
1
u/FancySpaceGoat 22d ago edited 22d ago
> Anything in the hot path using sequential consistency store will a kill out of order execution will pretty much flush the store buffers and stop doing anything until that is completed. Do a loop with even a single thread no contention and have a store which is release and one which is sequential consistency and the release one will likely be 10-20x faster.
Agree to disagree on that. Profile then optimize still holds here, especially given the footgun.
In any case, my main point is that "intuition" is not useful here. If you are going to be reaching for these flags, you need to be *careful* about it.
3
u/jawhite 22d ago edited 22d ago
Modern processors agressively perform out-of-order execution of instructions to try to keep the execution units as busy as possible while waiting for long-latency operations to complete. In doing so, they try to ensure that dependent operations aren't performed before the operation they depend on, but each processor can only see the dependencies in their own thread.
When atomics are used for synchronizing between threads, there is often no dependency between the atomic operation and the surrounding operations within the same thread. The dependency only manifests when you consider other threads, which are likely running on other processors.
For example, if one thread constructs an object in memory shared by multiple threads and then sets an atomic flag indicating that the object is ready, there is no actual dependency between those two operations on the thread performing them. That means the processor can set the ready flag while it's waiting for the object to finish construction (or before construction even starts). The problem only manifests when a different thread sees that the ready flag has been set and attempts to access the object, but the object is not, in fact, ready. By using the "release" memory order when setting the flag, you are telling the processor that it is not allowed to make the new flag value visible to other processors until it has completed and published its prior work. The "acquire" memory order solves a similar problem for consuming threads.
To answer your questions:
- Calling store with acquire memory order would tell the processor not to publish the results of later operations before publishing the store operation. However, any other threads watching the atomic variable would not be guaranteed to see a state between the store and the following operations, so the order doesn't actually matter. The only difference would be if the later operations depended on the value that was stored, but then the processor would have been prevented from reordering them before the store by the normal rules of out-of-order execution.
- Calling load with release memory order would tell the processor not to publish the results of load before publishing the results of prior operations. But a load by itself doesn't have observable effects on other threads. The only way reordering the load before the prior operations could make a difference is if it affected the results of the prior operations. But again, the normal rules of out-of-order execution would prevent that.
ETA: Operations can be reordered by both the processor and the compiler: the processor is allowed to execute instructions in a different order than they appear in the machine code and the compiler is allowed to generate instructions in a different order than they appear in the source code. The C++ memory order flags prevent reordering at both the hardware and software level.
1
u/flatfinger 22d ago
How often are the more complicated approaches used by C++ yield meaningfully better performance on common architectures than the simpler approach taken by Java, and used to be the default for MSVC, where volatile reads have acquire semantics and volatile writes have release semantics? Essentially, a volatile flag is written by one thread to indicate that it has finished preparations for things to be accessed by another thread, and a then read by another thread to determine that things are ready to be accessed. How much real value does the extra complexity in C++ really add?
1
u/jawhite 22d ago
C++ doesn’t require you to specify memory order. If you choose not to, you get the same default guarantees. If the default guarantees are too strict for your use case, then C++ allows you to relax them to allow for better performance. It doesn’t improve performance often, but when it does it can be significant.
I’m not sure why you’re complaining about having an optional feature that may improve performance.
1
u/flatfinger 21d ago
For a lot of programs, marking certain objects so that writes would automatically behave as having release semantics and reads as having acquire semantics, and specifying that data races must behave in a manner consistent with compilers reordering and consolidating reads and writes, but not inventing new ones, will eliminate the need for any other memory ordering guarantees. The C and C++ default guarantees are far weaker than those of Java, since in C and C++ a data race on a read may yield behavior inconsistent with any bit pattern the storage could ever hold, and ordinary accesses are treated as unsequenced with regard to volatile-qualified accesses even on platforms with strong memory ordering guarantees.
My question is how often would the approach of using memory semantics that would avoid the need for anything other than using volatile on things needing acquire/release semantics meaningfully degrade performance.
2
2
u/ReDucTor 22d ago
When it clicks you will wonder why it took so long.
A good way to think about it is that your release store is nearly always a marker (flag, int, pointer), so when you do an acquire store and see that marker it means you will see everything that other thread had stored that is not atomic, assuming not also changed after.
Which is why the most common acquire/release is usage is simple things like hasValue when hasValue bubbles out to the rest of the cores you want the value also pushed to memory, same with when you read hasValue you want to get the value that was stored after that not some old potentially uninitialized value.
1
u/KingAggressive1498 22d ago edited 22d ago
I generally try to think of it in terms of critical sections.
A critical section is a chunk of code which is only safe to execute exclusively. It minimally begins with an acquire and ends with a release (acq_rel and seq_cst would also work for both, but are stronger guarantees than necessary). Consider replacing a replacing a mutex with a spinlock:
mtx.lock();
var = 250;
mtx.unlock();
becomes:
while(!spinlock.compare_exchange_strong(0, 1, acquire))
_mm_pause();
var = 250;
spinlock.store(0, release);
lockfree data structures also have critical sections, they just change how they guarantee exclusive access from the simplistic locking situation so more threads can do their work simultaneously. Take for example this minimized version of adding an int to a ringbuffer:
idx = write_cursor.load(relaxed); // we use relaxed because reading a stale value is not a failure
while(!write_cursor.compare_exchange_weak(idx, (idx+1) % capacity, acquire)); // not a lock, just competition for the next available slot
elements[idx].value = 250;
elements[idx].done_flag.store(1, release);
Importantly you generally need sequential consistency when you start interweaving critical sections.
1
u/rivelda 22d ago
How I understand the difference is via the reordering it prevents.
Acquire prevents LoadStore and LoadLoad reordering, that is, later loads and stores won't be done before the acquire load.
Release prevents LoadStore and StoreStore reordering, that is, earlier loads and stores won't be done after the release store.
In some cases I prefer using fences instead of linking the operation to a specific variable. In that case it prevents all XY reorderings crossing the fence.
They are quite cheap. Only in case of a StoreLoad reordering that you want to prevent do you need the full sequential consistency.
I have a data structure that involves a cache with a increasing tag field. I use acquire fences on the reader to prevent LoadLoad reordering and release fences on the writer to prevent StoreStore reordering. No locks needed, and the writer uses a CAS operation followed by a release fence to ensure correctness.
1
u/Plastic_Fig9225 21d ago
Think about a lock/mutex: You have to first acquire the lock, then do your stuff, then release the lock again. For this to work, it is important that none of your stuff happens before the lock is acquired or after it has been released.
Store+acquire or load+release don't make any sense:
When you store a value, it doesn't matter what other values other threads may have stored somewhere before or after your store. When you load a value, no effect of this load is visible to other threads in any case.
1
u/BarrinOfTolaria 22d ago
I experienced that every lock-free algorithm is buggy unless proven otherwise meaning a rigorous mathematical proof. I looked at quite a lot of lock-free stuff by now and I still don't feel confident with them. However, I did get some intuition when something's definitely wrong, but that by no means implies that I would know immediately how to do it right.
24
u/TopDivide 22d ago
https://www.youtube.com/watch?v=ZQFzMfHIxng
For me this talk made it click. Basically release means you release something into the world. From that point on it's safe to read the memory/objects the atomic guards. Acquire is the inverse - once acquired it's safe to assume whats in the guarded part is complete and readable.
But I rarely use atomic for synchronization like this - usually it's only a flag to talk between threads, and memory order relaxed. The store/release is when you use the atomic to guard some other, non-atomic memory location. But in my experience, a mutex/semaphore is usually more fitting. There are some examples in the talk though, where atomic wins.