With the need for better scaling on increasingly larger multi-core
systems, we've continued to extend our CPU barriers in the kernel. Two important variants to prevent CPU reordering for lock-free shared memory synchronization are pairs of load/acquire and store/release barriers; also known as LOCK/UNLOCK barriers. These enable threads to cooperate between each other.
Multiple, yet pretty much equivalent, definitions of acquire/release semantics can be found all over the internet, but I like the version from the infamous 'Documentation/memory-barriers.txt' file for three reasons: (i) it is clear and concise, (ii) it explicitly warns that they are the minimum operations and not to assume anything about reordering of loads and stores before or after the acquire or release, respectively. Finally, (iii) it strongly mentions the need for pairing and thus portability:
(5) ACQUIRE operations.
This acts as a one-way permeable barrier. It guarantees that all memory operations after the ACQUIRE operation will appear to happen after the CQUIRE operation with respect to the other components of the system. ACQUIRE operations include LOCK operations and smp_load_acquire() operations.
Memory operations that occur before an ACQUIRE operation may appear tohappen after it completes.
An ACQUIRE operation should almost always be paired with a RELEASE operation.
(6) RELEASE operations.
This also acts as a one-way permeable barrier. It guarantees that all memory operations before the RELEASE operation will appear to happen before the RELEASE operation with respect to the other components of the system. RELEASE operations include UNLOCK operations and smp_store_release() operations.
Memory operations that occur after a RELEASE operation may appear to happen before it completes.
The use of ACQUIRE and RELEASE operations generally precludes the need for other sorts of memory barrier (but note the exceptions mentioned in the subsection "MMIO write barrier"). In addition, a RELEASE+ACQUIRE pair is -not- guaranteed to act as a full memory barrier. However, after an ACQUIRE on a given variable, all memory accesses preceding any prior RELEASE on that same variable are guaranteed to be visible. In other words, within a given variable's critical section, all accesses of all previous critical sections for that variable are guaranteed to have completed.
This means that ACQUIRE acts as a minimal "acquire" operation and RELEASE acts as a minimal "release" operation.
 | |||
| Thread B's ACQUIRE pairs with Thread A's RELEASE. Copyright (C) IBM. |
In lock-speak, all this means is that nothing leaks from the critical region that is protected by the primitive in question. A thread attempting to take a lock will synchronize/pair the load (ACQUIRE), for instance via Rmw (cmpxchg), when attempting to take the lock with the last store (RELEASE) when another thread is concurrently releasing the lock (for example, setting the counter to 0).
For v4.2, Will Deacon introduced more relaxed extensions of traditional atomic operations (including Rmw) which allow finer grained control over, what used to be, full barriers semantics on both sides of the instruction. This is also true for just about all atomic functions that return a value to the caller, ie: atomic_*_return(). As such weakly ordered architectures can make use of these -- currently only arm64 makes use of them, but efforts for PPC are being made.
- *_relaxed: No ordering guarantees. This is similar to what we have already for the non-return atomics (e.g. atomic_add).
- *_acquire: ACQUIRE semantics, similar to smp_load_acquire.
- *_release: RELEASE semantics, similar to smp_store_release.
So we now have goodies such as atomic_cmpxchg_acquire() or atomic_add_return_relaxed(). Most recently, aiming for v4.4, I've ported all our locks to make use of these optimizations, which can save almost half the
amount of barriers in the kernel's locking code -- which is specially nice under low or regular contention scenarios,
where the fastpaths are exercised. There are plenty of other examples of real world code making use of acquire/release semantics. Mostly by using smp_load_acquire()/smp_store_release() other primitives also use these semantics for common building blocks (as esoteric as they can get, ie RCU).
SOCIAL SHARE CARD GENERATOR