Skip to main content

Memory Fences and the C++ Memory Model

Adapted from: Jason Gregory, Game Engine Architecture, 3rd ed. (CRC Press, 2018), Chapter 4, Sections 4.9.3–4.9.6 — rewritten and condensed for teaching.

Causes of Data Races and Atomic Instructions named three causes of data races, but only fully solved the first one — critical-operation interruption — via atomic instructions. This lesson tackles the other two: instruction reordering (by the compiler and the CPU) and memory ordering effects (caused by multicore cache coherency itself). Both are real, both are subtle, and both are avoidable once you know what to reach for.

How Instruction Reordering Breaks Concurrent Code​

Compilers and CPUs both reorder instructions aggressively to hide pipeline stalls — and both are engineered under one ironclad rule: the reordering must be invisible to a single thread. That rule says nothing at all about other threads watching the same memory. Take this simplified, deliberately mutex-free producer-consumer pair:

int32_t g_data = 0;
int32_t g_ready = 0;

void ProducerThread() {
    g_data = 42;
    g_ready = 1;   // tells the consumer data is ready
}

void ConsumerThread() {
    while (!g_ready) { /* spin */ }
    assert(g_data == 42);   // could this ever fail?
}

Even if aligned 32-bit reads/writes are atomic on this CPU, nothing here stops the compiler — or the CPU's out-of-order logic — from silently swapping the two writes in ProducerThread, so g_ready becomes visible before g_data does. The reordering is invisible within ProducerThread itself (a single thread can't tell the difference), which is exactly the loophole that lets it happen. Reordering also isn't limited to reordering whole C++ statements — it happens at the assembly level, which can be far more granular than the source suggests. A = B + 1; B = 0; might compile to:

mov  eax, [B]
add  eax, 1
mov  [A], eax
mov  [B], 0

...and a compiler is free to reorder this into:

mov  eax, [B]
mov  [B], 0     ; B is zeroed before A is ever written!
add  eax, 1
mov  [A], eax

— perfectly harmless in a single thread, but capable of breaking a second thread that's specifically waiting for B to hit zero before trusting the value in A.

volatile Doesn't Save You Here​

C/C++'s volatile qualifier looks like the obvious fix — it guarantees a variable's reads/writes won't get optimized away or cached in a register. But that's all it reliably guarantees. volatile was really designed for memory-mapped I/O and signal handlers, and whether it also happens to block instruction reordering varies by compiler, target CPU, and command-line flags — the C/C++ standards themselves don't require it. And even in the best case, volatile does nothing whatsoever to stop the CPU's own out-of-order execution logic from reordering things at runtime, nor to prevent the cache-coherency effects covered later in this lesson. In C and C++, volatile is simply the wrong tool for concurrency — full stop. (Java and C# are a different story: their volatile genuinely does provide atomicity and ordering guarantees, precisely because those languages run inside a VM that fully controls instruction scheduling.)

Compiler Barriers​

The real, reliable fix for compiler-level reordering: explicitly tell the compiler not to reorder across a specific point in the code, via a compiler barrier — asm volatile("" ::: "memory") on GCC, or _ReadWriteBarrier() on MSVC. Ordinary function calls act as an implicit compiler barrier too, for a simple reason: the compiler generally can't see what side effects an opaque function call might have on memory, so it can't safely assume memory looks the same before and after the call.

A compiler barrier alone still isn't the whole answer, though — it says nothing about the CPU's own out-of-order execution engine reordering things at runtime. That requires a different tool entirely, covered next: the memory fence.

Check: instruction reordering​

Quiz

Instruction reordering and volatile

1 / 3

Why can a compiler legally reorder the two writes in ProducerThread (g_data = 42; g_ready = 1;) even though doing so can break the consumer thread?

Pick an answer to see if you are right.

Memory Ordering: Why Multicore Caching Adds a Second Problem​

Even with compiler and CPU reordering fully solved, a second, independent source of trouble remains on any real multicore machine: cache coherency.

A Quick Refresher on Caching​

A CPU core prefers reading and writing its local cache over slower main RAM whenever possible. Writes get applied to the cache immediately, with the actual write-back to main RAM deferred until later — a classic, essential performance optimization, engineered (like everything else so far) to be invisible to a single thread.

MESI: Keeping Multiple Cores' Caches Honest​

On a multicore machine, each core typically has its own private L1 cache, sharing a higher-level cache and main RAM with the others. A cache coherency protocol — most commonly MESI — lets cores exchange cache-line data directly with each other, rather than always bouncing through slow main RAM. Under MESI, every cache line sits in one of four states:

  • Modified — written locally; this core's copy is the only up-to-date one.
  • Exclusive — only this core has a copy at all, and it hasn't been modified.
  • Shared — multiple cores hold identical copies.
  • Invalid — this core's copy is stale; the next read has to fetch fresh data from elsewhere.

Walking through the producer-consumer example on a two-core machine: Core 1 (producer) first reads g_ready, pulling its cache line in as Exclusive. Core 2 (consumer) then reads the same line — both cores now hold it Shared. When Core 1 writes g_ready, its own copy flips to Modified, and an Invalidate message goes out over the interconnect bus, flipping Core 2's copy to Invalid. The next time Core 2 tries to read g_ready, it detects the invalidation, fetches Core 1's fresh value over the bus, and both copies settle back into Shared — with a write-back to main RAM triggered along the way. This is a genuinely elegant, largely self-managing system for keeping every core's view of memory consistent.

Where MESI Still Goes Wrong​

If MESI keeps every core's caches consistent, where do memory ordering bugs come from at all? One word: optimization. Real MESI implementations defer certain operations to shave off latency — again, engineered to stay invisible to a single thread, and again, this is exactly where concurrent programs get the short end of the stick.

Concretely: the producer writes g_data = 42 and then g_ready = 1. If Core 1 already happens to have g_ready's cache line locally but not yet g_data's, MESI-level optimization can let the new value of g_ready become visible to other cores before the new value of g_data does — even though both instructions genuinely executed in the intended order on Core 1. The consumer can observe g_ready == 1 while still seeing the old value of g_data, purely because of cache-coherency-level reordering, with the compiler and CPU behaving perfectly correctly the entire time.

The rule that ties this all together

Optimizations within a cache coherency protocol can make two reads and/or writes appear, from another core's point of view, to have happened in the opposite order from how they were actually executed.

Check: MESI and memory ordering​

Quiz

Cache coherency and memory ordering bugs

1 / 3

Under the MESI protocol, what does it mean for a cache line to be in the "Invalid" state?

Pick an answer to see if you are right.

Memory Fences: The Real Fix​

A memory fence (or memory barrier) is a special machine instruction that solves both remaining problems at once: it stops the compiler and the CPU's out-of-order logic from reordering instructions across it, and it prevents the memory-ordering effects MESI-style optimizations can introduce. Reasoning about specific fence instructions is easiest in terms of the semantic guarantee they provide, rather than their exact hardware mechanics:

  • Release semantics — a write with release semantics can never be passed by any read or write that comes before it in program order. Used on the second of two writes in a producer, this guarantees every prior write is fully visible before the release-tagged write becomes visible to anyone else.
  • Acquire semantics — a read with acquire semantics can never be passed by any read or write that comes after it in program order. Used on the first of two reads in a consumer, this guarantees nothing after it can be observed as happening before it.
  • Full fence semantics — the strongest, fully bidirectional guarantee: nothing crosses the fence in either direction. Genuinely useful, but also the most expensive to implement in hardware — which is exactly why most real fence instructions on real CPUs provide the cheaper, directional acquire or release semantics instead, reserving full fences for when you truly need both directions at once.

Applying this to the producer-consumer example directly: place a release fence immediately before the write to g_ready (making it a write-release), and an acquire fence immediately after the read of g_ready (making it a read-acquire):

int32_t g_data = 0;
int32_t g_ready = 0;

void ProducerThread() {
    g_data = 42;
    RELEASE_FENCE();   // ensures g_data's write is visible before g_ready's
    g_ready = 1;
}

void ConsumerThread() {
    while (!g_ready) { /* spin */ }
    ACQUIRE_FENCE();   // ensures we don't see g_data until it's truly ready
    assert(g_data == 42);   // now guaranteed to hold
}

Real ISAs expose these guarantees under CPU-specific instruction names: x86 provides sfence (release), lfence (acquire), and mfence (full fence) — though x86 is already strongly ordered by default, so explicit fences matter less there than on more weakly-ordered CPUs like PowerPC (sync/lwsync/isync) or ARM (dmb/dsb/isb, plus dedicated one-way ldar/stlr instructions). A CPU with weak default ordering (the DEC Alpha is the historically infamous example) needs fences almost everywhere; one with strong default ordering (x86) needs them only in specific, narrower situations.

C++11 std::atomic: Portable Fences Without the Assembly​

Hand-writing fence instructions is tedious, error-prone, and completely non-portable — which is exactly the gap std::atomic<T> fills. By default, every std::atomic operation applies full fence semantics, so the earlier example becomes trivially safe just by declaring the shared variables atomic:

std::atomic<int> g_data{0};
std::atomic<int> g_ready{0};

void ProducerThread() {
    g_data = 42;
    g_ready = 1;
}

void ConsumerThread() {
    while (!g_ready) { /* spin */ }
    assert(g_data == 42);   // guaranteed by std::atomic's default full-fence semantics
}

When full-fence-everywhere proves too expensive (verified by profiling, not guessed at), std::atomic's load()/store() accept an explicit std::memory_order, letting you dial in exactly the semantics you actually need:

std::memory_orderGuarantee
relaxedAtomicity only — no ordering guarantee at all
consumeNo later read/write in this thread can be reordered before this read (weaker than acquire; rarely used directly in practice)
releaseNo earlier read/write in this thread can be reordered after this write, and the write becomes visible to other threads' acquire-reads
acquireNo later read/write in this thread can be reordered before this read, and writes from other threads become visible after it
acq_rel / defaultThe strongest option — a full fence
std::atomic<int> g_data{0};
std::atomic<int> g_ready{0};

void ProducerThread() {
    g_data.store(42, std::memory_order_relaxed);
    g_ready.store(1, std::memory_order_release);
}

void ConsumerThread() {
    while (!g_ready.load(std::memory_order_acquire)) { /* spin */ }
    assert(g_data.load(std::memory_order_relaxed) == 42);
}
Relaxed ordering is easy to get wrong

Treat relaxed memory orders the way you'd treat any other risky micro-optimization: reach for them only once profiling has proven the default full-fence cost actually matters, and verify the relaxed version still behaves correctly. The default (full fence) is the safe choice for a reason.

Practice: fix a racy producer-consumer with explicit fences​

Convert an unsynchronized producer-consumer pair into one with the correct release/acquire semantics, using explicit std::memory_order arguments.

A Note on Interpreted Languages​

Everything in this lesson applies to compiled languages (C, C++, assembly) that run as raw machine code, where the compiler and CPU are both real, physical optimization concerns. Interpreted languages running on a managed virtual machine — Java on the JVM, C# on the CLR — are a genuinely different story: the VM controls scheduling and bytecode execution entirely in software, which is exactly why Java's and C#'s volatile can guarantee real atomicity and ordering (unlike C/C++'s), something explicitly called out earlier in this lesson. The underlying principles you've just learned transfer directly — but the concrete mechanisms differ, since a VM-mediated environment has strictly more control over execution than raw hardware does.

Key takeaways
  • Instruction reordering by the compiler and CPU is engineered to be invisible to a single thread — which is exactly the loophole that lets it silently break two threads relying on a specific write order.
  • volatile in C/C++ does not reliably solve this — it only guarantees a variable isn't cached in a register, nothing about instruction reordering or cache coherency. (Java/C# volatile is a different, stronger guarantee.)
  • MESI cache coherency keeps multiple cores' caches consistent — but its own internal latency optimizations can still let one core's writes become visible to others in a different order than they actually executed, a distinct bug class from instruction reordering.
  • Memory fences solve both instruction reordering and memory ordering at once, expressed through release, acquire, and full fence semantics — release goes before a producer's "ready" write, acquire goes after a consumer's "ready" read.
  • std::atomic<T> gives you all of this portably, defaulting to safe full-fence semantics, with explicit std::memory_order available (and genuinely risky) once profiling proves you need something cheaper.

Further Reading​