Mac Forum / Programming / Mac Programming / September 2007

Multiple CPUs can read from the same memory location without a problem.
With one important exception, multiple CPUs are the same, in theory, as a
single CPU running multiple threads.

The main problem with multithreaded access to shared data is ensuring data
consistency. A classic example is a simple counter, where you have
multiple threads incrementing a single variable. On many architectures, a
single increrment turns into three separate instructions: one read, one
add, and one write. If another thread increments the variable between the
read and the write, the data is no longer consistent, because the thread's
local copy of the value is out of sync. The result here is that you lose
increments.

Having multiple CPUs will make this worse. The window of time to break
this code is extremely small. It is far more likely that the other CPU
will hit it than a single CPU hitting its timer interrupt and switching
threads. But although it doesn't make it worse, it doesn't create a
problem where there is none, it merely magnifies it.

The problem that exists with multiple CPUs which does not exist on single
CPUs is one of ordering. For example, imagine you have a shared variable
to indicate that some data structure is ready. Thread A checks it
periodically to see if it's 0 or 1. When it's 1, it grabs a pointer to a
data structure out of another shared variable and goes to work on it.
At the same time thread B constructs this data structure. When it's
complete, it first sets the shared pointer to the structure, and then
sets the shared flag to 1.

On a single CPU, if the compiler doesn't mess with the order of your
writes, then this is perfectly safe. You set the shared pointer before you
set the flag, so that when thread A reads the flag as 1, it is guaranteed
to find a valid pointer.

On multiple CPUs, this is no longer safe. The problem is that CPUs are
free to re-order reads and writes to make things more efficient. And while
they have fancy electronics to ensure that things still look like they
happened in the same order on the same CPU, this is not assured between
CPUs. So Thread B runs on CPU 1 and writes the pointer first, then the
flag, but CPU one can decide to write the flag first and wait on the
pointer. Then Thread A on CPU 2 reads the flag, then follows the pointer,
and explodes.

The fix here is to place a memory barrier between the two reads and the
two writes. A memory barrier forces the CPU to finish all memory
operations before the barrier before it does any operations after the
barrier.

Note that in general you should never have to worry about memory barriers
and such. These definitely qualify as highly advanced techniques. Any time
you have shared data, you should use a lock. Locks automatically take care
of this trickery and are much more flexible and easy to use. If you have
any data shared between threads which is read/write, protect it with a
lock. Or better yet, don't have any such data, and communicate between
your threads with messages.