Re: C++11 atomic library notes

[Jeffrey Yasskin <jyasskin@google.com>]

On Fri, Sep 30, 2011 at 1:36 PM, Andrew MacLeod <amacleod@redhat.com> wrote:
> I've been working on GCC's C++11 atomic implementation. In discussions with
> Lawrence, I've recently discovered a fundamental change in what libstdc++-v3
> is likely to provide as far as an implementation.
>
> Previously, header files provided a choice between a locked or a lock-free
> implementation, preferring the lock-free version when available on the
> architecture and falling back to the locked version in other cases.
>
> Now the thought is to provide lock-free instructions when possible, and fall
> back to external function calls the rest of the time. These would then be
> resolved by an application or system library.
>
> If proceeding with that change, it would be convenient to make the same
> calls that other implementations are going to use, allowing OS or
> application providers to simply provide a single library with atomic
> routines that can be used  by multiple C++11 compilers.
>
> Since GCC 4.7 stage 1 is going to end shortly and it would be nice to get
> the cxx-mem-model branch integrated, I quickly wrote up what the current
> plan for the branch is regarding these external calls and such and brought
> up a couple of issues.  Its located in the gcc wiki at:
> http://gcc.gnu.org/wiki/Atomic/GCCMM/LIbrary
>
> Its my first cut at it, so hopefully its mostly correct :-)
>
> If anyone has any interest or input on this subject, the sooner it is
> brought up the better!

I wanted to comment on
http://gcc.gnu.org/wiki/Atomic/GCCMM/LIbrary#volatility. Say we have:

typedef pair<int*, int64> DWord;
std::atomic<DWord> shared_var;

void thread1() {
  use(shared_var.load());
}

void thread2() {
  // It's legal to add "volatile" in a pointer or reference cast.
  volatile std::atomic<DWord>* v_shared_var = &shared_var;
  // Now this looks identical to an access to a real volatile object.
  v_shared_var->store(DWord(ptr, val));
}

If, as the document proposes, "16 byte volatile will have to call the
external rotines, but 16 byte non-volatiles would be lock-free.", and
the external routines use locked accesses for 16-byte volatile
atomics, then this makes the concurrent accesses to shared_var not
thread-safe. To be thread-safe, we'd have to call the external
routines for every 16-byte atomic, not just the volatile ones, and
those routines would have to use locked accesses uniformly rather than
distinguishing between volatile and non-volatile accesses. Not good.

Even worse, on LL/SC architectures, every lock-free RMW operation
potentially involves multiple loads, so this interpretation of
volatility would prohibit lock-free access to all objects.

I see two ways out:
1) Say that accessing a non-volatile atomic through a volatile
reference or pointer causes undefined behavior. The standard doesn't
say that, and the casts are implicit, so this is icky.
2) Say that volatile atomic accesses may be implemented with more than
one instruction-level access.

(2) is something like how volatile reads of 128-bit structs involve
multiple mov instructions that execute in an arbitrary order. It's
also unlikely to cause problems in existing programs because nobody's
using volatile atomics yet, and they'll only start using them in ways
that work with what compilers implement.

Jeffrey