Re: [committed] libstdc++: Add TSan annotations to std::atomic<shared_ptr<T>>

[Jonathan Wakely <jwakely@redhat.com>]

On Wed, 14 Sept 2022 at 23:05, Jonathan Wakely via Libstdc++
<libstdc++@gcc.gnu.org> wrote:
>
> Tested powerpc64le-linux, pushed to trunk.
>
> -- >8 --
>
> This adds annotations to std::atomic<shared_ptr<T>> to enable TSan to
> understand the custom locking. Without this, TSan reports data races for
> accesses to the _M_ptr member, even though those are correctly
> synchronized using atomic operations on the tagged pointer.
>
> libstdc++-v3/ChangeLog:
>
>         * include/bits/shared_ptr_atomic.h (_GLIBCXX_TSAN_MUTEX_DESTROY)
>         (_GLIBCXX_TSAN_MUTEX_PRE_LOCK, _GLIBCXX_TSAN_MUTEX_POST_LOCK)
>         (_GLIBCXX_TSAN_MUTEX_PRE_UNLOCK, _GLIBCXX_TSAN_MUTEX_POST_UNLOCK)
>         (_GLIBCXX_TSAN_MUTEX_PRE_SIGNAL, _GLIBCXX_TSAN_MUTEX_POST_SIGNAL):
>         Define macros for TSan annotation functions.
>         (_Sp_atomic::_Atomic_count): Add annotations.
> ---
>  libstdc++-v3/include/bits/shared_ptr_atomic.h | 38 +++++++++++++++++++
>  1 file changed, 38 insertions(+)
>
> diff --git a/libstdc++-v3/include/bits/shared_ptr_atomic.h b/libstdc++-v3/include/bits/shared_ptr_atomic.h
> index d4bd712fc7d..4580807f42c 100644
> --- a/libstdc++-v3/include/bits/shared_ptr_atomic.h
> +++ b/libstdc++-v3/include/bits/shared_ptr_atomic.h
> @@ -32,6 +32,30 @@
>
>  #include <bits/atomic_base.h>
>
> +#if defined _GLIBCXX_TSAN && __has_include(<sanitizer/tsan_interface.h>)
> +#include <sanitizer/tsan_interface.h>
> +#define _GLIBCXX_TSAN_MUTEX_DESTROY(X) \
> +  __tsan_mutex_destroy(X, __tsan_mutex_not_static)
> +#define _GLIBCXX_TSAN_MUTEX_PRE_LOCK(X) \
> +  __tsan_mutex_pre_lock(X, __tsan_mutex_not_static)
> +#define _GLIBCXX_TSAN_MUTEX_POST_LOCK(X) \
> +  __tsan_mutex_post_lock(X, __tsan_mutex_not_static, 0)
> +#define _GLIBCXX_TSAN_MUTEX_PRE_UNLOCK(X) \
> +  __tsan_mutex_pre_unlock(X, __tsan_mutex_not_static)
> +#define _GLIBCXX_TSAN_MUTEX_POST_UNLOCK(X) \
> +  __tsan_mutex_post_unlock(X, __tsan_mutex_not_static)
> +#define _GLIBCXX_TSAN_MUTEX_PRE_SIGNAL(X) __tsan_mutex_pre_signal(X, 0)
> +#define _GLIBCXX_TSAN_MUTEX_POST_SIGNAL(X) __tsan_mutex_post_signal(X, 0)
> +#else
> +#define _GLIBCXX_TSAN_MUTEX_DESTROY(X)
> +#define _GLIBCXX_TSAN_MUTEX_PRE_LOCK(X)
> +#define _GLIBCXX_TSAN_MUTEX_POST_LOCK(X)
> +#define _GLIBCXX_TSAN_MUTEX_PRE_UNLOCK(X)
> +#define _GLIBCXX_TSAN_MUTEX_POST_UNLOCK(X)
> +#define _GLIBCXX_TSAN_MUTEX_PRE_SIGNAL(X)
> +#define _GLIBCXX_TSAN_MUTEX_POST_SIGNAL(X)
> +#endif
> +
>  namespace std _GLIBCXX_VISIBILITY(default)
>  {
>  _GLIBCXX_BEGIN_NAMESPACE_VERSION
> @@ -377,6 +401,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
>         ~_Atomic_count()
>         {
>           auto __val = _M_val.load(memory_order_relaxed);
> +         _GLIBCXX_TSAN_MUTEX_DESTROY(&_M_val);

After further thought, I'm not sure this is right. This tells tsan
that the "mutex" at &_M_val cannot be locked or unlocked again after
this. But what happens if the address is reused by a different
atomic<shared_ptr<T>> which happens to be at the same memory address?
Will tsan think that's an invalid use of the original "mutex" after
its destruction?

I will investigate.

We might need to stop using the __tsan_mutex_destroy call, and if so,
we can stop using the __tsan_mutex_not_static flag too. The pre/post
lock/unlock/signal pairs are still valuable without the lifetime
checking.