Re: [PATCH] libstdc++: Ensure active union member is correctly set

[Nathaniel Shead <nathanieloshead@gmail.com>]

On Wed, Sep 27, 2023 at 03:13:35PM +0100, Jonathan Wakely wrote:
> On Sat, 23 Sept 2023 at 08:30, Nathaniel Shead via Libstdc++
> <libstdc++@gcc.gnu.org> wrote:
> >
> > On Sat, Sep 23, 2023 at 07:40:48AM +0100, Jonathan Wakely wrote:
> > > On Sat, 23 Sept 2023, 01:39 Nathaniel Shead via Libstdc++, <
> > > libstdc++@gcc.gnu.org> wrote:
> > >
> > > > Now that bootstrap has finished, I have gotten regressions in the
> > > > following libstdc++ tests:
> > > >
> > > > Running libstdc++:libstdc++-dg/conformance.exp ...
> > > > FAIL: 20_util/bitset/access/constexpr.cc -std=gnu++23 (test for excess
> > > > errors)
> > > > FAIL: 20_util/bitset/access/constexpr.cc -std=gnu++26 (test for excess
> > > > errors)
> > > > FAIL: 20_util/variant/constexpr.cc -std=gnu++20 (test for excess errors)
> > > > FAIL: 20_util/variant/constexpr.cc -std=gnu++26 (test for excess errors)
> > > > FAIL: 21_strings/basic_string/cons/char/constexpr.cc -std=gnu++20 (test
> > > > for excess errors)
> > > > FAIL: 21_strings/basic_string/cons/char/constexpr.cc -std=gnu++26 (test
> > > > for excess errors)
> > > > FAIL: 21_strings/basic_string/cons/wchar_t/constexpr.cc -std=gnu++20 (test
> > > > for excess errors)
> > > > FAIL: 21_strings/basic_string/cons/wchar_t/constexpr.cc -std=gnu++26 (test
> > > > for excess errors)
> > > > FAIL: 21_strings/basic_string/modifiers/swap/constexpr-wchar_t.cc
> > > > -std=gnu++20 (test for excess errors)
> > > > FAIL: 21_strings/basic_string/modifiers/swap/constexpr-wchar_t.cc
> > > > -std=gnu++26 (test for excess errors)
> > > > FAIL: 21_strings/basic_string/modifiers/swap/constexpr.cc -std=gnu++20
> > > > (test for excess errors)
> > > > FAIL: 21_strings/basic_string/modifiers/swap/constexpr.cc -std=gnu++26
> > > > (test for excess errors)
> > > > FAIL: std/ranges/adaptors/join_with/1.cc -std=gnu++23 (test for excess
> > > > errors)
> > > > UNRESOLVED: std/ranges/adaptors/join_with/1.cc -std=gnu++23 compilation
> > > > failed to produce executable
> > > > FAIL: std/ranges/adaptors/join_with/1.cc -std=gnu++26 (test for excess
> > > > errors)
> > > > UNRESOLVED: std/ranges/adaptors/join_with/1.cc -std=gnu++26 compilation
> > > > failed to produce executable
> > > >
> > > > On investigation though it looks like the issue might be with libstdc++
> > > > rather than the patch itself; running the failing tests using clang with
> > > > libstdc++ also produces similar errors, and my reading of the code
> > > > suggests that this is correct.
> > > >
> > > > What's the way forward here? Should I look at creating a patch to fix
> > > > the libstdc++ issues before resubmitting this patch for the C++
> > > > frontend? Or should I submit a version of this patch without the
> > > > `std::construct_at` changes and wait till libstdc++ gets fixed for that?
> > > >
> > >
> > > I think we should fix libstdc++. There are probably only a few places that
> > > need a fix, which cause all those failures.
> > >
> > > I can help with those fixes. I'll look into it after the weekend.
> > >
> >
> > Thanks. I did end up getting a chance to look at it earlier today, and
> > with the following patch I had no regressions when applying the frontend
> > changes. Bootstrapped and regtested on x86_64-pc-linux-gnu.
> >
> > -- >8 --
> >
> > This patch ensures that the union members for std::string and
> > std::variant are always properly set when a change occurs.
> >
> > libstdc++-v3/ChangeLog:
> >
> >         * include/bits/basic_string.h: (basic_string(basic_string&&)):
> >         Activate _M_local_buf when needed.
> >         (basic_string(basic_string&&, const _Alloc&)): Likewise.
> >         * include/bits/basic_string.tcc: (basic_string::swap): Likewise.
> >         * include/std/variant: (__detail::__variant::__construct_n): New.
> >         (__detail::_variant::__emplace): Use __construct_n.
> >
> > Signed-off-by: Nathaniel Shead <nathanieloshead@gmail.com>
> > ---
> >  libstdc++-v3/include/bits/basic_string.h   |  7 +++--
> >  libstdc++-v3/include/bits/basic_string.tcc |  8 +++---
> >  libstdc++-v3/include/std/variant           | 32 ++++++++++++++++++++--
> >  3 files changed, 38 insertions(+), 9 deletions(-)
> >
> > diff --git a/libstdc++-v3/include/bits/basic_string.h b/libstdc++-v3/include/bits/basic_string.h
> > index 09fd62afa66..7c342879827 100644
> > --- a/libstdc++-v3/include/bits/basic_string.h
> > +++ b/libstdc++-v3/include/bits/basic_string.h
> > @@ -678,7 +678,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
> >        {
> >         if (__str._M_is_local())
> >           {
> > -           traits_type::copy(_M_local_buf, __str._M_local_buf,
> > +           traits_type::copy(_M_use_local_data(), __str._M_local_buf,
> >                               __str.length() + 1);
> >           }
> >         else
> > @@ -691,7 +691,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
> >         // basic_stringbuf relies on writing into unallocated capacity so
> >         // we mess up the contents if we put a '\0' in the string.
> >         _M_length(__str.length());
> > -       __str._M_data(__str._M_local_data());
> > +       __str._M_data(__str._M_use_local_data());
> >         __str._M_set_length(0);
> >        }
> >
> > @@ -717,6 +717,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
> >        {
> >         if (__str._M_is_local())
> >           {
> > +           _M_use_local_data();
> 
> Lets add a cast to void to make it clear we're intentionally
> discarding the return value here, and only calling it for its
> constexpr "side effects":
> 
>     (void) _M_use_local_data();
> 
> 
> >             traits_type::copy(_M_local_buf, __str._M_local_buf,
> >                               __str.length() + 1);
> >             _M_length(__str.length());
> > @@ -728,7 +729,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
> >             _M_data(__str._M_data());
> >             _M_length(__str.length());
> >             _M_capacity(__str._M_allocated_capacity);
> > -           __str._M_data(__str._M_local_buf);
> > +           __str._M_data(__str._M_use_local_data());
> >             __str._M_set_length(0);
> >           }
> >         else
> > diff --git a/libstdc++-v3/include/bits/basic_string.tcc b/libstdc++-v3/include/bits/basic_string.tcc
> > index 104a517f794..ee6e57da555 100644
> > --- a/libstdc++-v3/include/bits/basic_string.tcc
> > +++ b/libstdc++-v3/include/bits/basic_string.tcc
> > @@ -79,7 +79,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
> >               }
> >             else if (__s.length())
> >               {
> > -               traits_type::copy(_M_local_buf, __s._M_local_buf,
> > +               traits_type::copy(_M_use_local_data(), __s._M_local_buf,
> 
> I think we should call _M_use_local_data() before calling
> traits_type::copy, as you did for
> basic_string(basic_string&& __str, const _Alloc& __a)
> above.
> 
> The problem is that _M_use_local_data() returns the allocator's
> pointer type, but traits_type::copy expects char_type*, and those
> might not be the same type.
> So:
> 
> @@ -79,6 +79,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
>              }
>            else if (__s.length())
>              {
> +               (void) _M_use_local_data();
>                traits_type::copy(_M_local_buf, __s._M_local_buf,
>                                  __s.length() + 1);
>                _M_length(__s.length());
> 
> 
> 
> >                                   __s.length() + 1);
> >                 _M_length(__s.length());
> >                 __s._M_set_length(0);
> > @@ -87,7 +87,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
> >               }
> >             else if (length())
> >               {
> > -               traits_type::copy(__s._M_local_buf, _M_local_buf,
> > +               traits_type::copy(__s._M_use_local_data(), _M_local_buf,
> 
> Same here, for __s._M_use_local_data()
> 
> >                                   length() + 1);
> >                 __s._M_length(length());
> >                 _M_set_length(0);
> > @@ -97,7 +97,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
> >         else
> >           {
> >             const size_type __tmp_capacity = __s._M_allocated_capacity;
> > -           traits_type::copy(__s._M_local_buf, _M_local_buf,
> > +           traits_type::copy(__s._M_use_local_data(), _M_local_buf,
> 
> And again.
> 
> >                               length() + 1);
> >             _M_data(__s._M_data());
> >             __s._M_data(__s._M_local_buf);
> > @@ -108,7 +108,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
> >           const size_type __tmp_capacity = _M_allocated_capacity;
> >           if (__s._M_is_local())
> >             {
> > -             traits_type::copy(_M_local_buf, __s._M_local_buf,
> > +             traits_type::copy(_M_use_local_data(), __s._M_local_buf,
> 
> And again.
> 
> >                                 __s.length() + 1);
> >               __s._M_data(_M_data());
> >               _M_data(_M_local_buf);
> > diff --git a/libstdc++-v3/include/std/variant b/libstdc++-v3/include/std/variant
> > index c0e41740dcf..7f24e760bb1 100644
> > --- a/libstdc++-v3/include/std/variant
> > +++ b/libstdc++-v3/include/std/variant
> > @@ -320,6 +320,33 @@ namespace __variant
> >      __get(_Variant&& __v) noexcept
> >      { return __variant::__get_n<_Np>(std::forward<_Variant>(__v)._M_u); }
> >
> > +  // Gets the _Uninitialized to construct into for __u.
> > +  template<size_t _Np, typename _Union>
> > +    constexpr decltype(auto)
> > +    __construct_n(_Union& __u) noexcept
> > +    {
> > +      if constexpr (_Np == 0)
> > +       return &__u._M_first;
> > +      else if constexpr (_Np == 1)
> > +       {
> > +         std::_Construct(&__u._M_rest);
> > +         return &__u._M_rest._M_first;
> > +       }
> > +      else if constexpr (_Np == 2)
> > +       {
> > +         std::_Construct(&__u._M_rest);
> > +         std::_Construct(&__u._M_rest._M_rest);
> > +         return &__u._M_rest._M_rest._M_first;
> > +       }
> > +      else
> > +       {
> > +         std::_Construct(&__u._M_rest);
> > +         std::_Construct(&__u._M_rest._M_rest);
> > +         std::_Construct(&__u._M_rest._M_rest._M_rest);
> > +         return __variant::__construct_n<_Np - 3>(__u._M_rest._M_rest._M_rest);
> > +       }
> > +    }
> 
> This is nice, thanks for optimizing it to reduce the recursion depth.
> 
> > +
> >    template<typename... _Types>
> >      struct _Traits
> >      {
> > @@ -536,8 +563,9 @@ namespace __variant
> >      __emplace(_Variant_storage<_Triv, _Types...>& __v, _Args&&... __args)
> >      {
> >        __v._M_reset();
> > -      auto* __addr = std::__addressof(__variant::__get_n<_Np>(__v._M_u));
> > -      std::_Construct(__addr, std::forward<_Args>(__args)...);
> > +      auto* __addr = __variant::__construct_n<_Np>(__v._M_u);
> > +      std::_Construct(__addr, in_place_index<0>,
> > +                     std::forward<_Args>(__args)...);
> >        // Construction didn't throw, so can set the new index now:
> >        __v._M_index = _Np;
> >      }
> > --
> > 2.41.0
> >
> 

Thanks for the comments, here's an updated version of the patch.
Bootstrapped and regtested on x86_64-pc-linux-gnu.

I'll note that there are some existing calls to `_M_use_local_data()`
already used only for their side effects without a cast to void, e.g.

      /**
       *  @brief  Default constructor creates an empty string.
       */
      _GLIBCXX20_CONSTEXPR
      basic_string()
      _GLIBCXX_NOEXCEPT_IF(is_nothrow_default_constructible<_Alloc>::value)
      : _M_dataplus(_M_local_data())
      {
	_M_use_local_data();
	_M_set_length(0);
      }

I haven't updated these, but should this be changed for consistency?

-- >8 --

This patch ensures that the union members for std::string and
std::variant are always properly set when a change occurs.

libstdc++-v3/ChangeLog:

	* include/bits/basic_string.h: (basic_string(basic_string&&)):
	Activate _M_local_buf when needed.
	(basic_string(basic_string&&, const _Alloc&)): Likewise.
	* include/bits/basic_string.tcc: (basic_string::swap): Likewise.
	* include/std/variant: (__detail::__variant::__construct_n): New.
	(__detail::__variant::__emplace): Use __construct_n.

Signed-off-by: Nathaniel Shead <nathanieloshead@gmail.com>
---
 libstdc++-v3/include/bits/basic_string.h   |  6 ++--
 libstdc++-v3/include/bits/basic_string.tcc |  4 +++
 libstdc++-v3/include/std/variant           | 32 ++++++++++++++++++++--
 3 files changed, 38 insertions(+), 4 deletions(-)

diff --git a/libstdc++-v3/include/bits/basic_string.h b/libstdc++-v3/include/bits/basic_string.h
index 09fd62afa66..4f94cd967cf 100644
--- a/libstdc++-v3/include/bits/basic_string.h
+++ b/libstdc++-v3/include/bits/basic_string.h
@@ -678,6 +678,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
       {
 	if (__str._M_is_local())
 	  {
+	    (void)_M_use_local_data();
 	    traits_type::copy(_M_local_buf, __str._M_local_buf,
 			      __str.length() + 1);
 	  }
@@ -691,7 +692,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
 	// basic_stringbuf relies on writing into unallocated capacity so
 	// we mess up the contents if we put a '\0' in the string.
 	_M_length(__str.length());
-	__str._M_data(__str._M_local_data());
+	__str._M_data(__str._M_use_local_data());
 	__str._M_set_length(0);
       }
 
@@ -717,6 +718,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
       {
 	if (__str._M_is_local())
 	  {
+	    (void)_M_use_local_data();
 	    traits_type::copy(_M_local_buf, __str._M_local_buf,
 			      __str.length() + 1);
 	    _M_length(__str.length());
@@ -728,7 +730,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
 	    _M_data(__str._M_data());
 	    _M_length(__str.length());
 	    _M_capacity(__str._M_allocated_capacity);
-	    __str._M_data(__str._M_local_buf);
+	    __str._M_data(__str._M_use_local_data());
 	    __str._M_set_length(0);
 	  }
 	else
diff --git a/libstdc++-v3/include/bits/basic_string.tcc b/libstdc++-v3/include/bits/basic_string.tcc
index 104a517f794..4bc98f2aea7 100644
--- a/libstdc++-v3/include/bits/basic_string.tcc
+++ b/libstdc++-v3/include/bits/basic_string.tcc
@@ -79,6 +79,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
 	      }
 	    else if (__s.length())
 	      {
+		(void)_M_use_local_data();
 		traits_type::copy(_M_local_buf, __s._M_local_buf,
 				  __s.length() + 1);
 		_M_length(__s.length());
@@ -87,6 +88,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
 	      }
 	    else if (length())
 	      {
+		(void)__s._M_use_local_data();
 		traits_type::copy(__s._M_local_buf, _M_local_buf,
 				  length() + 1);
 		__s._M_length(length());
@@ -97,6 +99,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
 	else
 	  {
 	    const size_type __tmp_capacity = __s._M_allocated_capacity;
+	    (void)__s._M_use_local_data();
 	    traits_type::copy(__s._M_local_buf, _M_local_buf,
 			      length() + 1);
 	    _M_data(__s._M_data());
@@ -108,6 +111,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
 	  const size_type __tmp_capacity = _M_allocated_capacity;
 	  if (__s._M_is_local())
 	    {
+	      (void)_M_use_local_data();
 	      traits_type::copy(_M_local_buf, __s._M_local_buf,
 				__s.length() + 1);
 	      __s._M_data(_M_data());
diff --git a/libstdc++-v3/include/std/variant b/libstdc++-v3/include/std/variant
index c0e41740dcf..7f24e760bb1 100644
--- a/libstdc++-v3/include/std/variant
+++ b/libstdc++-v3/include/std/variant
@@ -320,6 +320,33 @@ namespace __variant
     __get(_Variant&& __v) noexcept
     { return __variant::__get_n<_Np>(std::forward<_Variant>(__v)._M_u); }
 
+  // Gets the _Uninitialized to construct into for __u.
+  template<size_t _Np, typename _Union>
+    constexpr decltype(auto)
+    __construct_n(_Union& __u) noexcept
+    {
+      if constexpr (_Np == 0)
+	return &__u._M_first;
+      else if constexpr (_Np == 1)
+	{
+	  std::_Construct(&__u._M_rest);
+	  return &__u._M_rest._M_first;
+	}
+      else if constexpr (_Np == 2)
+	{
+	  std::_Construct(&__u._M_rest);
+	  std::_Construct(&__u._M_rest._M_rest);
+	  return &__u._M_rest._M_rest._M_first;
+	}
+      else
+	{
+	  std::_Construct(&__u._M_rest);
+	  std::_Construct(&__u._M_rest._M_rest);
+	  std::_Construct(&__u._M_rest._M_rest._M_rest);
+	  return __variant::__construct_n<_Np - 3>(__u._M_rest._M_rest._M_rest);
+	}
+    }
+
   template<typename... _Types>
     struct _Traits
     {
@@ -536,8 +563,9 @@ namespace __variant
     __emplace(_Variant_storage<_Triv, _Types...>& __v, _Args&&... __args)
     {
       __v._M_reset();
-      auto* __addr = std::__addressof(__variant::__get_n<_Np>(__v._M_u));
-      std::_Construct(__addr, std::forward<_Args>(__args)...);
+      auto* __addr = __variant::__construct_n<_Np>(__v._M_u);
+      std::_Construct(__addr, in_place_index<0>,
+		      std::forward<_Args>(__args)...);
       // Construction didn't throw, so can set the new index now:
       __v._M_index = _Np;
     }
-- 
2.41.0