[Bug libstdc++/100164] New: [11 Regression] semaphore_impl not declaredon AIX

[Bug libstdc++/100164] New: [11 Regression] semaphore_impl not declaredon AIX

[dje at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

            Bug ID: 100164
           Summary: [11 Regression] semaphore_impl not declaredon AIX
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: blocker
          Priority: P3
         Component: libstdc++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: dje at gcc dot gnu.org
  Target Milestone: ---
            Target: powerpc-ibm-aix*

The last-minute change to libstdc++ at the time of the GCC 11 branch has caused
severe regressions on AIX.

All of the testsuite regressions are of the form:

/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/bits/semaphore_base.h:259: 
error: #error "No suitable semaphore implementation available"
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:43: error:
'__semaphore_impl' has not been declared
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:47: error:
'__semaphore_impl' has not been declared
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:49: error:
'__semaphore_impl' does not name a type
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:66: error:
'_M_sem' was not declared in this scope
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:70: error:
'_M_sem' was not declared in this scope
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:74: error:
'_M_sem' was not declared in this scope
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:53: error: class 
'std::counting_semaphore<__least_max_value>' does not have any field named
'_M_sem'
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:67: error:
'_M_sem' was not declared in this scope
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:71: error:
'_M_sem' was not declared in this scope
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:75: error:
'_M_sem' was not declared in this scope
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:80: error:
'_M_sem' was not declared in this scope
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:85: error:
'_M_sem' was not declared in this scope

[Bug libstdc++/100164] [11 Regression] semaphore_impl not declaredon AIX

[dje at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

David Edelsohn <dje at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever confirmed|0                           |1
           Priority|P3                          |P1
                 CC|                            |jakub at gcc dot gnu.org,
                   |                            |redi at gcc dot gnu.org,
                   |                            |rguenth at gcc dot gnu.org,
                   |                            |rodgertq at gcc dot gnu.org
            Version|unknown                     |11.0
   Last reconfirmed|                            |2021-04-20

--- Comment #1 from David Edelsohn <dje at gcc dot gnu.org> ---
Confirmed

[Bug libstdc++/100164] [11 Regression] semaphore_impl not declared on AIX

[rodgertq at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

Thomas Rodgers <rodgertq at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |rodgertq at gcc dot gnu.org

--- Comment #2 from Thomas Rodgers <rodgertq at gcc dot gnu.org> ---
Investigating

[Bug libstdc++/100164] [11 Regression] semaphore_impl not declared on AIX

[mpolacek at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

Marek Polacek <mpolacek at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mpolacek at gcc dot gnu.org
             Status|NEW                         |ASSIGNED

[Bug libstdc++/100164] [11 Regression] semaphore_impl not declared on AIX

[rodgertq at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

--- Comment #3 from Thomas Rodgers <rodgertq at gcc dot gnu.org> ---
Created attachment 50643
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=50643&action=edit
Disable <semaphore> on unsupported targets

This patch is probably not the most elegant way to do this, it probably should
be a dg-effective-target check and it disables <semaphore> on platforms where
there is a suitable Posix implementation, but this feature is experimental for
GCC11.

[Bug libstdc++/100164] [11 Regression] semaphore_impl not declared on AIX

[rodgertq at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

--- Comment #4 from Thomas Rodgers <rodgertq at gcc dot gnu.org> ---
It would appear that I cannot log into either of the AIX machines in the
compile farm.

[Bug libstdc++/100164] [11 Regression] semaphore_impl not declared on AIX

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

--- Comment #5 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Comment on attachment 50643
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=50643
Disable <semaphore> on unsupported targets

+#else
+      __gthread_yield();

The changelog said you want to use __thread_yield(); there instead...

[Bug libstdc++/100164] [11 Regression] semaphore_impl not declared on AIX

[dje at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

--- Comment #6 from David Edelsohn <dje at gcc dot gnu.org> ---
Created attachment 50644
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=50644&action=edit
pre-processed source for 17_intro/headers/c++2020/all_attributes.cc

[Bug libstdc++/100164] [11 Regression] semaphore_impl not declared on AIX

[rodgertq at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

--- Comment #7 from Thomas Rodgers <rodgertq at gcc dot gnu.org> ---
Created attachment 50645
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=50645&action=edit
Disable <semaphore> on unsupported targets

Let's try this with the right patch attached this time

[Bug libstdc++/100164] [11 Regression] semaphore_impl not declared on AIX

[dje at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

--- Comment #8 from David Edelsohn <dje at gcc dot gnu.org> ---
I am not certain why you cannot log in to the compile farm system.
I am testing the patch on one of the AIX systems inside IBM.

[Bug libstdc++/100164] [11 Regression] semaphore_impl not declared on AIX

[dje at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

--- Comment #9 from David Edelsohn <dje at gcc dot gnu.org> ---
The previous semaphore_base.h implementation had a fallback that hid a bug in
the macros:

#if defined _GLIBCXX_HAVE_LINUX_FUTEX && !_GLIBCXX_REQUIRE_POSIX_SEMAPHORE
  // Use futex if available and didn't force use of POSIX
  using __fast_semaphore = __atomic_semaphore<__detail::__platform_wait_t>;
#elif _GLIBCXX_HAVE_POSIX_SEMAPHORE
  using __fast_semaphore = __platform_semaphore;
#else
  using __fast_semaphore = __atomic_semaphore<ptrdiff_t>;
#endif

The problem is that libstdc++ configure defines
_GLIBCXX_HAVE_POSIX_SEMAPHORE in config.h.  libstdc++ uses sed to
rewrite config.h to c++config.h and prepends _GLIBCXX_, so c++config.h
contains

#define _GLIBCXX__GLIBCXX_HAVE_POSIX_SEMAPHORE 1

And bits/semaphore_base.h is not testing that corrupted macro.  Either
semaphore_base.h needs to test for the corrupted macro, or libtsdc++
configure needs to define HAVE_POSIX_SEMAPHORE without itself
prepending _GLIBCXX_  so that the c++config.h rewriting works
correctly and defines the correct macro for semaphore_base.h.

[Bug libstdc++/100164] [11 Regression] semaphore_impl not declared on AIX

[rodgertq at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

Thomas Rodgers <rodgertq at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #50643|0                           |1
        is obsolete|                            |
  Attachment #50645|0                           |1
        is obsolete|                            |

--- Comment #10 from Thomas Rodgers <rodgertq at gcc dot gnu.org> ---
Created attachment 50646
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=50646&action=edit
Work around broken macro name

This patch works around the borked macro name that David pointed out on the
mailing list. I left in the commented out _GLIBCXX_HAVE_POSIX_SEMAPHORE checks
and have temporarily replaced them with _GLIBCXX__GLIBCXX_HAVE_POSIX_SEMAPHORE,
and forced the posix semaphore test to always run if that macro is defined.

I am not sufficiently versed in the arcane ways in which config.h is
transformed to c++config.h but the borked macro transformation ideally should
be fixed and the commented out checks restored.

[Bug libstdc++/100164] [11/12 Regression] semaphore_impl not declared on AIX

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[11 Regression]             |[11/12 Regression]
                   |semaphore_impl not declared |semaphore_impl not declared
                   |on AIX                      |on AIX
   Target Milestone|---                         |11.0

[Bug libstdc++/100164] [11/12 Regression] semaphore_impl not declared on AIX

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

--- Comment #11 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jakub Jelinek <jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:2d4c3af94f84c874cfddc753dc0f34ebf7fc11d9

commit r12-22-g2d4c3af94f84c874cfddc753dc0f34ebf7fc11d9
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Wed Apr 21 11:09:25 2021 +0200

    Fix AIX libstdc++ semaphore support [PR100164]

    > > The #error would not be hit if _GLIBCXX_HAVE_POSIX_SEMAPHORE were
defined,
    > > but it shows up in your error report.

    > You now have pinpointed the problem.

    > It's not that AIX doesn't have semaphore, but that the code previously
    > had a fallback that hid a bug in the macros:

      // Use futex if available and didn't force use of POSIX
      using __fast_semaphore = __atomic_semaphore<__detail::__platform_wait_t>;
      using __fast_semaphore = __platform_semaphore;
      using __fast_semaphore = __atomic_semaphore<ptrdiff_t>;

    > The problem is that libstdc++ configure defines
    > _GLIBCXX_HAVE_POSIX_SEMAPHORE in config.h.  libstdc++ uses sed to
    > rewrite config.h to c++config.h and prepends _GLIBCXX_, so c++config.h
    > contains

    > And bits/semaphore_base.h is not testing that corrupted macro.  Either
    > semaphore_base.h needs to test for the corrupted macro, or libtsdc++
    > configure needs to define HAVE_POSIX_SEMAPHORE without itself
    > prepending _GLIBCXX_  so that the c++config.h rewriting works
    > correctly and defines the correct macro for semaphore_base.h.

    The include/Makefile.am sed is:
            sed -e 's/HAVE_/_GLIBCXX_HAVE_/g' \
                -e 's/PACKAGE/_GLIBCXX_PACKAGE/g' \
                -e 's/VERSION/_GLIBCXX_VERSION/g' \
                -e 's/WORDS_/_GLIBCXX_WORDS_/g' \
                -e
's/_DARWIN_USE_64_BIT_INODE/_GLIBCXX_DARWIN_USE_64_BIT_INODE/g' \
                -e 's/_FILE_OFFSET_BITS/_GLIBCXX_FILE_OFFSET_BITS/g' \
                -e 's/_LARGE_FILES/_GLIBCXX_LARGE_FILES/g' \
                -e 's/ICONV_CONST/_GLIBCXX_ICONV_CONST/g' \
                -e '/[       ]_GLIBCXX_LONG_DOUBLE_COMPAT[   ]/d' \
                -e '/[       ]_GLIBCXX_LONG_DOUBLE_ALT128_COMPAT[    ]/d' \
                < ${CONFIG_HEADER} >> $@ ;\
    so for many macros one needs _GLIBCXX_ prefixes already in configure,
    as can be seen in grep AC_DEFINE.*_GLIBCXX configure.ac acinclude.m4
    But _GLIBCXX_HAVE_POSIX_SEMAPHORE is the only one that shouldn't have
    that prefix because the sed is adding that.
    E.g. on i686-linux, I see
    grep _GLIBCXX__GLIBCXX c++config.h
    that proves it is the only broken one.

    So this change fixes the acinclude.m4 side.

    2021-04-21  Jakub Jelinek  <jakub@redhat.com>

            PR libstdc++/100164
            * acinclude.m4: For POSIX semaphores AC_DEFINE HAVE_POSIX_SEMAPHORE
            rather than _GLIBCXX_HAVE_POSIX_SEMAPHORE.
            * configure: Regenerated.
            * config.h.in: Regenerated.

[Bug libstdc++/100164] [11/12 Regression] semaphore_impl not declared on AIX

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

--- Comment #12 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #5)
> The changelog said you want to use __thread_yield(); there instead...

Yes, that was fixed in r11-7293 (the patch attached in comment 3 was r11-7248).

[Bug libstdc++/100164] [11/12 Regression] semaphore_impl not declared on AIX

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

--- Comment #13 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Jakub Jelinek
<jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:0857fc3820bee04771f60a86b9dd5627b3ea1904

commit r11-8270-g0857fc3820bee04771f60a86b9dd5627b3ea1904
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Wed Apr 21 11:09:25 2021 +0200

    Fix AIX libstdc++ semaphore support [PR100164]

    > > The #error would not be hit if _GLIBCXX_HAVE_POSIX_SEMAPHORE were
defined,
    > > but it shows up in your error report.

    > You now have pinpointed the problem.

    > It's not that AIX doesn't have semaphore, but that the code previously
    > had a fallback that hid a bug in the macros:

      // Use futex if available and didn't force use of POSIX
      using __fast_semaphore = __atomic_semaphore<__detail::__platform_wait_t>;
      using __fast_semaphore = __platform_semaphore;
      using __fast_semaphore = __atomic_semaphore<ptrdiff_t>;

    > The problem is that libstdc++ configure defines
    > _GLIBCXX_HAVE_POSIX_SEMAPHORE in config.h.  libstdc++ uses sed to
    > rewrite config.h to c++config.h and prepends _GLIBCXX_, so c++config.h
    > contains

    > And bits/semaphore_base.h is not testing that corrupted macro.  Either
    > semaphore_base.h needs to test for the corrupted macro, or libtsdc++
    > configure needs to define HAVE_POSIX_SEMAPHORE without itself
    > prepending _GLIBCXX_  so that the c++config.h rewriting works
    > correctly and defines the correct macro for semaphore_base.h.

    The include/Makefile.am sed is:
            sed -e 's/HAVE_/_GLIBCXX_HAVE_/g' \
                -e 's/PACKAGE/_GLIBCXX_PACKAGE/g' \
                -e 's/VERSION/_GLIBCXX_VERSION/g' \
                -e 's/WORDS_/_GLIBCXX_WORDS_/g' \
                -e
's/_DARWIN_USE_64_BIT_INODE/_GLIBCXX_DARWIN_USE_64_BIT_INODE/g' \
                -e 's/_FILE_OFFSET_BITS/_GLIBCXX_FILE_OFFSET_BITS/g' \
                -e 's/_LARGE_FILES/_GLIBCXX_LARGE_FILES/g' \
                -e 's/ICONV_CONST/_GLIBCXX_ICONV_CONST/g' \
                -e '/[       ]_GLIBCXX_LONG_DOUBLE_COMPAT[   ]/d' \
                -e '/[       ]_GLIBCXX_LONG_DOUBLE_ALT128_COMPAT[    ]/d' \
                < ${CONFIG_HEADER} >> $@ ;\
    so for many macros one needs _GLIBCXX_ prefixes already in configure,
    as can be seen in grep AC_DEFINE.*_GLIBCXX configure.ac acinclude.m4
    But _GLIBCXX_HAVE_POSIX_SEMAPHORE is the only one that shouldn't have
    that prefix because the sed is adding that.
    E.g. on i686-linux, I see
    grep _GLIBCXX__GLIBCXX c++config.h
    that proves it is the only broken one.

    So this change fixes the acinclude.m4 side.

    2021-04-21  Jakub Jelinek  <jakub@redhat.com>

            PR libstdc++/100164
            * acinclude.m4: For POSIX semaphores AC_DEFINE HAVE_POSIX_SEMAPHORE
            rather than _GLIBCXX_HAVE_POSIX_SEMAPHORE.
            * configure: Regenerated.
            * config.h.in: Regenerated.

    (cherry picked from commit 2d4c3af94f84c874cfddc753dc0f34ebf7fc11d9)

[Bug libstdc++/100164] [11/12 Regression] semaphore_impl not declared on AIX

[dje at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

--- Comment #14 from David Edelsohn <dje at gcc dot gnu.org> ---
Thanks for the patch. This now removes the failure that semaphore_impl is not
found.

Because the platform semaphore code never was exercised due to the bug in the
macro, there are more latent bugs.

/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:74: error:
'using
 __semaphore_impl = struct std::__platform_semaphore' has no member named
'_M_tr
y_acquire'; did you mean '_M_acquire'?
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:75: error:
'using
 __semaphore_impl = struct std::__platform_semaphore' has no member named
'_M_tr
y_acquire'; did you mean '_M_acquire'?

[Bug libstdc++/100164] [11/12 Regression] semaphore_impl not declared on AIX

[rodgertq at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

--- Comment #15 from Thomas Rodgers <rodgertq at gcc dot gnu.org> ---
>From the most recent patch -

+    _GLIBCXX_ALWAYS_INLINE bool
+    _M_try_acquire() noexcept
+    {
+      for (;;)
+       {
+         auto __err = sem_trywait(&_M_semaphore);
+         if (__err && (errno == EINTR))
+           continue;
+         else if (__err && (errno == EAGAIN))
+           return false;
+         else if (__err)
+           std::terminate();
+         else
+           break;
+       }
+      return true;
+    }
+

You are correct it was never exercised. I saw then when I forced it in the test
case and then added the above. I don't understand why you are seeing this error
if you've applied the 0001-libstdc-Work-around-for-PR100164 patch. I don't see
it locally.

[Bug libstdc++/100164] [11/12 Regression] semaphore_impl not declared on AIX

[rodgertq at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

--- Comment #16 from Thomas Rodgers <rodgertq at gcc dot gnu.org> ---
The _M_try_acquire() change should be on master and gcc-11 now.

[Bug libstdc++/100164] [11/12 Regression] semaphore_impl not declared on AIX

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #17 from Jonathan Wakely <redi at gcc dot gnu.org> ---
r12-44 and r11-8275

[Bug libstdc++/100164] [11/12 Regression] semaphore_impl not declared on AIX

[dje at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100164

--- Comment #18 from David Edelsohn <dje at gcc dot gnu.org> ---
Because the _M_try_acquire patch was not yet committed to trunk when I tested
and bootstrap on trunk now is broken due to other, unrelated patches.