[Bug libstdc++/100438] New: `_serialize` macro clashes with user defined identifiers

[Bug libstdc++/100438] New: `_serialize` macro clashes with user defined identifiers

[heinzisoft at web dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100438

            Bug ID: 100438
           Summary: `_serialize` macro clashes with user defined
                    identifiers
           Product: gcc
           Version: 11.1.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libstdc++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: heinzisoft at web dot de
  Target Milestone: ---

[CryFS](https://github.com/cryfs/cryfs) compiled fine with GCC 10, but it
breaks with GCC 11 because the standard library is now defining a `_serialize`
macro that leaks into user code. When CryFS is trying to call a local
`_serialize` function, the macro is picked instead.

It looks like the macro got introduced here:
https://patchwork.ozlabs.org/project/gcc/patch/CAMe9rOp_DLg55kRxw2v75PPeqj-8tDKob5z-+EWPpf-L3OuKKw@mail.gmail.com/

Also, there's reports in other bug trackers about the same issue:
- https://github.com/cryfs/cryfs/issues/389
- https://bugs.gentoo.org/786459

[Bug libstdc++/100438] `_serialize` macro clashes with user defined identifiers

[heinzisoft at web dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100438

--- Comment #1 from heinzisoft at web dot de ---
- https://bugs.gentoo.org/788403

[Bug target/100438] `_serialize` macro clashes with user defined identifiers

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100438

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|libstdc++                   |target
     Ever confirmed|0                           |1
             Status|UNCONFIRMED                 |WAITING
                 CC|                            |hjl at gcc dot gnu.org,
                   |                            |jakub at gcc dot gnu.org
             Target|                            |x86_64-*-* i?86-*-*
   Last reconfirmed|                            |2021-05-06

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
I think you have to expect _serialize being a macro if you include intrinsic
headers.  But then you did not provide a testcase here - does the intrinsic
header leak in via a C++ standard include?

[Bug target/100438] `_serialize` macro clashes with user defined identifiers

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100438

--- Comment #3 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
The _serialize intrinsic is a standard Intel intrinsic:
https://software.intel.com/sites/landingpage/IntrinsicsGuide/#text=_serialize&expand=4902
and any intrinsic can be implemented as an inline function, macro or even
something different.
>From standard libstdc++ headers, I think <random> includes emmintrin.h which
doesn't include serializeintrin.h, but <experimental/simd> includes x86intrin.h
which does include it.  Or if your package includes x86intrin.h, immintrin.h or
serializeintrin.h, you get it of course too.

[Bug target/100438] `_serialize` macro clashes with user defined identifiers

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100438

--- Comment #4 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Richard Biener from comment #2)
> does the intrinsic
> header leak in via a C++ standard include?

No, not as far as I know. A testcase is definitely needed to demonstrate the
reported problem.

[Bug target/100438] `_serialize` macro clashes with user defined identifiers

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100438

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|---                         |INVALID

--- Comment #5 from Jonathan Wakely <redi at gcc dot gnu.org> ---
The gentoo bug has the actual error:

In file included from
/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/x86gprintrin.h:71,
                 from
/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/x86intrin.h:27,
                 from /usr/include/cryptopp/misc.h:81,
                 from /usr/include/cryptopp/simple.h:17,
                 from /usr/include/cryptopp/filters.h:17,
                 from /usr/include/cryptopp/basecode.h:10,
                 from /usr/include/cryptopp/hex.h:10,
                 from ../src/cpp-utils/data/FixedSizeData.h:5,
                 from ../src/blockstore/utils/IdWrapper.h:6,
                 from ../src/blockstore/utils/BlockId.h:5,
                 from ../src/parallelaccessstore/ParallelAccessBaseStore.h:7,
                 from ../src/parallelaccessstore/ParallelAccessStore.h:13,
                 from
../src/blobstore/implementations/onblocks/parallelaccessdatatreestore/ParallelAccessDataTreeStoreAdapter.h:6,
                 from
../src/blobstore/implementations/onblocks/parallelaccessdatatreestore/ParallelAccessDataTreeStoreAdapter.cpp:1:
/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/serializeintrin.h:37: note:
macro "_serialize" defined here
   37 | #define _serialize()    __builtin_ia32_serialize ()
      | 

There's not a single libstdc++ header in sight.

[Bug target/100438] `_serialize` macro clashes with user defined identifiers

[heinzisoft at web dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100438

--- Comment #6 from heinzisoft at web dot de ---
Seems the crypto++ library includes x86intrin.h, and that leaks the macro. I
assume a library isn't supposed to do that?

https://github.com/weidai11/cryptopp/blob/master/misc.h#L83

[Bug target/100438] `_serialize` macro clashes with user defined identifiers

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100438

--- Comment #7 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
That is a decision of each library what headers it includes and what
identifiers it reserves.  Just change your source...

[Bug target/100438] `_serialize` macro clashes with user defined identifiers

[filbranden at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100438

Filipe Brandenburger <filbranden at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |filbranden at gmail dot com

--- Comment #8 from Filipe Brandenburger <filbranden at gmail dot com> ---
We encountered an issue due to the introduction of the `_serialize()` define in
the build of the fbthrift package on GCC 11 (Fedora 34/35.)

The error we get is coming from Cython code in a *.pyx file, which gets
compiled to *.cpp and then compiled. Note that the name `_serialize` in this
context is coming from Python code conventions, where this particular name is
the most appropriate for its particular situation (the `_` indicating a
protected method.)

Also interesting is that the error we're seeing is actually in a method *call*
and not even a function definition!

Backtrace here:

```
/builddir/build/BUILD/fbthrift-2021.06.28.00/x86_64-redhat-linux-gnu/thrift/lib/py3/cybld/thrift/py3/serializer.cpp:3482:154:
error: macro "_serialize" passed 2 arguments, but takes just 0
 3482 |   __pyx_t_1 = ((PyObject *)((struct
__pyx_vtabstruct_6thrift_3py3_5types_Struct
*)__pyx_v_cy_struct->__pyx_vtab)->_serialize(__pyx_v_cy_struct, __pyx_t_5)); if
(unlikely(!__pyx_t_1)) __PYX_ERR(0, 30, __pyx_L1_error)
      |                                                                        
                                                                               
 ^
In file included from
/usr/lib/gcc/x86_64-redhat-linux/11/include/x86gprintrin.h:71,
                 from
/usr/lib/gcc/x86_64-redhat-linux/11/include/immintrin.h:27,
                 from /usr/include/folly/container/detail/F14Table.h:74,
                 from /usr/include/folly/container/detail/F14Policy.h:28,
                 from /usr/include/folly/container/F14Map.h:42,
                 from /usr/include/folly/io/async/Request.h:26,
                 from /usr/include/folly/io/async/AsyncTimeout.h:23,
                 from /usr/include/folly/io/async/HHWheelTimer.h:28,
                 from /usr/include/folly/fibers/Baton.h:24,
                 from /usr/include/folly/futures/Future.h:36,
                 from
/builddir/build/BUILD/fbthrift-2021.06.28.00/x86_64-redhat-linux-gnu/thrift/lib/py3/cybld/thrift/py3/serializer.cpp:645:
/usr/lib/gcc/x86_64-redhat-linux/11/include/serializeintrin.h:37: note: macro
"_serialize" defined here
   37 | #define _serialize()    __builtin_ia32_serialize ()
      |
```

This is coming from the F14Map.h header from folly, which defines a map
container type. That header file includes intrinsics in order to use a __m128i
type definition.

It's very unfortunate that intrinsics produce this kind of namespace pollution
and that this particular one was defined with a name that's not uncommon enough
not to cause real world problems.

At the very least, can this particular case be changed from a `#define` to a
`static inline void _serialize() { __builtin_ia32_serialize(); }` so that it
only clashes with a `_serialize()` name defined in the top-level?

For C++ code, would it be possible to have a C++ version of intrinsics headers
that would define them inside a C++ namespace, so that it would be easy to
introduce new intrinsics without further polluting the main namespace?

Thanks!
Filipe

[Bug target/100438] `_serialize` macro clashes with user defined identifiers

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100438

--- Comment #9 from H.J. Lu <hjl.tools at gmail dot com> ---
(In reply to Filipe Brandenburger from comment #8)
> We encountered an issue due to the introduction of the `_serialize()` define
> in the build of the fbthrift package on GCC 11 (Fedora 34/35.)
> 
> The error we get is coming from Cython code in a *.pyx file, which gets
> compiled to *.cpp and then compiled. Note that the name `_serialize` in this
> context is coming from Python code conventions, where this particular name
> is the most appropriate for its particular situation (the `_` indicating a
> protected method.)
> 
> Also interesting is that the error we're seeing is actually in a method
> *call* and not even a function definition!
> 
> Backtrace here:
> 
> ```
> /builddir/build/BUILD/fbthrift-2021.06.28.00/x86_64-redhat-linux-gnu/thrift/
> lib/py3/cybld/thrift/py3/serializer.cpp:3482:154: error: macro "_serialize"
> passed 2 arguments, but takes just 0
>  3482 |   __pyx_t_1 = ((PyObject *)((struct
> __pyx_vtabstruct_6thrift_3py3_5types_Struct
> *)__pyx_v_cy_struct->__pyx_vtab)->_serialize(__pyx_v_cy_struct, __pyx_t_5));
> if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 30, __pyx_L1_error)
>       |                                                                     
> ^
> In file included from
> /usr/lib/gcc/x86_64-redhat-linux/11/include/x86gprintrin.h:71,
>                  from
> /usr/lib/gcc/x86_64-redhat-linux/11/include/immintrin.h:27,
>                  from /usr/include/folly/container/detail/F14Table.h:74,
>                  from /usr/include/folly/container/detail/F14Policy.h:28,
>                  from /usr/include/folly/container/F14Map.h:42,
>                  from /usr/include/folly/io/async/Request.h:26,
>                  from /usr/include/folly/io/async/AsyncTimeout.h:23,
>                  from /usr/include/folly/io/async/HHWheelTimer.h:28,
>                  from /usr/include/folly/fibers/Baton.h:24,
>                  from /usr/include/folly/futures/Future.h:36,
>                  from
> /builddir/build/BUILD/fbthrift-2021.06.28.00/x86_64-redhat-linux-gnu/thrift/
> lib/py3/cybld/thrift/py3/serializer.cpp:645:
> /usr/lib/gcc/x86_64-redhat-linux/11/include/serializeintrin.h:37: note:
> macro "_serialize" defined here
>    37 | #define _serialize()    __builtin_ia32_serialize ()
>       |
> ```
> 
> This is coming from the F14Map.h header from folly, which defines a map
> container type. That header file includes intrinsics in order to use a
> __m128i type definition.
> 
> It's very unfortunate that intrinsics produce this kind of namespace
> pollution and that this particular one was defined with a name that's not
> uncommon enough not to cause real world problems.
> 
> At the very least, can this particular case be changed from a `#define` to a
> `static inline void _serialize() { __builtin_ia32_serialize(); }` so that it
> only clashes with a `_serialize()` name defined in the top-level?
> 
> For C++ code, would it be possible to have a C++ version of intrinsics
> headers that would define them inside a C++ namespace, so that it would be
> easy to introduce new intrinsics without further polluting the main
> namespace?

Please try:

https://gcc.gnu.org/pipermail/gcc-patches/2021-August/576603.html

on GCC master branch.  I can create a backport branch for GCC 11 if needed.

[Bug target/100438] `_serialize` macro clashes with user defined identifiers

[filbranden at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100438

--- Comment #10 from Filipe Brandenburger <filbranden at gmail dot com> ---
(In reply to H.J. Lu from comment #9)
> Please try:
> 
> https://gcc.gnu.org/pipermail/gcc-patches/2021-August/576603.html
> 
> on GCC master branch.  I can create a backport branch for GCC 11 if needed.

Thanks!

While I didn't manage to test that patch fully (by buliding a new gcc including
that change), I managed to locally apply the relevant snippet of the patch to
my local /usr/lib/gcc/x86_64-redhat-linux/11/include/serializeintrin.h and
confirmed that this did indeed fix the build issue I was seeing.

So indeed I consider this issue to be resolved for me with the adoption of that
patch.

[Bug target/100438] `_serialize` macro clashes with user defined identifiers

[filbranden at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100438

--- Comment #11 from Filipe Brandenburger <filbranden at gmail dot com> ---
BTW, I just checked and see that the commit is already pushed upstream:

https://github.com/gcc-mirror/gcc/commit/72264a639729a5dcc21dbee304717ce22b338bfd

Thanks!

[Bug target/100438] `_serialize` macro clashes with user defined identifiers

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100438

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|INVALID                     |DUPLICATE

--- Comment #12 from H.J. Lu <hjl.tools at gmail dot com> ---
I backported all relevant patches to GCC 11:

https://gitlab.com/x86-gcc/gcc/-/tree/users/hjl/x86gprintrin.h/gcc-11

I will wait a week before submitting them to GCC 11 branch.

*** This bug has been marked as a duplicate of bug 99744 ***