[Bug libstdc++/108030] New: `std::experimental::simd` not inlined

[Bug libstdc++/108030] New: `std::experimental::simd` not inlined

[bernhardmgruber at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

            Bug ID: 108030
           Summary: `std::experimental::simd` not inlined
           Product: gcc
           Version: 12.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libstdc++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: bernhardmgruber at gmail dot com
  Target Milestone: ---

Created attachment 54052
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=54052&action=edit
Diff we applied to a local copy of the <experimental/simd>headers.

We tried to explicitely vectorize a C++ function using
`std::experimental::simd` in our particle-in-cell simulation
[picongpu](https://github.com/ComputationalRadiationPhysics/picongpu). The
function is already called from a long call tree of functions marked
`__attribute__((always_inline))`. Profiling the code shows that several
constructs of `std::experimental::simd` where not inlined, leading to
catastrophic performance (several times slower than scalar code).

We compiled, among other flags, with:
```
-g
-march=native
-mtune=native
-fopenmp
-O3
-DNDEBUG
-pthread
-std=c++17
```

We mostly used multiplication/addition as well as the broadcast and generator
constructors of SIMD types. We saw several calls to `_S_multiplies` (IIRC) and
`_S_generate`/`_S_generator` that were not inlined, depending on whether we
used `std::experimental::native_simd` or `std::experimental::fixed_size_simd`.

Upon inspection of the `<experimental/bits/simd_*>` headers, we saw that
several functions are not annotated with `_GLIBCXX_SIMD_INTRINSIC` or other
ways to force inlining. We think this is a missed optimization opportunity.

We tried `-finline-limit=1000000` without success.

We thus applied `_GLIBCXX_SIMD_INTRINSIC` and `__attribute__((always_inline))`
to functions from the SIMD headers that showed up in the profiler (perf) until
all calls were inlined.

Please apply further attributes to SIMD intrinsics to force their inlining.
Mind, that this also affects lambda expressions.

I have attached a diff which our changes to the SIMD headers, but we also bulk
replaced several declaration specifiers, so we may have added more
force-inlines than potentially necessary.

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[mkretz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

Matthias Kretz (Vir) <mkretz at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
                 CC|                            |mkretz at gcc dot gnu.org
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2022-12-09

--- Comment #1 from Matthias Kretz (Vir) <mkretz at gcc dot gnu.org> ---
All of these omissions where actually conscious choices. I had hoped the
compiler would make the right choice about inlining in these cases, because I
can think of uses of stdx::simd where inlining these functions would not be
optimal.

However, if the compiler decides against inlining, the result can potentially
exhibit catastrophic performance, like you experienced. This defeats the
purpose of the type and makes using it a liability. You don't have to fear for
`int * int` to ever turn into a function call either. So going forward I'll
adjust the always_inline policy for simd to follow:

  "simd is like a builtin type. Primary operations (i.e. not math functions,
etc.) are never function calls. This is independent of element type and ABI
tag."

The consequence is, that, like for builtin types, the user must consider how to
split large functions into smaller ones.

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[mkretz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

Matthias Kretz (Vir) <mkretz at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |mkretz at gcc dot gnu.org
   Target Milestone|---                         |13.0

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hubicka at gcc dot gnu.org,
                   |                            |jakub at gcc dot gnu.org

--- Comment #2 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
I bet by adding too many always_inline functions that call normal inlines that
is what is bound to happen, one runs into inline growth limits.  It is better
to use always_inline on the leaf functions rather than on what calls them.

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[mkretz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #3 from Matthias Kretz (Vir) <mkretz at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #2)
> I bet by adding too many always_inline functions that call normal inlines
> that is what is bound to happen, one runs into inline growth limits.  It is
> better to use always_inline on the leaf functions rather than on what calls
> them.

How is the inline growth limit determined? I mean, in the cases where it really
hurts, the resulting function compiles down to a single instruction (plus
parameter passing boilerplate). The optimizer cannot know about the number of
instructions, so what is the measure it uses?

Especially with the helper functions necessary to work with parameter packs /
index_sequence, it's not enough to use always_inline on the leaf functions.
E.g. any simd binary operator basically should be [[gnu::always_inline,
gnu::flatten]]. However, simd maybe shouldn't use 'flatten' for functions that
call a user-provided callable (e.g. the simd generator constructor).

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #4 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
(In reply to Matthias Kretz (Vir) from comment #3)
> (In reply to Jakub Jelinek from comment #2)
> > I bet by adding too many always_inline functions that call normal inlines
> > that is what is bound to happen, one runs into inline growth limits.  It is
> > better to use always_inline on the leaf functions rather than on what calls
> > them.
> 
> How is the inline growth limit determined? I mean, in the cases where it
> really hurts, the resulting function compiles down to a single instruction
> (plus parameter passing boilerplate). The optimizer cannot know about the
> number of instructions, so what is the measure it uses?

I've CCed Honza, who should know the answers.
The inliner can't know if say some builtin will fold into a single instruction
or not,
it uses some heuristics on GIMPLE IL sizes.  Bet -fdump-ipa-inline-details
contain
all the reasons, but at least for me those dumps are hard to read.

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #5 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Matthias Kretz <mkretz@gcc.gnu.org>:

https://gcc.gnu.org/g:53b55701aed6896f456cdec7997ac6bbef1d6074

commit r13-6090-g53b55701aed6896f456cdec7997ac6bbef1d6074
Author: Matthias Kretz <m.kretz@gsi.de>
Date:   Sat Jan 14 17:07:59 2023 +0100

    libstdc++: Annotate most lambdas with always_inline

    All of the annotated lambdas are simply a necessary means for
    implementing these functions and should never result in an actual
    function call. Many of these lambdas would go away if C++ had better
    language support for packs.

    Signed-off-by: Matthias Kretz <m.kretz@gsi.de>

    libstdc++-v3/ChangeLog:

            PR libstdc++/108030
            * include/experimental/bits/simd_detail.h: Define
            _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA.
            * include/experimental/bits/simd.h: Annotate lambdas with
            _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA.
            * include/experimental/bits/simd_builtin.h: Ditto.
            * include/experimental/bits/simd_converter.h: Ditto.
            * include/experimental/bits/simd_fixed_size.h: Ditto.
            * include/experimental/bits/simd_math.h: Ditto.
            * include/experimental/bits/simd_neon.h: Ditto.
            * include/experimental/bits/simd_x86.h: Ditto.

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[mkretz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #6 from Matthias Kretz (Vir) <mkretz at gcc dot gnu.org> ---
This improves the situation. But according to tests of fixed_size_simd in a GNU
Radio prototype it still isn't enough. It seems like I need to always_inline
all non-cmath operations on fixed_size. It's a shame to see GCC choose "4 (or
more) stores, vzeroupper, function call, 4 loads, 4 arithmetic SIMD ops, 4
stores, ret, 4 (or more) loads" over "4 arithmetic SIMD ops inlined".
For simd, the semantics should similar to builtin types, so always_inline would
anyway be the right choice (i.e. inline on -O0 as well). But there seems to be
room for improvement in GCC. I'll try to find time to produce reasonable
testcases for a new PR.

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[bernhardmgruber at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #7 from Bernhard Manfred Gruber <bernhardmgruber at gmail dot com> ---
I just want to tell you that I very much appreciate the work you are doing
here! Thank you for the time and thorough commitment!

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #8 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Matthias Kretz <mkretz@gcc.gnu.org>:

https://gcc.gnu.org/g:2e29e2fbeb8936e5c85cefaf547cba42e17e137b

commit r13-6298-g2e29e2fbeb8936e5c85cefaf547cba42e17e137b
Author: Matthias Kretz <m.kretz@gsi.de>
Date:   Sat Jan 14 23:32:38 2023 +0100

    libstdc++: Simplify three helper functions into one

    Broadcast is a very common function. This should reduce compile-time
    effort.

    Signed-off-by: Matthias Kretz <m.kretz@gsi.de>

    libstdc++-v3/ChangeLog:

            PR libstdc++/108030
            * include/experimental/bits/simd.h (__vector_broadcast):
            Implement via __vector_broadcast_impl instead of
            __call_with_n_evaluations + 2 lambdas.
            (__vector_broadcast_impl): New.

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #9 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Matthias Kretz <mkretz@gcc.gnu.org>:

https://gcc.gnu.org/g:92c47b15d5af3e7f93d11ad69a45b6d1cb8661c5

commit r13-6299-g92c47b15d5af3e7f93d11ad69a45b6d1cb8661c5
Author: Matthias Kretz <m.kretz@gsi.de>
Date:   Mon Feb 20 11:13:44 2023 +0100

    libstdc++: Fix simd build failure on clang

    Clang does not support __attribute__ on lambdas. Therefore, only set
    _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA if __clang__ is not defined.

    Signed-off-by: Matthias Kretz <m.kretz@gsi.de>

    libstdc++-v3/ChangeLog:

            PR libstdc++/108030
            * include/experimental/bits/simd_detail.h
            (_GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA): Define as empty for
            __clang__.

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #10 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-12 branch has been updated by Matthias Kretz
<mkretz@gcc.gnu.org>:

https://gcc.gnu.org/g:8e171d840584a564993201101cd1f2e920e7aecb

commit r12-9200-g8e171d840584a564993201101cd1f2e920e7aecb
Author: Matthias Kretz <m.kretz@gsi.de>
Date:   Sat Jan 14 23:32:38 2023 +0100

    libstdc++: Simplify three helper functions into one

    Broadcast is a very common function. This should reduce compile-time
    effort.

    Signed-off-by: Matthias Kretz <m.kretz@gsi.de>

    libstdc++-v3/ChangeLog:

            PR libstdc++/108030
            * include/experimental/bits/simd.h (__vector_broadcast):
            Implement via __vector_broadcast_impl instead of
            __call_with_n_evaluations + 2 lambdas.
            (__vector_broadcast_impl): New.

    (cherry picked from commit 2e29e2fbeb8936e5c85cefaf547cba42e17e137b)

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #11 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Matthias Kretz
<mkretz@gcc.gnu.org>:

https://gcc.gnu.org/g:a1f70114f9bae5a1dbcec4b556c16716601fccf1

commit r11-10543-ga1f70114f9bae5a1dbcec4b556c16716601fccf1
Author: Matthias Kretz <m.kretz@gsi.de>
Date:   Sat Jan 14 23:32:38 2023 +0100

    libstdc++: Simplify three helper functions into one

    Broadcast is a very common function. This should reduce compile-time
    effort.

    Signed-off-by: Matthias Kretz <m.kretz@gsi.de>

    libstdc++-v3/ChangeLog:

            PR libstdc++/108030
            * include/experimental/bits/simd.h (__vector_broadcast):
            Implement via __vector_broadcast_impl instead of
            __call_with_n_evaluations + 2 lambdas.
            (__vector_broadcast_impl): New.

    (cherry picked from commit 2e29e2fbeb8936e5c85cefaf547cba42e17e137b)

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #12 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Matthias Kretz <mkretz@gcc.gnu.org>:

https://gcc.gnu.org/g:e37b04328ae68f91efe1fb2c5de9122be34bc74a

commit r13-6334-ge37b04328ae68f91efe1fb2c5de9122be34bc74a
Author: Matthias Kretz <m.kretz@gsi.de>
Date:   Mon Feb 20 17:49:37 2023 +0100

    libstdc++: Always-inline most of non-cmath fixed_size implementation

    For simd, the inlining behavior should be similar to builtin types. (No
    operator on buitin types is ever translated into a function call.)
    Therefore, always_inline is the right choice (i.e. inline on -O0 as
    well).

    Signed-off-by: Matthias Kretz <m.kretz@gsi.de>

    libstdc++-v3/ChangeLog:

            PR libstdc++/108030
            * include/experimental/bits/simd_fixed_size.h
            (_SimdImplFixedSize::_S_broadcast): Replace inline with
            _GLIBCXX_SIMD_INTRINSIC.
            (_SimdImplFixedSize::_S_generate): Likewise.
            (_SimdImplFixedSize::_S_load): Likewise.
            (_SimdImplFixedSize::_S_masked_load): Likewise.
            (_SimdImplFixedSize::_S_store): Likewise.
            (_SimdImplFixedSize::_S_masked_store): Likewise.
            (_SimdImplFixedSize::_S_min): Likewise.
            (_SimdImplFixedSize::_S_max): Likewise.
            (_SimdImplFixedSize::_S_complement): Likewise.
            (_SimdImplFixedSize::_S_unary_minus): Likewise.
            (_SimdImplFixedSize::_S_plus): Likewise.
            (_SimdImplFixedSize::_S_minus): Likewise.
            (_SimdImplFixedSize::_S_multiplies): Likewise.
            (_SimdImplFixedSize::_S_divides): Likewise.
            (_SimdImplFixedSize::_S_modulus): Likewise.
            (_SimdImplFixedSize::_S_bit_and): Likewise.
            (_SimdImplFixedSize::_S_bit_or): Likewise.
            (_SimdImplFixedSize::_S_bit_xor): Likewise.
            (_SimdImplFixedSize::_S_bit_shift_left): Likewise.
            (_SimdImplFixedSize::_S_bit_shift_right): Likewise.
            (_SimdImplFixedSize::_S_remquo): Add inline keyword (to be
            explicit about not always-inline, yet).
            (_SimdImplFixedSize::_S_isinf): Likewise.
            (_SimdImplFixedSize::_S_isfinite): Likewise.
            (_SimdImplFixedSize::_S_isnan): Likewise.
            (_SimdImplFixedSize::_S_isnormal): Likewise.
            (_SimdImplFixedSize::_S_signbit): Likewise.

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[mkretz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

Matthias Kretz (Vir) <mkretz at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED

--- Comment #13 from Matthias Kretz (Vir) <mkretz at gcc dot gnu.org> ---
Fixed on trunk. Will backport later if no issues crop up.

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|13.0                        |13.2

--- Comment #14 from Richard Biener <rguenth at gcc dot gnu.org> ---
GCC 13.1 is being released, retargeting bugs to GCC 13.2.

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[mkretz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

Matthias Kretz (Vir) <mkretz at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|13.2                        |12.4

--- Comment #15 from Matthias Kretz (Vir) <mkretz at gcc dot gnu.org> ---
Ah, my bad. This was fixed for 13 already, retargeting for backport to GCC
12.4.

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #16 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-12 branch has been updated by Matthias Kretz
<mkretz@gcc.gnu.org>:

https://gcc.gnu.org/g:d3217028725f06f4eb67a4c80b12e1a3219d3502

commit r12-9637-gd3217028725f06f4eb67a4c80b12e1a3219d3502
Author: Matthias Kretz <m.kretz@gsi.de>
Date:   Sat Jan 14 17:07:59 2023 +0100

    libstdc++: Annotate most lambdas with always_inline

    All of the annotated lambdas are simply a necessary means for
    implementing these functions and should never result in an actual
    function call. Many of these lambdas would go away if C++ had better
    language support for packs.

    Signed-off-by: Matthias Kretz <m.kretz@gsi.de>

    libstdc++-v3/ChangeLog:

            PR libstdc++/108030
            * include/experimental/bits/simd_detail.h: Define
            _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA.
            * include/experimental/bits/simd.h: Annotate lambdas with
            _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA.
            * include/experimental/bits/simd_builtin.h: Ditto.
            * include/experimental/bits/simd_converter.h: Ditto.
            * include/experimental/bits/simd_fixed_size.h: Ditto.
            * include/experimental/bits/simd_math.h: Ditto.
            * include/experimental/bits/simd_neon.h: Ditto.
            * include/experimental/bits/simd_x86.h: Ditto.

    (cherry picked from commit 53b55701aed6896f456cdec7997ac6bbef1d6074)

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #17 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-12 branch has been updated by Matthias Kretz
<mkretz@gcc.gnu.org>:

https://gcc.gnu.org/g:41f75cfea48d97ea5cadd220d932163a1e17cf9a

commit r12-9638-g41f75cfea48d97ea5cadd220d932163a1e17cf9a
Author: Matthias Kretz <m.kretz@gsi.de>
Date:   Mon Feb 20 11:13:44 2023 +0100

    libstdc++: Fix simd build failure on clang

    Clang does not support __attribute__ on lambdas. Therefore, only set
    _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA if __clang__ is not defined.

    Signed-off-by: Matthias Kretz <m.kretz@gsi.de>

    libstdc++-v3/ChangeLog:

            PR libstdc++/108030
            * include/experimental/bits/simd_detail.h
            (_GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA): Define as empty for
            __clang__.

    (cherry picked from commit 92c47b15d5af3e7f93d11ad69a45b6d1cb8661c5)

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #18 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-12 branch has been updated by Matthias Kretz
<mkretz@gcc.gnu.org>:

https://gcc.gnu.org/g:c989c9196f6900c2ab6f0665751ca0320d2f3cdc

commit r12-9641-gc989c9196f6900c2ab6f0665751ca0320d2f3cdc
Author: Matthias Kretz <m.kretz@gsi.de>
Date:   Mon Feb 20 17:49:37 2023 +0100

    libstdc++: Always-inline most of non-cmath fixed_size implementation

    For simd, the inlining behavior should be similar to builtin types. (No
    operator on buitin types is ever translated into a function call.)
    Therefore, always_inline is the right choice (i.e. inline on -O0 as
    well).

    Signed-off-by: Matthias Kretz <m.kretz@gsi.de>

    libstdc++-v3/ChangeLog:

            PR libstdc++/108030
            * include/experimental/bits/simd_fixed_size.h
            (_SimdImplFixedSize::_S_broadcast): Replace inline with
            _GLIBCXX_SIMD_INTRINSIC.
            (_SimdImplFixedSize::_S_generate): Likewise.
            (_SimdImplFixedSize::_S_load): Likewise.
            (_SimdImplFixedSize::_S_masked_load): Likewise.
            (_SimdImplFixedSize::_S_store): Likewise.
            (_SimdImplFixedSize::_S_masked_store): Likewise.
            (_SimdImplFixedSize::_S_min): Likewise.
            (_SimdImplFixedSize::_S_max): Likewise.
            (_SimdImplFixedSize::_S_complement): Likewise.
            (_SimdImplFixedSize::_S_unary_minus): Likewise.
            (_SimdImplFixedSize::_S_plus): Likewise.
            (_SimdImplFixedSize::_S_minus): Likewise.
            (_SimdImplFixedSize::_S_multiplies): Likewise.
            (_SimdImplFixedSize::_S_divides): Likewise.
            (_SimdImplFixedSize::_S_modulus): Likewise.
            (_SimdImplFixedSize::_S_bit_and): Likewise.
            (_SimdImplFixedSize::_S_bit_or): Likewise.
            (_SimdImplFixedSize::_S_bit_xor): Likewise.
            (_SimdImplFixedSize::_S_bit_shift_left): Likewise.
            (_SimdImplFixedSize::_S_bit_shift_right): Likewise.
            (_SimdImplFixedSize::_S_remquo): Add inline keyword (to be
            explicit about not always-inline, yet).
            (_SimdImplFixedSize::_S_isinf): Likewise.
            (_SimdImplFixedSize::_S_isfinite): Likewise.
            (_SimdImplFixedSize::_S_isnan): Likewise.
            (_SimdImplFixedSize::_S_isnormal): Likewise.
            (_SimdImplFixedSize::_S_signbit): Likewise.

    (cherry picked from commit e37b04328ae68f91efe1fb2c5de9122be34bc74a)

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #19 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Matthias Kretz
<mkretz@gcc.gnu.org>:

https://gcc.gnu.org/g:2da5976cf683d84a1857bfa34c5c3d0d661fffbd

commit r11-10811-g2da5976cf683d84a1857bfa34c5c3d0d661fffbd
Author: Matthias Kretz <m.kretz@gsi.de>
Date:   Sat Jan 14 17:07:59 2023 +0100

    libstdc++: Annotate most lambdas with always_inline

    All of the annotated lambdas are simply a necessary means for
    implementing these functions and should never result in an actual
    function call. Many of these lambdas would go away if C++ had better
    language support for packs.

    Signed-off-by: Matthias Kretz <m.kretz@gsi.de>

    libstdc++-v3/ChangeLog:

            PR libstdc++/108030
            * include/experimental/bits/simd_detail.h: Define
            _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA.
            * include/experimental/bits/simd.h: Annotate lambdas with
            _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA.
            * include/experimental/bits/simd_builtin.h: Ditto.
            * include/experimental/bits/simd_converter.h: Ditto.
            * include/experimental/bits/simd_fixed_size.h: Ditto.
            * include/experimental/bits/simd_math.h: Ditto.
            * include/experimental/bits/simd_neon.h: Ditto.
            * include/experimental/bits/simd_x86.h: Ditto.

    (cherry picked from commit 53b55701aed6896f456cdec7997ac6bbef1d6074)

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #20 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Matthias Kretz
<mkretz@gcc.gnu.org>:

https://gcc.gnu.org/g:a2a953d11e80ac4b351bbaa4a8fa4e6b76de317c

commit r11-10812-ga2a953d11e80ac4b351bbaa4a8fa4e6b76de317c
Author: Matthias Kretz <m.kretz@gsi.de>
Date:   Mon Feb 20 11:13:44 2023 +0100

    libstdc++: Fix simd build failure on clang

    Clang does not support __attribute__ on lambdas. Therefore, only set
    _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA if __clang__ is not defined.

    Signed-off-by: Matthias Kretz <m.kretz@gsi.de>

    libstdc++-v3/ChangeLog:

            PR libstdc++/108030
            * include/experimental/bits/simd_detail.h
            (_GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA): Define as empty for
            __clang__.

    (cherry picked from commit 92c47b15d5af3e7f93d11ad69a45b6d1cb8661c5)

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

--- Comment #21 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Matthias Kretz
<mkretz@gcc.gnu.org>:

https://gcc.gnu.org/g:0911926f37cf41313c26e1e9d89424d64af972c9

commit r11-10815-g0911926f37cf41313c26e1e9d89424d64af972c9
Author: Matthias Kretz <m.kretz@gsi.de>
Date:   Mon Feb 20 17:49:37 2023 +0100

    libstdc++: Always-inline most of non-cmath fixed_size implementation

    For simd, the inlining behavior should be similar to builtin types. (No
    operator on buitin types is ever translated into a function call.)
    Therefore, always_inline is the right choice (i.e. inline on -O0 as
    well).

    Signed-off-by: Matthias Kretz <m.kretz@gsi.de>

    libstdc++-v3/ChangeLog:

            PR libstdc++/108030
            * include/experimental/bits/simd_fixed_size.h
            (_SimdImplFixedSize::_S_broadcast): Replace inline with
            _GLIBCXX_SIMD_INTRINSIC.
            (_SimdImplFixedSize::_S_generate): Likewise.
            (_SimdImplFixedSize::_S_load): Likewise.
            (_SimdImplFixedSize::_S_masked_load): Likewise.
            (_SimdImplFixedSize::_S_store): Likewise.
            (_SimdImplFixedSize::_S_masked_store): Likewise.
            (_SimdImplFixedSize::_S_min): Likewise.
            (_SimdImplFixedSize::_S_max): Likewise.
            (_SimdImplFixedSize::_S_complement): Likewise.
            (_SimdImplFixedSize::_S_unary_minus): Likewise.
            (_SimdImplFixedSize::_S_plus): Likewise.
            (_SimdImplFixedSize::_S_minus): Likewise.
            (_SimdImplFixedSize::_S_multiplies): Likewise.
            (_SimdImplFixedSize::_S_divides): Likewise.
            (_SimdImplFixedSize::_S_modulus): Likewise.
            (_SimdImplFixedSize::_S_bit_and): Likewise.
            (_SimdImplFixedSize::_S_bit_or): Likewise.
            (_SimdImplFixedSize::_S_bit_xor): Likewise.
            (_SimdImplFixedSize::_S_bit_shift_left): Likewise.
            (_SimdImplFixedSize::_S_bit_shift_right): Likewise.
            (_SimdImplFixedSize::_S_remquo): Add inline keyword (to be
            explicit about not always-inline, yet).
            (_SimdImplFixedSize::_S_isinf): Likewise.
            (_SimdImplFixedSize::_S_isfinite): Likewise.
            (_SimdImplFixedSize::_S_isnan): Likewise.
            (_SimdImplFixedSize::_S_isnormal): Likewise.
            (_SimdImplFixedSize::_S_signbit): Likewise.

    (cherry picked from commit e37b04328ae68f91efe1fb2c5de9122be34bc74a)

[Bug libstdc++/108030] `std::experimental::simd` not inlined

[mkretz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108030

Matthias Kretz (Vir) <mkretz at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #22 from Matthias Kretz (Vir) <mkretz at gcc dot gnu.org> ---
Resolved on all branches.