From: Matthias Kretz <m.kretz@gsi.de>
To: <gcc-patches@gcc.gnu.org>, <libstdc++@gcc.gnu.org>
Subject: [PATCH 5/8] libstdc++: Always-inline most of non-cmath fixed_size implementation
Date: Thu, 23 Feb 2023 09:49:57 +0100 [thread overview]
Message-ID: <3667544.MHq7AAxBmi@minbar> (raw)
In-Reply-To: <2218250.iZASKD2KPV@minbar>
[-- Attachment #1: Type: text/plain, Size: 2375 bytes --]
For simd, the inlining behavior should be similar to builtin types. (No
operator on buitin types is ever translated into a function call.)
Therefore, always_inline is the right choice (i.e. inline on -O0 as
well).
Signed-off-by: Matthias Kretz <m.kretz@gsi.de>
libstdc++-v3/ChangeLog:
PR libstdc++/108030
* include/experimental/bits/simd_fixed_size.h
(_SimdImplFixedSize::_S_broadcast): Replace inline with
_GLIBCXX_SIMD_INTRINSIC.
(_SimdImplFixedSize::_S_generate): Likewise.
(_SimdImplFixedSize::_S_load): Likewise.
(_SimdImplFixedSize::_S_masked_load): Likewise.
(_SimdImplFixedSize::_S_store): Likewise.
(_SimdImplFixedSize::_S_masked_store): Likewise.
(_SimdImplFixedSize::_S_min): Likewise.
(_SimdImplFixedSize::_S_max): Likewise.
(_SimdImplFixedSize::_S_complement): Likewise.
(_SimdImplFixedSize::_S_unary_minus): Likewise.
(_SimdImplFixedSize::_S_plus): Likewise.
(_SimdImplFixedSize::_S_minus): Likewise.
(_SimdImplFixedSize::_S_multiplies): Likewise.
(_SimdImplFixedSize::_S_divides): Likewise.
(_SimdImplFixedSize::_S_modulus): Likewise.
(_SimdImplFixedSize::_S_bit_and): Likewise.
(_SimdImplFixedSize::_S_bit_or): Likewise.
(_SimdImplFixedSize::_S_bit_xor): Likewise.
(_SimdImplFixedSize::_S_bit_shift_left): Likewise.
(_SimdImplFixedSize::_S_bit_shift_right): Likewise.
(_SimdImplFixedSize::_S_remquo): Add inline keyword (to be
explicit about not always-inline, yet).
(_SimdImplFixedSize::_S_isinf): Likewise.
(_SimdImplFixedSize::_S_isfinite): Likewise.
(_SimdImplFixedSize::_S_isnan): Likewise.
(_SimdImplFixedSize::_S_isnormal): Likewise.
(_SimdImplFixedSize::_S_signbit): Likewise.
---
.../experimental/bits/simd_fixed_size.h | 60 +++++++++----------
1 file changed, 30 insertions(+), 30 deletions(-)
--
──────────────────────────────────────────────────────────────────────────
Dr. Matthias Kretz https://mattkretz.github.io
GSI Helmholtz Centre for Heavy Ion Research https://gsi.de
stdₓ::simd
──────────────────────────────────────────────────────────────────────────
[-- Attachment #2: 0005-libstdc-Always-inline-most-of-non-cmath-fixed_size-i.patch --]
[-- Type: text/x-patch, Size: 8252 bytes --]
diff --git a/libstdc++-v3/include/experimental/bits/simd_fixed_size.h b/libstdc++-v3/include/experimental/bits/simd_fixed_size.h
index 3ac6eaa3f6b..88a9b27e359 100644
--- a/libstdc++-v3/include/experimental/bits/simd_fixed_size.h
+++ b/libstdc++-v3/include/experimental/bits/simd_fixed_size.h
@@ -1284,7 +1284,8 @@ struct _SimdImplFixedSize
// broadcast {{{2
template <typename _Tp>
- static constexpr inline _SimdMember<_Tp> _S_broadcast(_Tp __x) noexcept
+ _GLIBCXX_SIMD_INTRINSIC static constexpr _SimdMember<_Tp>
+ _S_broadcast(_Tp __x) noexcept
{
return _SimdMember<_Tp>::_S_generate(
[&](auto __meta) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA {
@@ -1294,8 +1295,8 @@ struct _SimdImplFixedSize
// _S_generator {{{2
template <typename _Fp, typename _Tp>
- static constexpr inline _SimdMember<_Tp> _S_generator(_Fp&& __gen,
- _TypeTag<_Tp>)
+ _GLIBCXX_SIMD_INTRINSIC static constexpr _SimdMember<_Tp>
+ _S_generator(_Fp&& __gen, _TypeTag<_Tp>)
{
return _SimdMember<_Tp>::_S_generate(
[&__gen](auto __meta) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA {
@@ -1310,8 +1311,8 @@ struct _SimdImplFixedSize
// _S_load {{{2
template <typename _Tp, typename _Up>
- static inline _SimdMember<_Tp> _S_load(const _Up* __mem,
- _TypeTag<_Tp>) noexcept
+ _GLIBCXX_SIMD_INTRINSIC static _SimdMember<_Tp>
+ _S_load(const _Up* __mem, _TypeTag<_Tp>) noexcept
{
return _SimdMember<_Tp>::_S_generate(
[&](auto __meta) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA {
@@ -1321,7 +1322,7 @@ struct _SimdImplFixedSize
// _S_masked_load {{{2
template <typename _Tp, typename... _As, typename _Up>
- static inline _SimdTuple<_Tp, _As...>
+ _GLIBCXX_SIMD_INTRINSIC static _SimdTuple<_Tp, _As...>
_S_masked_load(const _SimdTuple<_Tp, _As...>& __old,
const _MaskMember __bits, const _Up* __mem) noexcept
{
@@ -1344,8 +1345,8 @@ _S_masked_load(const _SimdTuple<_Tp, _As...>& __old,
// _S_store {{{2
template <typename _Tp, typename _Up>
- static inline void _S_store(const _SimdMember<_Tp>& __v, _Up* __mem,
- _TypeTag<_Tp>) noexcept
+ _GLIBCXX_SIMD_INTRINSIC static void
+ _S_store(const _SimdMember<_Tp>& __v, _Up* __mem, _TypeTag<_Tp>) noexcept
{
__for_each(__v, [&](auto __meta, auto __native) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA {
__meta._S_store(__native, &__mem[__meta._S_offset], _TypeTag<_Tp>());
@@ -1354,9 +1355,9 @@ _S_masked_load(const _SimdTuple<_Tp, _As...>& __old,
// _S_masked_store {{{2
template <typename _Tp, typename... _As, typename _Up>
- static inline void _S_masked_store(const _SimdTuple<_Tp, _As...>& __v,
- _Up* __mem,
- const _MaskMember __bits) noexcept
+ _GLIBCXX_SIMD_INTRINSIC static void
+ _S_masked_store(const _SimdTuple<_Tp, _As...>& __v, _Up* __mem,
+ const _MaskMember __bits) noexcept
{
__for_each(__v, [&](auto __meta, auto __native) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA {
if (__meta._S_submask(__bits).any())
@@ -1464,7 +1465,7 @@ __for_each(
// _S_min, _S_max {{{2
template <typename _Tp, typename... _As>
- static inline constexpr _SimdTuple<_Tp, _As...>
+ _GLIBCXX_SIMD_INTRINSIC static constexpr _SimdTuple<_Tp, _As...>
_S_min(const _SimdTuple<_Tp, _As...>& __a,
const _SimdTuple<_Tp, _As...>& __b)
{
@@ -1476,7 +1477,7 @@ _S_min(const _SimdTuple<_Tp, _As...>& __a,
}
template <typename _Tp, typename... _As>
- static inline constexpr _SimdTuple<_Tp, _As...>
+ _GLIBCXX_SIMD_INTRINSIC static constexpr _SimdTuple<_Tp, _As...>
_S_max(const _SimdTuple<_Tp, _As...>& __a,
const _SimdTuple<_Tp, _As...>& __b)
{
@@ -1489,7 +1490,7 @@ _S_max(const _SimdTuple<_Tp, _As...>& __a,
// _S_complement {{{2
template <typename _Tp, typename... _As>
- static inline constexpr _SimdTuple<_Tp, _As...>
+ _GLIBCXX_SIMD_INTRINSIC static constexpr _SimdTuple<_Tp, _As...>
_S_complement(const _SimdTuple<_Tp, _As...>& __x) noexcept
{
return __x._M_apply_per_chunk(
@@ -1500,7 +1501,7 @@ _S_complement(const _SimdTuple<_Tp, _As...>& __x) noexcept
// _S_unary_minus {{{2
template <typename _Tp, typename... _As>
- static inline constexpr _SimdTuple<_Tp, _As...>
+ _GLIBCXX_SIMD_INTRINSIC static constexpr _SimdTuple<_Tp, _As...>
_S_unary_minus(const _SimdTuple<_Tp, _As...>& __x) noexcept
{
return __x._M_apply_per_chunk(
@@ -1513,7 +1514,7 @@ _S_unary_minus(const _SimdTuple<_Tp, _As...>& __x) noexcept
#define _GLIBCXX_SIMD_FIXED_OP(name_, op_) \
template <typename _Tp, typename... _As> \
- static inline constexpr _SimdTuple<_Tp, _As...> name_( \
+ _GLIBCXX_SIMD_INTRINSIC static constexpr _SimdTuple<_Tp, _As...> name_( \
const _SimdTuple<_Tp, _As...>& __x, const _SimdTuple<_Tp, _As...>& __y) \
{ \
return __x._M_apply_per_chunk( \
@@ -1536,7 +1537,7 @@ _S_unary_minus(const _SimdTuple<_Tp, _As...>& __x) noexcept
#undef _GLIBCXX_SIMD_FIXED_OP
template <typename _Tp, typename... _As>
- static inline constexpr _SimdTuple<_Tp, _As...>
+ _GLIBCXX_SIMD_INTRINSIC static constexpr _SimdTuple<_Tp, _As...>
_S_bit_shift_left(const _SimdTuple<_Tp, _As...>& __x, int __y)
{
return __x._M_apply_per_chunk(
@@ -1546,7 +1547,7 @@ _S_bit_shift_left(const _SimdTuple<_Tp, _As...>& __x, int __y)
}
template <typename _Tp, typename... _As>
- static inline constexpr _SimdTuple<_Tp, _As...>
+ _GLIBCXX_SIMD_INTRINSIC static constexpr _SimdTuple<_Tp, _As...>
_S_bit_shift_right(const _SimdTuple<_Tp, _As...>& __x, int __y)
{
return __x._M_apply_per_chunk(
@@ -1665,10 +1666,9 @@ _S_bit_shift_right(const _SimdTuple<_Tp, _As...>& __x, int __y)
#undef _GLIBCXX_SIMD_APPLY_ON_TUPLE
template <typename _Tp, typename... _Abis>
- static _SimdTuple<_Tp, _Abis...> _S_remquo(
- const _SimdTuple<_Tp, _Abis...>& __x,
- const _SimdTuple<_Tp, _Abis...>& __y,
- __fixed_size_storage_t<int, _SimdTuple<_Tp, _Abis...>::_S_size()>* __z)
+ static inline _SimdTuple<_Tp, _Abis...>
+ _S_remquo(const _SimdTuple<_Tp, _Abis...>& __x, const _SimdTuple<_Tp, _Abis...>& __y,
+ __fixed_size_storage_t<int, _SimdTuple<_Tp, _Abis...>::_S_size()>* __z)
{
return __x._M_apply_per_chunk(
[](auto __impl, const auto __xx, const auto __yy, auto& __zz)
@@ -1689,14 +1689,14 @@ _S_frexp(const _SimdTuple<_Tp, _As...>& __x,
}, __exp);
}
-#define _GLIBCXX_SIMD_TEST_ON_TUPLE_(name_) \
- template <typename _Tp, typename... _As> \
- static inline _MaskMember \
- _S_##name_(const _SimdTuple<_Tp, _As...>& __x) noexcept \
- { \
- return _M_test([](auto __impl, \
- auto __xx) { return __impl._S_##name_(__xx); }, \
- __x); \
+#define _GLIBCXX_SIMD_TEST_ON_TUPLE_(name_) \
+ template <typename _Tp, typename... _As> \
+ static inline _MaskMember \
+ _S_##name_(const _SimdTuple<_Tp, _As...>& __x) noexcept \
+ { \
+ return _M_test([] (auto __impl, auto __xx) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { \
+ return __impl._S_##name_(__xx); \
+ }, __x); \
}
_GLIBCXX_SIMD_TEST_ON_TUPLE_(isinf)
next prev parent reply other threads:[~2023-02-23 8:50 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-23 8:48 [PATCH 0/8] std::experimental::simd patchset Matthias Kretz
2023-02-23 8:49 ` [PATCH 1/8] libstdc++: Simplify three helper functions into one Matthias Kretz
2023-02-23 11:05 ` Jonathan Wakely
2023-02-23 8:49 ` [PATCH 2/8] libstdc++: Fix simd build failure on clang Matthias Kretz
2023-02-23 11:06 ` Jonathan Wakely
2023-02-23 8:49 ` [PATCH 3/8] libstdc++: More efficient masked inc-/decrement implementation Matthias Kretz
2023-02-24 17:12 ` Jonathan Wakely
2023-02-23 8:49 ` [PATCH 4/8] libstdc++: Add missing constexpr on simd shift implementation Matthias Kretz
2023-02-23 11:07 ` Jonathan Wakely
2023-02-23 11:33 ` Matthias Kretz
2023-02-23 8:49 ` Matthias Kretz [this message]
2023-02-24 17:10 ` [PATCH 5/8] libstdc++: Always-inline most of non-cmath fixed_size implementation Jonathan Wakely
2023-02-23 8:50 ` [PATCH 6/8] libstdc++: Fix formatting Matthias Kretz
2023-02-24 17:14 ` Jonathan Wakely
2023-02-24 18:44 ` Matthias Kretz
2023-02-23 8:50 ` [PATCH 7/8] libstdc++: Fix -Wsign-compare issue Matthias Kretz
2023-02-23 11:07 ` Jonathan Wakely
2023-02-23 8:50 ` [PATCH 8/8] libstdc++: Test that integral simd reductions are precise Matthias Kretz
2023-02-23 11:08 ` Jonathan Wakely
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3667544.MHq7AAxBmi@minbar \
--to=m.kretz@gsi.de \
--cc=gcc-patches@gcc.gnu.org \
--cc=libstdc++@gcc.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).