[Bug libstdc++/99396] New: std::rotl and std::rotr do not invoke intrinsics, leading to serious performance issues

[Bug libstdc++/99396] New: std::rotl and std::rotr do not invoke intrinsics, leading to serious performance issues

[unlvsur at live dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

            Bug ID: 99396
           Summary: std::rotl and std::rotr do not invoke intrinsics,
                    leading to serious performance issues
           Product: gcc
           Version: 11.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libstdc++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: unlvsur at live dot com
  Target Milestone: ---

https://godbolt.org/z/snG9fs

[Bug tree-optimization/99396] std::rotl and std::rotr do not invoke intrinsics, leading to serious performance issues

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|libstdc++                   |tree-optimization
             Status|UNCONFIRMED                 |NEW
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2021-03-05

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
    return std::rotr(v,((unsigned int)t)&63);
Fixes the problem ....
    int a = std::abs(t);
    a &= 63;
    return std::rotr(v,a);

Also improves the situtation.
I am thinking the std::rotr could be rewritten or we could optimize this at the
tree level into the almost the intrinsics.
Note I think I Might have gotten the above incorrect a little bit.

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|std::rotl and std::rotr do  |std::rotl and std::rotr
                   |not invoke intrinsics,      |Does not convert into
                   |leading to serious          |ROTATE on the gimple level
                   |performance issues          |
             Target|i386,x86_64. Does arm have  |
                   |intrinsics for this?        |
           Severity|normal                      |enhancement
           Keywords|                            |missed-optimization

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[unlvsur at live dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

--- Comment #2 from cqwrteur <unlvsur at live dot com> ---
(In reply to Andrew Pinski from comment #1)
>     return std::rotr(v,((unsigned int)t)&63);
> Fixes the problem ....
>     int a = std::abs(t);
>     a &= 63;
>     return std::rotr(v,a);
> 
> Also improves the situtation.
> I am thinking the std::rotr could be rewritten or we could optimize this at
> the tree level into the almost the intrinsics.
> Note I think I Might have gotten the above incorrect a little bit.

I would like to see __builtin_cpp_rotl and __builtin_cpp_rotr to allow more
aggressive optimizations since rotl and rotr are so important for cryptography.

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

--- Comment #3 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to cqwrteur from comment #2)
> I would like to see __builtin_cpp_rotl and __builtin_cpp_rotr to allow more
> aggressive optimizations since rotl and rotr are so important for
> cryptography.

You don't need them.
The following code will already produce the rotate instruction:
auto f3(std::size_t v, int t)
{
  return (v<<t) | (v>>(sizeof(v)*8 - t));
}

Note std::rotl/std::rotr has specific behavior dealing with 0 and negative
values and all.

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[unlvsur at live dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

--- Comment #4 from cqwrteur <unlvsur at live dot com> ---
(In reply to Andrew Pinski from comment #3)
> (In reply to cqwrteur from comment #2)
> > I would like to see __builtin_cpp_rotl and __builtin_cpp_rotr to allow more
> > aggressive optimizations since rotl and rotr are so important for
> > cryptography.
> 
> You don't need them.
> The following code will already produce the rotate instruction:
> auto f3(std::size_t v, int t)
> {
>   return (v<<t) | (v>>(sizeof(v)*8 - t));
> }
> 
> Note std::rotl/std::rotr has specific behavior dealing with 0 and negative
> values and all.

but if it is constant, it would get optimized to just ror or rol

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|jwakely.gcc at gmail dot com       |

--- Comment #5 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Stop CCing me on every bug you file, or I will ban your account, permanently
this time.

I read the gcc-bugs mailing list, so I will see the bug anyway. Stop CCing me.
I won't ask again, I'll just ban you.

Also, don't just post a godbolt link with no other comments, that's not a valid
but report.

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #6 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
There is absolutely no reason why libstdc++ should use any intrinsics for the
rotates, gcc recognizes a lot of patterns into rotates.
Just not the extra verbose one used in libstdc++.
The comment in gcc says:
/* Recognize rotation patterns.  Return true if a transformation
   applied, otherwise return false.

   We are looking for X with unsigned type T with bitsize B, OP being
   +, | or ^, some type T2 wider than T.  For:
   (X << CNT1) OP (X >> CNT2)                           iff CNT1 + CNT2 == B
   ((T) ((T2) X << CNT1)) OP ((T) ((T2) X >> CNT2))     iff CNT1 + CNT2 == B

   transform these into:
   X r<< CNT1

   Or for:
   (X << Y) OP (X >> (B - Y))
   (X << (int) Y) OP (X >> (int) (B - Y))
   ((T) ((T2) X << Y)) OP ((T) ((T2) X >> (B - Y)))
   ((T) ((T2) X << (int) Y)) OP ((T) ((T2) X >> (int) (B - Y)))
   (X << Y) | (X >> ((-Y) & (B - 1)))
   (X << (int) Y) | (X >> (int) ((-Y) & (B - 1)))
   ((T) ((T2) X << Y)) | ((T) ((T2) X >> ((-Y) & (B - 1))))
   ((T) ((T2) X << (int) Y)) | ((T) ((T2) X >> (int) ((-Y) & (B - 1))))

   transform these into:
   X r<< Y

   Or for:
   (X << (Y & (B - 1))) | (X >> ((-Y) & (B - 1)))
   (X << (int) (Y & (B - 1))) | (X >> (int) ((-Y) & (B - 1)))
   ((T) ((T2) X << (Y & (B - 1)))) | ((T) ((T2) X >> ((-Y) & (B - 1))))
   ((T) ((T2) X << (int) (Y & (B - 1)))) \
     | ((T) ((T2) X >> (int) ((-Y) & (B - 1))))

   transform these into:
   X r<< (Y & (B - 1))

   Note, in the patterns with T2 type, the type of OP operands
   might be even a signed type, but should have precision B.
   Expressions with & (B - 1) should be recognized only if B is
   a power of 2.  */

but libstdc++ does e.g.
      constexpr auto _Nd = __gnu_cxx::__int_traits<_Tp>::__digits;
      const int __r = __s % _Nd;
      if (__r == 0)
        return __x;
      else if (__r > 0)
        return (__x << __r) | (__x >> ((_Nd - __r) % _Nd));
      else
        return (__x >> -__r) | (__x << ((_Nd + __r) % _Nd)); // rotr(x, -r)
So, can't it e.g. use
      constexpr auto _Nd = __gnu_cxx::__int_traits<_Tp>::__digits;
      const auto __r = static_cast<unsigned int>(__s);
      return (__x << (__r % _Nd)) | (__x >> ((-__r) % _Nd));
?

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

--- Comment #7 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Ah, but __digits is one smaller than we want for signed types.
Plus before C++20 the left shifts of negative values are UB?
Maybe all the rotates should be implemented using the corresponding unsigned
types...

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

--- Comment #8 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
so
      using __unsigned_type = __make_unsigned<_Tp>::__type;
      constexpr auto _Nd = __gnu_cxx::__int_traits<__unsigned_type>::__digits;
      const auto __r = static_cast<unsigned int>(__s);
      const auto __y = static_cast<__unsigned_type>(__x;
      const auto __z = (__y << (__r % _Nd)) | (__y >> ((-__r) % _Nd));
      return static_cast<_Tp>(__z);
?  Of course, if _Tp can be anything other than integral types, it would need
to be a specialization for integral types (though it will work for __int128
fine if that doesn't count as integral).

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |jakub at gcc dot gnu.org

--- Comment #9 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Created attachment 50303
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=50303&action=edit
gcc11-pr99396.patch

This seems to work for me (almost, there is a useless and for uchar/ushort) on:
#include <bit>

unsigned char f1 (unsigned char x, int y) { return std::rotl (x, y); }
unsigned char f2 (unsigned char x, int y) { return std::rotr (x, y); }
unsigned short f3 (unsigned short x, int y) { return std::rotl (x, y); }
unsigned short f4 (unsigned short x, int y) { return std::rotr (x, y); }
unsigned int f5 (unsigned int x, int y) { return std::rotl (x, y); }
unsigned int f6 (unsigned int x, int y) { return std::rotr (x, y); }
unsigned long int f7 (unsigned long int x, int y) { return std::rotl (x, y); }
unsigned long int f8 (unsigned long int x, int y) { return std::rotr (x, y); }
unsigned long long int f9 (unsigned long long int x, int y) { return std::rotl
(x, y); }
unsigned long long int f10 (unsigned long long int x, int y) { return std::rotr
(x, y); }
//unsigned __int128 f11 (unsigned __int128 x, int y) { return std::rotl (x, y);
}
//unsigned __int128 f12 (unsigned __int128 x, int y) { return std::rotr (x, y);
}

constexpr auto a = std::rotl (1234U, 0);
constexpr auto b = std::rotl (1234U, 5);
constexpr auto c = std::rotl (1234U, -5);
constexpr auto d = std::rotl (1234U, -__INT_MAX__ - 1);

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

--- Comment #10 from Jonathan Wakely <redi at gcc dot gnu.org> ---
As you noted on IRC, these functions are undefined for anything except unsigned
integral types. Adding that here for observers wondering about comments 7 and
8.

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #50303|0                           |1
        is obsolete|                            |

--- Comment #11 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Created attachment 50304
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=50304&action=edit
gcc11-pr99396.patch

Updated patch.  I'm afraid the previous patch wouldn't work properly on weirdo
types that have _Nd which is not a power of two.
In such cases, (e.g. for __int20 type on some targets), ~0U + 1ULL is not
divisible by _Nd and so I think it wouldn't handle properly std::rot[rl] with
negative second arguments.

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #50304|0                           |1
        is obsolete|                            |

--- Comment #12 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Created attachment 50305
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=50305&action=edit
gcc11-pr99396.patch

Some further tweaks based on IRC discussions.

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[unlvsur at live dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

--- Comment #13 from cqwrteur <unlvsur at live dot com> ---
(In reply to Jakub Jelinek from comment #12)
> Created attachment 50305 [details]
> gcc11-pr99396.patch
> 
> Some further tweaks based on IRC discussions.

shouldn't this be

if constexpr ((_Nd & (_Nd - 1)) == 0)
{
}
else
{
}

instead of if ((_Nd & (_Nd - 1)) == 0) ?

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

--- Comment #14 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
I believe std::__rot{l,r} can be used even in C++14 and if constexpr is only
supported in C++17 and later.
With optimizations enabled (_Nd & (_Nd - 1)) == 0 will optimize into constant
anyway.

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

--- Comment #15 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Could be if _GLIBCXX17_CONSTEXPR (...)
sure.

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[unlvsur at live dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

--- Comment #16 from cqwrteur <unlvsur at live dot com> ---
(In reply to Jakub Jelinek from comment #15)
> Could be if _GLIBCXX17_CONSTEXPR (...)
> sure.

bit header is a new C++20 header. There is no reason the compiler does not
support if constexpr.

If you compile it with pre-C++20, the code should even get compiled

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[unlvsur at live dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

--- Comment #17 from cqwrteur <unlvsur at live dot com> ---
(In reply to cqwrteur from comment #16)
> (In reply to Jakub Jelinek from comment #15)
> > Could be if _GLIBCXX17_CONSTEXPR (...)
> > sure.
> 
> bit header is a new C++20 header. There is no reason the compiler does not
> support if constexpr.
> 
> If you compile it with pre-C++20, the code should even get compiled

sorry. should not even get compiled

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[unlvsur at live dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

--- Comment #18 from cqwrteur <unlvsur at live dot com> ---
(In reply to cqwrteur from comment #17)
> (In reply to cqwrteur from comment #16)
> > (In reply to Jakub Jelinek from comment #15)
> > > Could be if _GLIBCXX17_CONSTEXPR (...)
> > > sure.
> > 
> > bit header is a new C++20 header. There is no reason the compiler does not
> > support if constexpr.
> > 
> > If you compile it with pre-C++20, the code should even get compiled
> 
> sorry. should not even get compiled

okay. it looks like you folks want to support C++14 and C++17 as an extension.

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

--- Comment #19 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jakub Jelinek <jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:84185598dc7470bad4e7f8c22b64e3c944efb670

commit r11-7542-g84185598dc7470bad4e7f8c22b64e3c944efb670
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Sat Mar 6 11:11:30 2021 +0100

    libstdc++: Improve std::rot[lr] [PR99396]

    As can be seen on:

    unsigned char f1 (unsigned char x, int y) { return std::rotl (x, y); }
    unsigned char f2 (unsigned char x, int y) { return std::rotr (x, y); }
    unsigned short f3 (unsigned short x, int y) { return std::rotl (x, y); }
    unsigned short f4 (unsigned short x, int y) { return std::rotr (x, y); }
    unsigned int f5 (unsigned int x, int y) { return std::rotl (x, y); }
    unsigned int f6 (unsigned int x, int y) { return std::rotr (x, y); }
    unsigned long int f7 (unsigned long int x, int y) { return std::rotl (x,
y); }
    unsigned long int f8 (unsigned long int x, int y) { return std::rotr (x,
y); }
    unsigned long long int f9 (unsigned long long int x, int y) { return
std::rotl (x, y); }
    unsigned long long int f10 (unsigned long long int x, int y) { return
std::rotr (x, y); }
    //unsigned __int128 f11 (unsigned __int128 x, int y) { return std::rotl (x,
y); }
    //unsigned __int128 f12 (unsigned __int128 x, int y) { return std::rotr (x,
y); }

    constexpr auto a = std::rotl (1234U, 0);
    constexpr auto b = std::rotl (1234U, 5);
    constexpr auto c = std::rotl (1234U, -5);
    constexpr auto d = std::rotl (1234U, -__INT_MAX__ - 1);
    the current <bit> definitions of std::__rot[lr] aren't pattern recognized
    as rotates, they are too long/complex for that, starting with signed
modulo,
    special case for 0 and different cases for positive and negative.

    For types with power of two bits the following patch adds definitions that
    the compiler can pattern recognize and turn e.g. on x86_64 into
ro[lr][bwlq]
    instructions.  For weirdo types like unsigned __int20 etc. it keeps the
    current definitions.

    2021-03-06  Jakub Jelinek  <jakub@redhat.com>

            PR libstdc++/99396
            * include/std/bit (__rotl, __rotr): Add optimized variants for
power of
            two _Nd which the compiler can pattern match the rotates.

[Bug tree-optimization/99396] std::rotl and std::rotr Does not convert into ROTATE on the gimple level

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99396

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #20 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Fixed.