[Bug c++/109883] New: Stack Overflow in <cmath> functions with <stdfloat> types

[Bug c++/109883] New: Stack Overflow in <cmath> functions with <stdfloat> types

[matt at mattborland dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

            Bug ID: 109883
           Summary: Stack Overflow in <cmath> functions with <stdfloat>
                    types
           Product: gcc
           Version: 13.1.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: matt at mattborland dot com
  Target Milestone: ---

The two or more argument functions in <cmath> cause a stack overflow when
called with an <stdfloat> type and any integer type. Running with ASAN yields
"AddressSanitizer: nested bug in the same thread". I am able to reproduce this
bug on Fedora 38 with GCC 13.1.1 on x86_64, and macOS 13.3.1 with GCC 13.1.0 on
M1.

A minimal reproducer:

#include <stdfloat>
#include <cmath>

int main()
{
    std::pow(0.5F64, 2);
}

[Bug libstdc++/109883] Stack Overflow in <cmath> functions with <stdfloat> types

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

Xi Ruoyao <xry111 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |xry111 at gcc dot gnu.org

--- Comment #1 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
Cannot reproduce for me.  Note that in this case GCC optimizes the entire
function call away (see https://godbolt.org/z/968bPTvh9) even with -O0 so I can
see no way how this will lead to a runtime error.

And GCC for aarch64-darwin target (i. e. "macOS 13.3.1 on M1") is not a part of
this project, so are you using another fork?

[Bug libstdc++/109883] Stack Overflow in <cmath> functions with <stdfloat> types

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

Xi Ruoyao <xry111 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2023-05-17
             Status|UNCONFIRMED                 |WAITING

[Bug libstdc++/109883] Stack Overflow in <cmath> functions with <stdfloat> types

[matt at mattborland dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

--- Comment #2 from Matt Borland <matt at mattborland dot com> ---
(In reply to Xi Ruoyao from comment #1)
> Cannot reproduce for me.  Note that in this case GCC optimizes the entire
> function call away (see https://godbolt.org/z/968bPTvh9) even with -O0 so I
> can see no way how this will lead to a runtime error.

Here is an updated reproducer:

#include <iostream>
#include <stdfloat>
#include <cmath>

int main()
{
    auto val = std::pow(0.5F64, 2);
    std::cout << val << std::endl;
}

The failure can be seen godbolt here: https://godbolt.org/z/ej5nPn7o4. Running
this same snippet locally with ASAN yields:

AddressSanitizer:DEADLYSIGNAL
=================================================================
==110879==ERROR: AddressSanitizer: stack-overflow on address 0x7fff6e2e7ff8 (pc
0x00000040126e bp 0x7fff6e2e8010 sp 0x7fff6e2e8000 T0)

    #0 0x40126e in __gnu_cxx::__promote_2<decltype
(((__gnu_cxx::__promote_2<_Float64,
std::__is_integer<_Float64>::__value>::__type)(0))+((__gnu_cxx::__promote_2<_Float64,
std::__is_integer<_Float64>::__value>::__type)(0))), std::__is_integer<decltype
(((__gnu_cxx::__promote_2<_Float64,
std::__is_integer<_Float64>::__value>::__type)(0))+((__gnu_cxx::__promote_2<_Float64,
std::__is_integer<_Float64>::__value>::__type)(0)))>::__value>::__type
std::pow<_Float64, _Float64>(_Float64, _Float64)
(/home/mborland/Documents/boost/libs/math/test/so+0x40126e) (BuildId:
6f720390f8d2a24a6dabec3c85e9cf5bb4c192ea)

SUMMARY: AddressSanitizer: stack-overflow
(/home/mborland/Documents/boost/libs/math/test/so+0x40126e) (BuildId:
6f720390f8d2a24a6dabec3c85e9cf5bb4c192ea) in __gnu_cxx::__promote_2<decltype
(((__gnu_cxx::__promote_2<_Float64,
std::__is_integer<_Float64>::__value>::__type)(0))+((__gnu_cxx::__promote_2<_Float64,
std::__is_integer<_Float64>::__value>::__type)(0))), std::__is_integer<decltype
(((__gnu_cxx::__promote_2<_Float64,
std::__is_integer<_Float64>::__value>::__type)(0))+((__gnu_cxx::__promote_2<_Float64,
std::__is_integer<_Float64>::__value>::__type)(0)))>::__value>::__type
std::pow<_Float64, _Float64>(_Float64, _Float64)
==110879==ABORTING

For brevity I snipped out 245 more instances of the message next to #0.

> And GCC for aarch64-darwin target (i. e. "macOS 13.3.1 on M1") is not a part
> of this project, so are you using another fork?

It is provided by homebrew as gcc@13. For this reply I am using my Fedora 38
system with "gcc version 13.1.1 20230511 (Red Hat 13.1.1-2) (GCC)"

[Bug libstdc++/109883] Stack Overflow in <cmath> functions with <stdfloat> types and -std=c++23

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

Xi Ruoyao <xry111 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Known to fail|                            |13.1.0, 14.0
             Status|WAITING                     |NEW
            Summary|Stack Overflow in <cmath>   |Stack Overflow in <cmath>
                   |functions with <stdfloat>   |functions with <stdfloat>
                   |types                       |types and -std=c++23

--- Comment #3 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
Confirmed.  -std=c++23 is needed to reproduce.

[Bug libstdc++/109883] Stack Overflow in <cmath> functions with <stdfloat> types and -std=c++23

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

--- Comment #4 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
It seems the function

__gnu_cxx::__promote_2<decltype (((__gnu_cxx::__promote_2<_Float64,
std::__is_integer<_Float64>::__value>::__type)(0))+((__gnu_cxx::__promote_2<_Float64,
std::__is_integer<_Float64>::__value>::__type)(0))), std::__is_integer<decltype
(((__gnu_cxx::__promote_2<_Float64,
std::__is_integer<_Float64>::__value>::__type)(0))+((__gnu_cxx::__promote_2<_Float64,
std::__is_integer<_Float64>::__value>::__type)(0)))>::__value>::__type
std::pow<_Float64, _Float64>(_Float64, _Float64)

is recursing infinitely.

[Bug libstdc++/109883] Stack Overflow in <cmath> functions with <stdfloat> types and -std=c++23

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

--- Comment #5 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
0000000000001203
<_ZSt3powIDF64_DF64_EN9__gnu_cxx11__promote_2IDTplcvNS1_IT_XsrSt12__is_integerIS2_E7__valueEE6__typeELi0EcvNS1_IT0_XsrS3_IS7_E7__valueEE6__typeELi0EEXsrS3_ISB_E7__valueEE6__typeES2_S7_>:
    1203:       55                      push   %rbp
    1204:       48 89 e5                mov    %rsp,%rbp
    1207:       48 83 ec 10             sub    $0x10,%rsp
    120b:       f2 0f 11 45 f8          movsd  %xmm0,-0x8(%rbp)
    1210:       f2 0f 11 4d f0          movsd  %xmm1,-0x10(%rbp)
    1215:       f2 0f 10 45 f0          movsd  -0x10(%rbp),%xmm0
    121a:       48 8b 45 f8             mov    -0x8(%rbp),%rax
    121e:       66 0f 28 c8             movapd %xmm0,%xmm1
    1222:       66 48 0f 6e c0          movq   %rax,%xmm0
    1227:       e8 d7 ff ff ff          call   1203
<_ZSt3powIDF64_DF64_EN9__gnu_cxx11__promote_2IDTplcvNS1_IT_XsrSt12__is_integerIS2_E7__valueEE6__typeELi0EcvNS1_IT0_XsrS3_IS7_E7__valueEE6__typeELi0EEXsrS3_ISB_E7__valueEE6__typeES2_S7_>
    122c:       66 48 0f 7e c0          movq   %xmm0,%rax
    1231:       66 48 0f 6e c0          movq   %rax,%xmm0
    1236:       c9                      leave
    1237:       c3                      ret

This is just stupid...

[Bug libstdc++/109883] Stack Overflow in <cmath> functions with <stdfloat> types and -std=c++23

[matt at mattborland dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

--- Comment #6 from Matt Borland <matt at mattborland dot com> ---
(In reply to Xi Ruoyao from comment #4)
> It seems the function
> 
> __gnu_cxx::__promote_2<decltype (((__gnu_cxx::__promote_2<_Float64,
> std::__is_integer<_Float64>::__value>::__type)(0))+((__gnu_cxx::
> __promote_2<_Float64, std::__is_integer<_Float64>::__value>::__type)(0))),
> std::__is_integer<decltype (((__gnu_cxx::__promote_2<_Float64,
> std::__is_integer<_Float64>::__value>::__type)(0))+((__gnu_cxx::
> __promote_2<_Float64,
> std::__is_integer<_Float64>::__value>::__type)(0)))>::__value>::__type
> std::pow<_Float64, _Float64>(_Float64, _Float64)
> 
> is recursing infinitely.

For Boost.Math's implementation of promote_2 we found template specializations
to be effective:
https://github.com/boostorg/math/pull/978/files#diff-2463d99030329b154489b8b34ce1068a34e736cab268c3421b058ca0e516680cR189.

[Bug libstdc++/109883] Stack Overflow in <cmath> functions with <stdfloat> types and -std=c++23

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

--- Comment #7 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
I think we need to move those __promote_{2,3} using templates for atan2, fmod,
pow, copysign, fdim, fmax, fmin, hypot, nextafter, remainder, remquo and fma
later, because right now we have the overloads with float, double and long
double arguments, then these templates and later on _Float{16,32,64,128} and
bfloat16_t overloads, and as those __promote_{2,3} templates call itself with
promoted arguments, they self-recurse if the promoted arguments are
_Float{16,32,64,128} or bfloat16_t.

[Bug libstdc++/109883] Stack Overflow in <cmath> functions with <stdfloat> types and -std=c++23

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #8 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Created attachment 55100
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=55100&action=edit
gcc14-pr109883.patch

Untested fix.  Still need to add some testsuite coverage.

[Bug libstdc++/109883] Stack Overflow in <cmath> functions with <stdfloat> types and -std=c++23

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #55100|0                           |1
        is obsolete|                            |
             Status|NEW                         |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |jakub at gcc dot gnu.org

--- Comment #9 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Created attachment 55102
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=55102&action=edit
gcc14-pr109883.patch

Updated patch including testsuite coverage.

[Bug libstdc++/109883] Stack Overflow in <cmath> functions with <stdfloat> types and -std=c++23

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

--- Comment #10 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jakub Jelinek <jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:883f1e25dc7907c9bb37f480b900336a050218f1

commit r14-949-g883f1e25dc7907c9bb37f480b900336a050218f1
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Wed May 17 21:21:23 2023 +0200

    libstdc++: Fix up some <cmath> templates [PR109883]

    As can be seen on the following testcase, for
   
std::{atan2,fmod,pow,copysign,fdim,fmax,fmin,hypot,nextafter,remainder,remquo,fma}
    if one operand type is std::float{16,32,64,128}_t or std::bfloat16_t and
    another one some integral type or some other floating point type which
    promotes to the other operand's type, we can end up with endless recursion.
    This is because of a declaration ordering problem in <cmath>, where the
    float, double and long double overloads of those functions come before
    the templates which use __gnu_cxx::__promote_{2,3}, but the
    std::float{16,32,64,128}_t and std::bfloat16_t overloads come later in the
    file.  If the result of those promotions is _Float{16,32,64,128} or
    __gnu_cxx::__bfloat16_t, say std::pow(_Float64, int) calls
    std::pow(_Float64, _Float64) and the latter calls itself.

    The following patch fixes that by moving those templates later in the file,
    so that the calls from those templates see also the other overloads.

    I think other templates in the file like e.g. isgreater etc. shouldn't be
    a problem, because those just use __builtin_isgreater etc. in their bodies.

    2023-05-17  Jakub Jelinek  <jakub@redhat.com>

            PR libstdc++/109883
            * include/c_global/cmath (atan2, fmod, pow): Move
            __gnu_cxx::__promote_2 using templates after _Float{16,32,64,128}
and
            __gnu_cxx::__bfloat16_t overloads.
            (copysign, fdim, fmax, fmin, hypot, nextafter, remainder, remquo):
            Likewise.
            (fma): Move __gnu_cxx::__promote_3 using template after
            _Float{16,32,64,128} and __gnu_cxx::__bfloat16_t overloads.

            * testsuite/26_numerics/headers/cmath/constexpr_std_c++23.cc: New
test.

[Bug libstdc++/109883] Stack Overflow in <cmath> functions with <stdfloat> types and -std=c++23

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

--- Comment #11 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-13 branch has been updated by Jakub Jelinek
<jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:c42950e2d380d0da26203fb1eb39497c0a400b2d

commit r13-7341-gc42950e2d380d0da26203fb1eb39497c0a400b2d
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Wed May 17 21:21:23 2023 +0200

    libstdc++: Fix up some <cmath> templates [PR109883]

    As can be seen on the following testcase, for
   
std::{atan2,fmod,pow,copysign,fdim,fmax,fmin,hypot,nextafter,remainder,remquo,fma}
    if one operand type is std::float{16,32,64,128}_t or std::bfloat16_t and
    another one some integral type or some other floating point type which
    promotes to the other operand's type, we can end up with endless recursion.
    This is because of a declaration ordering problem in <cmath>, where the
    float, double and long double overloads of those functions come before
    the templates which use __gnu_cxx::__promote_{2,3}, but the
    std::float{16,32,64,128}_t and std::bfloat16_t overloads come later in the
    file.  If the result of those promotions is _Float{16,32,64,128} or
    __gnu_cxx::__bfloat16_t, say std::pow(_Float64, int) calls
    std::pow(_Float64, _Float64) and the latter calls itself.

    The following patch fixes that by moving those templates later in the file,
    so that the calls from those templates see also the other overloads.

    I think other templates in the file like e.g. isgreater etc. shouldn't be
    a problem, because those just use __builtin_isgreater etc. in their bodies.

    2023-05-17  Jakub Jelinek  <jakub@redhat.com>

            PR libstdc++/109883
            * include/c_global/cmath (atan2, fmod, pow): Move
            __gnu_cxx::__promote_2 using templates after _Float{16,32,64,128}
and
            __gnu_cxx::__bfloat16_t overloads.
            (copysign, fdim, fmax, fmin, hypot, nextafter, remainder, remquo):
            Likewise.
            (fma): Move __gnu_cxx::__promote_3 using template after
            _Float{16,32,64,128} and __gnu_cxx::__bfloat16_t overloads.

            * testsuite/26_numerics/headers/cmath/constexpr_std_c++23.cc: New
test.

    (cherry picked from commit 883f1e25dc7907c9bb37f480b900336a050218f1)

[Bug libstdc++/109883] Stack Overflow in <cmath> functions with <stdfloat> types and -std=c++23

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|ASSIGNED                    |RESOLVED

--- Comment #12 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Fixed for 13.2/14.1.

[Bug libstdc++/109883] Stack Overflow in <cmath> functions with <stdfloat> types and -std=c++23

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109883

Xi Ruoyao <xry111 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |13.2