[Bug libstdc++/92978] std::gcd mishandles mixed-signedness

[Bug libstdc++/92978] std::gcd mishandles mixed-signedness

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92978

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |redi at gcc dot gnu.org

[Bug libstdc++/92978] std::gcd mishandles mixed-signedness

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92978

--- Comment #1 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jonathan Wakely <redi@gcc.gnu.org>:

https://gcc.gnu.org/g:82db1a42e9254c9009bbf8ac01366da4d1ab6df5

commit r11-2929-g82db1a42e9254c9009bbf8ac01366da4d1ab6df5
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Fri Aug 28 22:45:24 2020 +0100

    libstdc++: Fix std::gcd and std::lcm for unsigned integers [PR 92978]

    This fixes a bug with mixed signed and unsigned types, where converting
    a negative value to the unsigned result type alters the value. The
    solution is to obtain the absolute values of the arguments immediately
    and to perform the actual GCD or LCM algorithm on two arguments of the
    same type.

    In order to operate on the most negative number without overflow when
    taking its absolute, use an unsigned type for the result of the abs
    operation. For example, -INT_MIN will overflow, but -(unsigned)INT_MIN
    is (unsigned)INT_MAX+1U which is the correct value.

    libstdc++-v3/ChangeLog:

            PR libstdc++/92978
            * include/std/numeric (__abs_integral): Replace with ...
            (__detail::__absu): New function template that returns an
            unsigned type, guaranteeing it can represent the most
            negative signed value.
            (__detail::__gcd, __detail::__lcm): Require arguments to
            be unsigned and therefore already non-negative.
            (gcd, lcm): Convert arguments to absolute value as unsigned
            type before calling __detail::__gcd or __detail::__lcm.
            * include/experimental/numeric (gcd, lcm): Likewise.
            * testsuite/26_numerics/gcd/gcd_neg.cc: Adjust expected
            errors.
            * testsuite/26_numerics/lcm/lcm_neg.cc: Likewise.
            * testsuite/26_numerics/gcd/92978.cc: New test.
            * testsuite/26_numerics/lcm/92978.cc: New test.
            * testsuite/experimental/numeric/92978.cc: New test.

[Bug libstdc++/92978] std::gcd mishandles mixed-signedness

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92978

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |9.4

--- Comment #2 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Fixed on trunk so far.

[Bug libstdc++/92978] std::gcd mishandles mixed-signedness

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92978

--- Comment #3 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jonathan Wakely <redi@gcc.gnu.org>:

https://gcc.gnu.org/g:c71644776f4e8477289a4de16239dbb420db6945

commit r11-2983-gc71644776f4e8477289a4de16239dbb420db6945
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Wed Sep 2 17:20:37 2020 +0100

    libstdc++: Fix test to use correct function

    This was copied from a test for std::lcm but I forgot to change one of
    the calls to use the experimental version of the function.

    libstdc++-v3/ChangeLog:

            PR libstdc++/92978
            * testsuite/experimental/numeric/92978.cc: Use experimental::lcm
            not std::lcm.

[Bug libstdc++/92978] std::gcd mishandles mixed-signedness

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92978

--- Comment #4 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-10 branch has been updated by Jonathan Wakely
<redi@gcc.gnu.org>:

https://gcc.gnu.org/g:31782bd45331ef3006f624d7b1cc9cd11b4abb84

commit r10-8703-g31782bd45331ef3006f624d7b1cc9cd11b4abb84
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Fri Aug 28 22:45:24 2020 +0100

    libstdc++: Fix std::gcd and std::lcm for unsigned integers [PR 92978]

    This fixes a bug with mixed signed and unsigned types, where converting
    a negative value to the unsigned result type alters the value. The
    solution is to obtain the absolute values of the arguments immediately
    and to perform the actual GCD or LCM algorithm on two arguments of the
    same type.

    In order to operate on the most negative number without overflow when
    taking its absolute, use an unsigned type for the result of the abs
    operation. For example, -INT_MIN will overflow, but -(unsigned)INT_MIN
    is (unsigned)INT_MAX+1U which is the correct value.

    libstdc++-v3/ChangeLog:

            PR libstdc++/92978
            * include/std/numeric (__abs_integral): Replace with ...
            (__detail::__absu): New function template that returns an
            unsigned type, guaranteeing it can represent the most
            negative signed value.
            (__detail::__gcd, __detail::__lcm): Require arguments to
            be unsigned and therefore already non-negative.
            (gcd, lcm): Convert arguments to absolute value as unsigned
            type before calling __detail::__gcd or __detail::__lcm.
            * include/experimental/numeric (gcd, lcm): Likewise.
            * testsuite/26_numerics/gcd/gcd_neg.cc: Adjust expected
            errors.
            * testsuite/26_numerics/lcm/lcm_neg.cc: Likewise.
            * testsuite/26_numerics/gcd/92978.cc: New test.
            * testsuite/26_numerics/lcm/92978.cc: New test.
            * testsuite/experimental/numeric/92978.cc: New test.

    (cherry picked from commit 82db1a42e9254c9009bbf8ac01366da4d1ab6df5)

[Bug libstdc++/92978] std::gcd mishandles mixed-signedness

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92978

--- Comment #5 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-9 branch has been updated by Jonathan Wakely
<redi@gcc.gnu.org>:

https://gcc.gnu.org/g:b3043e490896ea37cd0273e6e149c3eeb3298720

commit r9-8928-gb3043e490896ea37cd0273e6e149c3eeb3298720
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Fri Aug 28 22:45:24 2020 +0100

    libstdc++: Fix std::gcd and std::lcm for unsigned integers [PR 92978]

    This fixes a bug with mixed signed and unsigned types, where converting
    a negative value to the unsigned result type alters the value. The
    solution is to obtain the absolute values of the arguments immediately
    and to perform the actual GCD or LCM algorithm on two arguments of the
    same type.

    In order to operate on the most negative number without overflow when
    taking its absolute, use an unsigned type for the result of the abs
    operation. For example, -INT_MIN will overflow, but -(unsigned)INT_MIN
    is (unsigned)INT_MAX+1U which is the correct value.

    libstdc++-v3/ChangeLog:

            PR libstdc++/92978
            * include/std/numeric (__abs_integral): Replace with ...
            (__detail::__absu): New function template that returns an
            unsigned type, guaranteeing it can represent the most
            negative signed value.
            (__detail::__gcd, __detail::__lcm): Require arguments to
            be unsigned and therefore already non-negative.
            (gcd, lcm): Convert arguments to absolute value as unsigned
            type before calling __detail::__gcd or __detail::__lcm.
            * include/experimental/numeric (gcd, lcm): Likewise.
            * testsuite/26_numerics/gcd/gcd_neg.cc: Adjust expected
            errors.
            * testsuite/26_numerics/lcm/lcm_neg.cc: Likewise.
            * testsuite/26_numerics/gcd/92978.cc: New test.
            * testsuite/26_numerics/lcm/92978.cc: New test.
            * testsuite/experimental/numeric/92978.cc: New test.

    (cherry picked from commit 82db1a42e9254c9009bbf8ac01366da4d1ab6df5)

[Bug libstdc++/92978] std::gcd mishandles mixed-signedness

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92978

--- Comment #6 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-8 branch has been updated by Jonathan Wakely
<redi@gcc.gnu.org>:

https://gcc.gnu.org/g:7dbed9dead9002ee0cd4aa9c22b20942c6f13757

commit r8-10624-g7dbed9dead9002ee0cd4aa9c22b20942c6f13757
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Fri Aug 28 22:45:24 2020 +0100

    libstdc++: Fix std::gcd and std::lcm for unsigned integers [PR 92978]

    This fixes a bug with mixed signed and unsigned types, where converting
    a negative value to the unsigned result type alters the value. The
    solution is to obtain the absolute values of the arguments immediately
    and to perform the actual GCD or LCM algorithm on two arguments of the
    same type.

    In order to operate on the most negative number without overflow when
    taking its absolute, use an unsigned type for the result of the abs
    operation. For example, -INT_MIN will overflow, but -(unsigned)INT_MIN
    is (unsigned)INT_MAX+1U which is the correct value.

    libstdc++-v3/ChangeLog:

            PR libstdc++/92978
            * include/std/numeric (__abs_integral): Replace with ...
            (__detail::__absu): New function template that returns an
            unsigned type, guaranteeing it can represent the most
            negative signed value.
            (__detail::__gcd, __detail::__lcm): Require arguments to
            be unsigned and therefore already non-negative.
            (gcd, lcm): Convert arguments to absolute value as unsigned
            type before calling __detail::__gcd or __detail::__lcm.
            * include/experimental/numeric (gcd, lcm): Likewise.
            * testsuite/26_numerics/gcd/gcd_neg.cc: Adjust expected
            errors.
            * testsuite/26_numerics/lcm/lcm_neg.cc: Likewise.
            * testsuite/26_numerics/gcd/92978.cc: New test.
            * testsuite/26_numerics/lcm/92978.cc: New test.
            * testsuite/experimental/numeric/92978.cc: New test.

    (cherry picked from commit 82db1a42e9254c9009bbf8ac01366da4d1ab6df5)

[Bug libstdc++/92978] std::gcd mishandles mixed-signedness

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92978

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|ASSIGNED                    |RESOLVED
   Target Milestone|9.4                         |8.5

--- Comment #7 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Fixed for 8.4, 9.4, 10.3 and 11.

[Bug libstdc++/92978] std::gcd mishandles mixed-signedness

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92978

--- Comment #8 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jonathan Wakely <redi@gcc.gnu.org>:

https://gcc.gnu.org/g:671970a5621e18e7079b4ca113e56434c858db66

commit r13-1040-g671970a5621e18e7079b4ca113e56434c858db66
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Fri Jun 10 14:39:13 2022 +0100

    libstdc++: Make std::lcm and std::gcd detect overflow [PR105844]

    When I fixed PR libstdc++/92978 I introduced a regression whereby
    std::lcm(INT_MIN, 1) and std::lcm(50000, 49999) would no longer produce
    errors during constant evaluation. Those calls are undefined, because
    they violate the preconditions that |m| and the result can be
    represented in the return type (which is int in both those cases). The
    regression occurred because __absu<unsigned>(INT_MIN) is well-formed,
    due to the explicit casts to unsigned in that new helper function, and
    the out-of-range multiplication is well-formed, because unsigned
    arithmetic wraps instead of overflowing.

    To fix 92978 I made std::gcm and std::lcm calculate |m| and |n|
    immediately, yielding a common unsigned type that was used to calculate
    the result. That was partly correct, but there's no need to use an
    unsigned type. Doing so only suppresses the overflow errors so the
    compiler can't detect them. This change replaces __absu with __abs_r
    that returns the common type (not its corresponding unsigned type). This
    way we can detect overflow in __abs_r when required, while still
    supporting the most-negative value when it can be represented in the
    result type. To detect LCM results that are out of range of the result
    type we still need explicit checks, because neither constant evaluation
    nor UBsan will complain about unsigned wrapping for cases such as
    std::lcm(500000u, 499999u). We can detect those overflows efficiently by
    using __builtin_mul_overflow and asserting.

    libstdc++-v3/ChangeLog:

            PR libstdc++/105844
            * include/experimental/numeric (experimental::gcd): Simplify
            assertions. Use __abs_r instead of __absu.
            (experimental::lcm): Likewise. Remove use of __detail::__lcm so
            overflow can be detected.
            * include/std/numeric (__detail::__absu): Rename to __abs_r and
            change to allow signed result type, so overflow can be detected.
            (__detail::__lcm): Remove.
            (gcd): Simplify assertions. Use __abs_r instead of __absu.
            (lcm): Likewise. Remove use of __detail::__lcm so overflow can
            be detected.
            * testsuite/26_numerics/gcd/gcd_neg.cc: Adjust dg-error lines.
            * testsuite/26_numerics/lcm/lcm_neg.cc: Likewise.
            * testsuite/26_numerics/gcd/105844.cc: New test.
            * testsuite/26_numerics/lcm/105844.cc: New test.

[Bug libstdc++/92978] std::gcd mishandles mixed-signedness

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92978

--- Comment #9 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-12 branch has been updated by Jonathan Wakely
<redi@gcc.gnu.org>:

https://gcc.gnu.org/g:8a57deb926cd660c2eae7ed621d61a301ae0d523

commit r12-8654-g8a57deb926cd660c2eae7ed621d61a301ae0d523
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Fri Jun 10 14:39:13 2022 +0100

    libstdc++: Make std::lcm and std::gcd detect overflow [PR105844]

    When I fixed PR libstdc++/92978 I introduced a regression whereby
    std::lcm(INT_MIN, 1) and std::lcm(50000, 49999) would no longer produce
    errors during constant evaluation. Those calls are undefined, because
    they violate the preconditions that |m| and the result can be
    represented in the return type (which is int in both those cases). The
    regression occurred because __absu<unsigned>(INT_MIN) is well-formed,
    due to the explicit casts to unsigned in that new helper function, and
    the out-of-range multiplication is well-formed, because unsigned
    arithmetic wraps instead of overflowing.

    To fix 92978 I made std::gcm and std::lcm calculate |m| and |n|
    immediately, yielding a common unsigned type that was used to calculate
    the result. That was partly correct, but there's no need to use an
    unsigned type. Doing so only suppresses the overflow errors so the
    compiler can't detect them. This change replaces __absu with __abs_r
    that returns the common type (not its corresponding unsigned type). This
    way we can detect overflow in __abs_r when required, while still
    supporting the most-negative value when it can be represented in the
    result type. To detect LCM results that are out of range of the result
    type we still need explicit checks, because neither constant evaluation
    nor UBsan will complain about unsigned wrapping for cases such as
    std::lcm(500000u, 499999u). We can detect those overflows efficiently by
    using __builtin_mul_overflow and asserting.

    libstdc++-v3/ChangeLog:

            PR libstdc++/105844
            * include/experimental/numeric (experimental::gcd): Simplify
            assertions. Use __abs_r instead of __absu.
            (experimental::lcm): Likewise. Remove use of __detail::__lcm so
            overflow can be detected.
            * include/std/numeric (__detail::__absu): Rename to __abs_r and
            change to allow signed result type, so overflow can be detected.
            (__detail::__lcm): Remove.
            (gcd): Simplify assertions. Use __abs_r instead of __absu.
            (lcm): Likewise. Remove use of __detail::__lcm so overflow can
            be detected.
            * testsuite/26_numerics/gcd/gcd_neg.cc: Adjust dg-error lines.
            * testsuite/26_numerics/lcm/lcm_neg.cc: Likewise.
            * testsuite/26_numerics/gcd/105844.cc: New test.
            * testsuite/26_numerics/lcm/105844.cc: New test.

    (cherry picked from commit 671970a5621e18e7079b4ca113e56434c858db66)

[Bug libstdc++/92978] std::gcd mishandles mixed-signedness

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92978

--- Comment #10 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Jonathan Wakely
<redi@gcc.gnu.org>:

https://gcc.gnu.org/g:7c31a9c66269805d57591549edb031f84d84f9b0

commit r11-10739-g7c31a9c66269805d57591549edb031f84d84f9b0
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Fri Jun 10 14:39:13 2022 +0100

    libstdc++: Make std::lcm and std::gcd detect overflow [PR105844]

    When I fixed PR libstdc++/92978 I introduced a regression whereby
    std::lcm(INT_MIN, 1) and std::lcm(50000, 49999) would no longer produce
    errors during constant evaluation. Those calls are undefined, because
    they violate the preconditions that |m| and the result can be
    represented in the return type (which is int in both those cases). The
    regression occurred because __absu<unsigned>(INT_MIN) is well-formed,
    due to the explicit casts to unsigned in that new helper function, and
    the out-of-range multiplication is well-formed, because unsigned
    arithmetic wraps instead of overflowing.

    To fix 92978 I made std::gcm and std::lcm calculate |m| and |n|
    immediately, yielding a common unsigned type that was used to calculate
    the result. That was partly correct, but there's no need to use an
    unsigned type. Doing so only suppresses the overflow errors so the
    compiler can't detect them. This change replaces __absu with __abs_r
    that returns the common type (not its corresponding unsigned type). This
    way we can detect overflow in __abs_r when required, while still
    supporting the most-negative value when it can be represented in the
    result type. To detect LCM results that are out of range of the result
    type we still need explicit checks, because neither constant evaluation
    nor UBsan will complain about unsigned wrapping for cases such as
    std::lcm(500000u, 499999u). We can detect those overflows efficiently by
    using __builtin_mul_overflow and asserting.

    libstdc++-v3/ChangeLog:

            PR libstdc++/105844
            * include/experimental/numeric (experimental::gcd): Simplify
            assertions. Use __abs_r instead of __absu.
            (experimental::lcm): Likewise. Remove use of __detail::__lcm so
            overflow can be detected.
            * include/std/numeric (__detail::__absu): Rename to __abs_r and
            change to allow signed result type, so overflow can be detected.
            (__detail::__lcm): Remove.
            (gcd): Simplify assertions. Use __abs_r instead of __absu.
            (lcm): Likewise. Remove use of __detail::__lcm so overflow can
            be detected.
            * testsuite/26_numerics/gcd/gcd_neg.cc: Adjust dg-error lines.
            * testsuite/26_numerics/lcm/lcm_neg.cc: Likewise.
            * testsuite/26_numerics/gcd/105844.cc: New test.
            * testsuite/26_numerics/lcm/105844.cc: New test.

    (cherry picked from commit 671970a5621e18e7079b4ca113e56434c858db66)

[Bug libstdc++/92978] std::gcd mishandles mixed-signedness

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92978

--- Comment #11 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-10 branch has been updated by Jonathan Wakely
<redi@gcc.gnu.org>:

https://gcc.gnu.org/g:14175f2262c94868e26f01eef1e14418f2b5e6a9

commit r10-11388-g14175f2262c94868e26f01eef1e14418f2b5e6a9
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Fri Jun 10 14:39:13 2022 +0100

    libstdc++: Make std::lcm and std::gcd detect overflow [PR105844]

    When I fixed PR libstdc++/92978 I introduced a regression whereby
    std::lcm(INT_MIN, 1) and std::lcm(50000, 49999) would no longer produce
    errors during constant evaluation. Those calls are undefined, because
    they violate the preconditions that |m| and the result can be
    represented in the return type (which is int in both those cases). The
    regression occurred because __absu<unsigned>(INT_MIN) is well-formed,
    due to the explicit casts to unsigned in that new helper function, and
    the out-of-range multiplication is well-formed, because unsigned
    arithmetic wraps instead of overflowing.

    To fix 92978 I made std::gcm and std::lcm calculate |m| and |n|
    immediately, yielding a common unsigned type that was used to calculate
    the result. That was partly correct, but there's no need to use an
    unsigned type. Doing so only suppresses the overflow errors so the
    compiler can't detect them. This change replaces __absu with __abs_r
    that returns the common type (not its corresponding unsigned type). This
    way we can detect overflow in __abs_r when required, while still
    supporting the most-negative value when it can be represented in the
    result type. To detect LCM results that are out of range of the result
    type we still need explicit checks, because neither constant evaluation
    nor UBsan will complain about unsigned wrapping for cases such as
    std::lcm(500000u, 499999u). We can detect those overflows efficiently by
    using __builtin_mul_overflow and asserting.

    libstdc++-v3/ChangeLog:

            PR libstdc++/105844
            * include/experimental/numeric (experimental::gcd): Simplify
            assertions. Use __abs_r instead of __absu.
            (experimental::lcm): Likewise. Remove use of __detail::__lcm so
            overflow can be detected.
            * include/std/numeric (__detail::__absu): Rename to __abs_r and
            change to allow signed result type, so overflow can be detected.
            (__detail::__lcm): Remove.
            (gcd): Simplify assertions. Use __abs_r instead of __absu.
            (lcm): Likewise. Remove use of __detail::__lcm so overflow can
            be detected.
            * testsuite/26_numerics/gcd/gcd_neg.cc: Adjust dg-error lines.
            * testsuite/26_numerics/lcm/lcm_neg.cc: Likewise.
            * testsuite/26_numerics/gcd/105844.cc: New test.
            * testsuite/26_numerics/lcm/105844.cc: New test.

    (cherry picked from commit 671970a5621e18e7079b4ca113e56434c858db66)