[Bug target/113175] New: [14 Regression] MMIX: testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[Bug target/113175] New: [14 Regression] MMIX: testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[hp at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

            Bug ID: 113175
           Summary: [14 Regression] MMIX:
                    testsuite/std/ranges/iota/max_size_type.cc 5x times
                    slower
           Product: gcc
           Version: 14.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: hp at gcc dot gnu.org
  Target Milestone: ---
              Host: x86_64-linux-gnu
            Target: mmix-knuth-mmixware

(Perhaps a "[13 Regression]" as well - haven't checked.)

I ran the testsuite for mmix-knuth-mmixware (a 64-bit newlib cross) and found
that testsuite/std/ranges/iota/max_size_type.cc times out, despite the
timeout-factor increased by a factor 4 (from 360 seconds to 1440 seconds) in
r12-2799-ge9b639c4b53221.

For a tree at r12-2797-g307e0d40367996 running this test takes 11:43.61 (mm:ss)
wall-clock time and now at r14-6859-gd1eacedc6d9b it takes 58:45.78 (on one and
the same host and same session within the same hour; no host-side changes and
the test itself apparently hasn't changed significantly).

I haven't checked whether this affects other targets, but if so, a 5x
performance regression should have been picked up by other testers, thus I'm
starting with "target" until firther investigated.

[Bug libstdc++/113175] [14 Regression] testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[hp at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

Hans-Peter Nilsson <hp at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Target|mmix-knuth-mmixware         |mmix-knuth-mmixware,
                   |                            |x86_64-linux-gnu
            Summary|[14 Regression] MMIX:       |[14 Regression]
                   |testsuite/std/ranges/iota/m |testsuite/std/ranges/iota/m
                   |ax_size_type.cc 5x times    |ax_size_type.cc 5x times
                   |slower                      |slower
          Component|target                      |libstdc++

--- Comment #1 from Hans-Peter Nilsson <hp at gcc dot gnu.org> ---
I'm afraid this isn't target-specific.
I bootstrapped native x86_64-linux-gnu off the both source trees, and ran \time
env LD_LIBRARY_PATH=(long curse) ./max_size_type.exe

r12-2797-g307e0d40367996:
0.32user 0.00system 0:00.32elapsed 99%CPU (0avgtext+0avgdata 2620maxresident)k
0inputs+0outputs (0major+184minor)pagefaults 0swaps

r14-6859-gd1eacedc6d9b:
1.73user 0.00system 0:01.73elapsed 100%CPU (0avgtext+0avgdata 3564maxresident)k
0inputs+0outputs (0major+194minor)pagefaults 0swaps

...which is a factor of about 5.4, consistent with the MMIX observation.
Still unknown whether libstdc++ or code-generator of course, but I'm leaning
towards the latter, for the same reasons as in the precious comment.  Maybe the
test-case triggers peculiar details.

[Bug libstdc++/113175] [14 Regression] testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[hp at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

--- Comment #2 from Hans-Peter Nilsson <hp at gcc dot gnu.org> ---
Bisecting (native) has progressed beyond the r13 mark, i.e. this is indeed a
"[14 Regression]" only.

[Bug testsuite/113175] [14 Regression] testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[hp at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

Hans-Peter Nilsson <hp at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|libstdc++                   |testsuite

--- Comment #3 from Hans-Peter Nilsson <hp at gcc dot gnu.org> ---
There's one single regression event, bringing the host runtime to about 1.63
seconds.  Then some time later, an additional 0.1 second was added
(accumulated).
I did not look into that latter regression.  The big one is clouded by a large
range of commits where max_size_type failed, due to r14-159-g03cebd304955a6.
This was fixed in r14-205-g83470a5cd4c3d2, at which time there the big
regression is seen for the first time.  That is also the "cause" for the
commit, because applying that commit to r14-158-g7d115e01411156 shows the same
number as for r14-205-g83470a5cd4c3d2.

Actually, it's the testsuite part of that patch, because with that reverted the
execution time backs down to 0.33 seconds.  IOW, this while PR is /testsuites. 
Not sure what to do to improve the execution time, as plain disabling the using
"signed_rep_t = __int128;" by making the first line "+#if 0 &&
__SIZEOF_INT128__" yields
/x/testsuite/std/ranges/iota/max_size_type.cc:36: note: the comparison reduces
to '(16 == 8)' 

Maybe the higher number for the execution time is actually the "right" one and
the range should be cut down to -100..100 for *all* targets?

HNY!

[Bug testsuite/113175] [14 Regression] testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

Xi Ruoyao <xry111 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |xry111 at gcc dot gnu.org

--- Comment #4 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
(In reply to Hans-Peter Nilsson from comment #3)
> There's one single regression event, bringing the host runtime to about 1.63
> seconds.  Then some time later, an additional 0.1 second was added
> (accumulated).
> I did not look into that latter regression.  The big one is clouded by a
> large range of commits where max_size_type failed, due to
> r14-159-g03cebd304955a6.
> This was fixed in r14-205-g83470a5cd4c3d2, at which time there the big
> regression is seen for the first time.  That is also the "cause" for the
> commit, because applying that commit to r14-158-g7d115e01411156 shows the
> same number as for r14-205-g83470a5cd4c3d2.
> 
> Actually, it's the testsuite part of that patch, because with that reverted
> the execution time backs down to 0.33 seconds.  IOW, this while PR is
> /testsuites.  Not sure what to do to improve the execution time, as plain
> disabling the using "signed_rep_t = __int128;" by making the first line
> "+#if 0 && __SIZEOF_INT128__" yields
> /x/testsuite/std/ranges/iota/max_size_type.cc:36: note: the comparison
> reduces to '(16 == 8)' 
> 
> Maybe the higher number for the execution time is actually the "right" one
> and the range should be cut down to -100..100 for *all* targets?
> 
> HNY!

IIRC the "signed_rep_t = __int128;" case has really detected a compiler bug, so
IMO we shouldn't just disable it.

Maybe my memory is flawed though.

[Bug testsuite/113175] [14 Regression] testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[hp at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

--- Comment #5 from Hans-Peter Nilsson <hp at gcc dot gnu.org> ---
(In reply to Xi Ruoyao from comment #4)
> IIRC the "signed_rep_t = __int128;" case has really detected a compiler bug,
> so IMO we shouldn't just disable it.

Maybe I should have been explicit: that was just for investigation purposes.

> Maybe my memory is flawed though.

Please link that PR here if you have it!

[Bug testsuite/113175] [14 Regression] testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://gcc.gnu.org/bugzill
                   |                            |a/show_bug.cgi?id=108099

--- Comment #6 from Jonathan Wakely <redi at gcc dot gnu.org> ---
PR108099 I think

[Bug testsuite/113175] [14 Regression] testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

--- Comment #7 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Xi Ruoyao from comment #4)
> IIRC the "signed_rep_t = __int128;" case has really detected a compiler bug,
> so IMO we shouldn't just disable it.
> 
> Maybe my memory is flawed though.

I think it was the opposite. Using `signed rep_t` relied on a compiler bug.
When that bug was fixed we had to introduce a typedef for that type. I have no
idea why it would make the test slower though.

[Bug testsuite/113175] [11/12/13/14 Regression] testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[ppalka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

Patrick Palka <ppalka at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |ppalka at gcc dot gnu.org
     Ever confirmed|0                           |1
             Status|UNCONFIRMED                 |ASSIGNED
            Summary|[14 Regression]             |[11/12/13/14 Regression]
                   |testsuite/std/ranges/iota/m |testsuite/std/ranges/iota/m
                   |ax_size_type.cc 5x times    |ax_size_type.cc 5x times
                   |slower                      |slower
   Target Milestone|---                         |11.5
   Last reconfirmed|                            |2024-01-02

--- Comment #8 from Patrick Palka <ppalka at gcc dot gnu.org> ---
(In reply to Hans-Peter Nilsson from comment #3)
> There's one single regression event, bringing the host runtime to about 1.63
> seconds.  Then some time later, an additional 0.1 second was added
> (accumulated).
> I did not look into that latter regression.  The big one is clouded by a
> large range of commits where max_size_type failed, due to
> r14-159-g03cebd304955a6.
> This was fixed in r14-205-g83470a5cd4c3d2, at which time there the big
> regression is seen for the first time.  That is also the "cause" for the
> commit, because applying that commit to r14-158-g7d115e01411156 shows the
> same number as for r14-205-g83470a5cd4c3d2.

Interesting, thanks for bisecting this.  Before r14-205, the two main loops in
max_size_type.cc were effectively dead in the signed case because the 'signed'
qualifier in

  using hw_type = std::conditional_t<signed_p, signed rep_t, rep_t>;

was being silently dropped (until r14-159, after which it was correctly
rejected).  This meant that when signed_p is true, the for loop

  for (hw_type i = min; i <= max; i++)

was equivalent to

  for (unsigned __int128 i = -1000; i <= 1000; i++)

so the loop body was never run!  When r14-205 fixed that, the total number of
loop iterations increased by approximately 4x (2x due to symmetry with
unsigned, 2x due to the loop range being -limit...limit instead of 0...limit)
which is roughly consistent with a 5x increase in test execution time.

> Maybe the higher number for the execution time is actually the "right" one and the range > should be cut down to -100..100 for *all* targets?

That sounds good to me..  I can prepare a simple patch for that, which probably
should be backported to all release branches that r14-205 was backported to.

[Bug testsuite/113175] [11/12/13/14 Regression] testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[ppalka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

--- Comment #9 from Patrick Palka <ppalka at gcc dot gnu.org> ---
> When r14-205 fixed that, the total number of loop iterations increased by approximately 4x (2x due to symmetry with unsigned, 2x due to the loop range being -limit...limit instead of 0...limit) which is roughly consistent with a 5x increase in test execution time.

Oops, the increase is more like 5x due to loop nesting.  So the signed_p case
performs 4x more iterations than the unsigned case, and previously the signed_p
case was dead.

[Bug testsuite/113175] [11/12/13/14 Regression] testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

--- Comment #10 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Patrick Palka <ppalka@gcc.gnu.org>:

https://gcc.gnu.org/g:a138b99646a5551c53b860648521adb5bfe8c2fa

commit r14-6888-ga138b99646a5551c53b860648521adb5bfe8c2fa
Author: Patrick Palka <ppalka@redhat.com>
Date:   Tue Jan 2 21:31:20 2024 -0500

    libstdc++: testsuite: Reduce max_size_type.cc exec time [PR113175]

    The adjustment to max_size_type.cc in r14-205-g83470a5cd4c3d2
    inadvertently increased the execution time of this test by over 5x due
    to making the two main loops actually run in the signed_p case instead
    of being dead code.

    To compensate, this patch cuts the relevant loops' range [-1000,1000] by
    10x as proposed in the PR.  This shouldn't significantly weaken the test
    since the same important edge cases are still checked in the smaller range
    and/or elsewhere.  On my machine this reduces the test's execution time by
    roughly 10x (and 1.6x relative to before r14-205).

            PR testsuite/113175

    libstdc++-v3/ChangeLog:

            * testsuite/std/ranges/iota/max_size_type.cc (test02): Reduce
            'limit' to 100 from 1000 and adjust 'log2_limit' accordingly.
            (test03): Likewise.

[Bug testsuite/113175] [11/12/13/14 Regression] testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

--- Comment #11 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-13 branch has been updated by Patrick Palka
<ppalka@gcc.gnu.org>:

https://gcc.gnu.org/g:af767e0b9e926fa1ef217087ce3b076be361965f

commit r13-8189-gaf767e0b9e926fa1ef217087ce3b076be361965f
Author: Patrick Palka <ppalka@redhat.com>
Date:   Tue Jan 2 21:31:20 2024 -0500

    libstdc++: testsuite: Reduce max_size_type.cc exec time [PR113175]

    The adjustment to max_size_type.cc in r14-205-g83470a5cd4c3d2
    inadvertently increased the execution time of this test by over 5x due
    to making the two main loops actually run in the signed_p case instead
    of being dead code.

    To compensate, this patch cuts the relevant loops' range [-1000,1000] by
    10x as proposed in the PR.  This shouldn't significantly weaken the test
    since the same important edge cases are still checked in the smaller range
    and/or elsewhere.  On my machine this reduces the test's execution time by
    roughly 10x (and 1.6x relative to before r14-205).

            PR testsuite/113175

    libstdc++-v3/ChangeLog:

            * testsuite/std/ranges/iota/max_size_type.cc (test02): Reduce
            'limit' to 100 from 1000 and adjust 'log2_limit' accordingly.
            (test03): Likewise.

    (cherry picked from commit a138b99646a5551c53b860648521adb5bfe8c2fa)

[Bug testsuite/113175] [11/12/13/14 Regression] testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

--- Comment #12 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-12 branch has been updated by Patrick Palka
<ppalka@gcc.gnu.org>:

https://gcc.gnu.org/g:1fb2b5c9a7fb212a5936cc702e49175ef988e5ad

commit r12-10079-g1fb2b5c9a7fb212a5936cc702e49175ef988e5ad
Author: Patrick Palka <ppalka@redhat.com>
Date:   Tue Jan 2 21:31:20 2024 -0500

    libstdc++: testsuite: Reduce max_size_type.cc exec time [PR113175]

    The adjustment to max_size_type.cc in r14-205-g83470a5cd4c3d2
    inadvertently increased the execution time of this test by over 5x due
    to making the two main loops actually run in the signed_p case instead
    of being dead code.

    To compensate, this patch cuts the relevant loops' range [-1000,1000] by
    10x as proposed in the PR.  This shouldn't significantly weaken the test
    since the same important edge cases are still checked in the smaller range
    and/or elsewhere.  On my machine this reduces the test's execution time by
    roughly 10x (and 1.6x relative to before r14-205).

            PR testsuite/113175

    libstdc++-v3/ChangeLog:

            * testsuite/std/ranges/iota/max_size_type.cc (test02): Reduce
            'limit' to 100 from 1000 and adjust 'log2_limit' accordingly.
            (test03): Likewise.

    (cherry picked from commit a138b99646a5551c53b860648521adb5bfe8c2fa)

[Bug testsuite/113175] [11/12/13/14 Regression] testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

--- Comment #13 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Patrick Palka
<ppalka@gcc.gnu.org>:

https://gcc.gnu.org/g:2110667d43486174dda37a95f73d71941b394655

commit r11-11179-g2110667d43486174dda37a95f73d71941b394655
Author: Patrick Palka <ppalka@redhat.com>
Date:   Tue Jan 2 21:31:20 2024 -0500

    libstdc++: testsuite: Reduce max_size_type.cc exec time [PR113175]

    The adjustment to max_size_type.cc in r14-205-g83470a5cd4c3d2
    inadvertently increased the execution time of this test by over 5x due
    to making the two main loops actually run in the signed_p case instead
    of being dead code.

    To compensate, this patch cuts the relevant loops' range [-1000,1000] by
    10x as proposed in the PR.  This shouldn't significantly weaken the test
    since the same important edge cases are still checked in the smaller range
    and/or elsewhere.  On my machine this reduces the test's execution time by
    roughly 10x (and 1.6x relative to before r14-205).

            PR testsuite/113175

    libstdc++-v3/ChangeLog:

            * testsuite/std/ranges/iota/max_size_type.cc (test02): Reduce
            'limit' to 100 from 1000 and adjust 'log2_limit' accordingly.
            (test03): Likewise.

    (cherry picked from commit a138b99646a5551c53b860648521adb5bfe8c2fa)

[Bug testsuite/113175] [11/12/13/14 Regression] testsuite/std/ranges/iota/max_size_type.cc 5x times slower

[ppalka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113175

Patrick Palka <ppalka at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #14 from Patrick Palka <ppalka at gcc dot gnu.org> ---
Fixed.