[Bug libstdc++/98678] New: 30_threads/future/members/poll.cc execution test FAILs

[Bug libstdc++/98678] New: 30_threads/future/members/poll.cc execution test FAILs

[ro at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

            Bug ID: 98678
           Summary: 30_threads/future/members/poll.cc execution test FAILs
           Product: gcc
           Version: 11.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libstdc++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: ro at gcc dot gnu.org
  Target Milestone: ---
            Target: i386-pc-solaris2.11, x86_64-pc-solaris2.11,
                    arm-none-linux-gnueabi, hppa-unknown-linux-gnu,
                    x86_64-pc-linux-gnu

The 30_threads/future/members/poll.cc reliably FAILs on 32 and 64-bit
Solaris/x86:

/vol/gcc/src/hg/master/local/libstdc++-v3/testsuite/30_threads/future/members/poll.cc:95:
int main(): Assertion 'wait_until_sys_min < (ready * 100)' failed.
wait_for(0s): 11892ns for 200 calls, avg 59.46ns per call
wait_until(system_clock minimum): 1304458ns for 200 calls, avg 6522.29ns per
call
wait_until(steady_clock minimum): 1403221ns for 200 calls, avg 7016.1ns per
call
wait_until(system_clock epoch): 3343806ns for 200 calls, avg 16719ns per call
wait_until(steady_clock epoch: 2959581ns for 200 calls, avg 14797.9ns per call
wait_for when ready: 10969ns for 200 calls, avg 54.845ns per call

While I don't see the failure on Solaris/SPARC, there are reports for a couple
of other targets on gcc-testresults.  However, the Linux/x86_64 ones seem to
only affect the gcc-10 branch.

[Bug libstdc++/98678] 30_threads/future/members/poll.cc execution test FAILs

[ro at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

Rainer Orth <ro at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |11.0

[Bug libstdc++/98678] 30_threads/future/members/poll.cc execution test FAILs

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2021-01-14
             Status|UNCONFIRMED                 |NEW

--- Comment #1 from Jonathan Wakely <redi at gcc dot gnu.org> ---
The relevant part of the std::future code on the gcc-10 branch is very
different, so the timings are different on the branch too.

This test is a bit tricky. The whole point is to check that performance of one
operation is acceptable compared to a baseline. But the definition of
"acceptable" and the relative difference between the speed of the different
operations varies with arch. We could just increase the tolerances, but then we
allow worse performance on the targets that don't need it. Maybe we want to
change the 30 and 100 magic numbers to depend on the target.

[Bug libstdc++/98678] 30_threads/future/members/poll.cc execution test FAILs

[seurer at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

seurer at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |seurer at gcc dot gnu.org
             Target|i386-pc-solaris2.11,        |i386-pc-solaris2.11,
                   |x86_64-pc-solaris2.11,      |x86_64-pc-solaris2.11,
                   |arm-none-linux-gnueabi,     |arm-none-linux-gnueabi,
                   |hppa-unknown-linux-gnu,     |hppa-unknown-linux-gnu,
                   |x86_64-pc-linux-gnu         |x86_64-pc-linux-gnu,
                   |                            |powerpc64-linux-gnu

--- Comment #2 from seurer at gcc dot gnu.org ---
FYI I am seeing this every so often on powerpc64 BE on an older power 7
machine, too.

[Bug libstdc++/98678] 30_threads/future/members/poll.cc execution test FAILs

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|11.0                        |11.2

--- Comment #3 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
GCC 11.1 has been released, retargeting bugs to GCC 11.2.

[Bug libstdc++/98678] 30_threads/future/members/poll.cc execution test FAILs

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|11.2                        |11.3

--- Comment #4 from Richard Biener <rguenth at gcc dot gnu.org> ---
GCC 11.2 is being released, retargeting bugs to GCC 11.3

[Bug libstdc++/98678] 30_threads/future/members/poll.cc execution test FAILs

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|11.3                        |11.4

--- Comment #5 from Richard Biener <rguenth at gcc dot gnu.org> ---
GCC 11.3 is being released, retargeting bugs to GCC 11.4.

[Bug libstdc++/98678] 30_threads/future/members/poll.cc execution test FAILs

[danglin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

John David Anglin <danglin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |danglin at gcc dot gnu.org

--- Comment #6 from John David Anglin <danglin at gcc dot gnu.org> ---
Fails consistently with timeout on hppa-unknown-linux-gnu. Increasing timeout
doesn't help.

dave@mx3210:~/gnu/gcc/objdir/hppa-linux-gnu/libstdc++-v3/testsuite$ time
./poll.exe
wait_for(0s): 10000098ns for 169420 calls, avg 59.0255ns per call
wait_until(system_clock minimum): 40000394ns for 169420 calls, avg 236.102ns
per call
wait_until(steady_clock minimum): 50000492ns for 169420 calls, avg 295.127ns
per call
wait_until(system_clock epoch): 1695216533549ns for 169420 calls, avg
1.0006e+07ns per call
wait_until(steady_clock epoch: 1694205876189ns for 169420 calls, avg 1e+07ns
per call
wait_for when ready: 0ns for 169420 calls, avg 0ns per call
/home/dave/gnu/gcc/gcc/libstdc++-v3/testsuite/30_threads/future/members/poll.cc:129:
int main(): Assertion 'wait_for_0 < (ready * 30)' failed.
Aborted (core dumped)

real    56m29.632s
user    0m5.355s
sys     0m8.722s

[Bug libstdc++/98678] 30_threads/future/members/poll.cc execution test FAILs

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|11.4                        |11.5

--- Comment #7 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
GCC 11.4 is being released, retargeting bugs to GCC 11.5.

[Bug libstdc++/98678] 30_threads/future/members/poll.cc execution test FAILs

[ro at CeBiTec dot Uni-Bielefeld.DE]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

--- Comment #8 from ro at CeBiTec dot Uni-Bielefeld.DE <ro at CeBiTec dot Uni-Bielefeld.DE> ---
> --- Comment #1 from Jonathan Wakely <redi at gcc dot gnu.org> ---

> This test is a bit tricky. The whole point is to check that performance of one
> operation is acceptable compared to a baseline. But the definition of
> "acceptable" and the relative difference between the speed of the different
> operations varies with arch. We could just increase the tolerances, but then we
> allow worse performance on the targets that don't need it. Maybe we want to
> change the 30 and 100 magic numbers to depend on the target.

I've made some more checks on Solaris now.  The test consistently PASSes
on Solaris/SPARC, both 32 and 64-bit.  However, on Solaris/x86 the
failure is just as reliable, e.g.

/vol/gcc/src/hg/master/local/libstdc++-v3/testsuite/30_threads/future/members/poll.cc:132:
int main(): Assertion 'wait_until_sys_min < (ready * 100)' failed.
wait_for(0s): 3674ns for 200 calls, avg 18.37ns per call
wait_until(system_clock minimum): 419918ns for 200 calls, avg 2099.59ns per
call
wait_until(steady_clock minimum): 459775ns for 200 calls, avg 2298.88ns per
call
wait_until(system_clock epoch): 1117280ns for 200 calls, avg 5586.4ns per call
wait_until(steady_clock epoch: 956073ns for 200 calls, avg 4780.36ns per call
wait_for when ready: 3194ns for 200 calls, avg 15.97ns per call

It also makes no difference if the system is under full load or
completely idle.  I've also checked a wider range of systems/CPUs:

  host                  32-bit          64-bit          

  nahe                  1.31            1.40            2.60 GHz Xeon Gold 6132
  lokon                 1.43            1.66            3.10 GHz Core i5-2400
  itzacchiuatl          0.69            1.53            3.20 GHz Core i7-8700
  manam                 0.89            2.22            3.50 GHz Xeon E3-1245
  lucy                  0.54            0.59            2.00 GHz Xeon E7-4850

The attached patch uses a scale factor of 2.5 to accomodate this.

[Bug libstdc++/98678] 30_threads/future/members/poll.cc execution test FAILs

[ro at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

--- Comment #9 from Rainer Orth <ro at gcc dot gnu.org> ---
Created attachment 58355
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=58355&action=edit
Proposed patch

[Bug libstdc++/98678] 30_threads/future/members/poll.cc execution test FAILs

[danglin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

--- Comment #10 from John David Anglin <danglin at gcc dot gnu.org> ---
On hppa-linux, we have:

dave@atlas:~/gnu/gcc/objdir/hppa-linux-gnu/libstdc++-v3/testsuite$ ./poll.exe
wait_for(0s): 3999963ns for 81820 calls, avg 48.8874ns per call
wait_until(system_clock minimum): 15999855ns for 81820 calls, avg 195.549ns per
call
wait_until(steady_clock minimum): 19999817ns for 81820 calls, avg 244.437ns per
call
wait_until(system_clock epoch): 327345016058ns for 81820 calls, avg
4.00079e+06ns per call
wait_until(steady_clock epoch: 327417023513ns for 81820 calls, avg
4.00167e+06ns per call
wait_for when ready: 0ns for 81820 calls, avg 0ns per call
/home/dave/gnu/gcc/gcc/libstdc++-v3/testsuite/30_threads/future/members/poll.cc:129:
int main(): Assertion 'wait_for_0 < (ready * 30)' failed.
Aborted (core dumped)

[Bug libstdc++/98678] 30_threads/future/members/poll.cc execution test FAILs

[danglin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

--- Comment #11 from John David Anglin <danglin at gcc dot gnu.org> ---
(In reply to John David Anglin from comment #10)
> On hppa-linux, we have:
> 
> dave@atlas:~/gnu/gcc/objdir/hppa-linux-gnu/libstdc++-v3/testsuite$ ./poll.exe
> wait_for(0s): 3999963ns for 81820 calls, avg 48.8874ns per call
> wait_until(system_clock minimum): 15999855ns for 81820 calls, avg 195.549ns
> per call
> wait_until(steady_clock minimum): 19999817ns for 81820 calls, avg 244.437ns
> per call
> wait_until(system_clock epoch): 327345016058ns for 81820 calls, avg
> 4.00079e+06ns per call
> wait_until(steady_clock epoch: 327417023513ns for 81820 calls, avg
> 4.00167e+06ns per call
> wait_for when ready: 0ns for 81820 calls, avg 0ns per call
> /home/dave/gnu/gcc/gcc/libstdc++-v3/testsuite/30_threads/future/members/poll.
> cc:129: int main(): Assertion 'wait_for_0 < (ready * 30)' failed.
> Aborted (core dumped)

HZ is 250.

[Bug libstdc++/98678] 30_threads/future/members/poll.cc execution test FAILs

[danglin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

--- Comment #12 from John David Anglin <danglin at gcc dot gnu.org> ---
Created attachment 58500
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=58500&action=edit
Hack to fix failure of poll.cc on hppa-unknown-linux-gnu

The number of iterations is specific to my current kernel configuration.  The
code to determine the number of iterations returned inconsistent results.

Current kernel config is 1000 Hz (to reduce test time) and stable clock.

Here is a typical run:
dave@atlas:~/gnu/gcc/objdir/hppa-linux-gnu/libstdc++-v3/testsuite$ ./poll.exe
wait_for(0s): 999992ns for 19500 calls, avg 51.2816ns per call
wait_until(system_clock minimum): 3999969ns for 19500 calls, avg 205.127ns per
call
wait_until(steady_clock minimum): 4999961ns for 19500 calls, avg 256.408ns per
call
wait_until(system_clock epoch): 19499847791ns for 19500 calls, avg 999992ns per
call
wait_until(steady_clock epoch: 19498848587ns for 19500 calls, avg 999941ns per
call
wait_for when ready: 999992ns for 19500 calls, avg 51.2816ns per call
missed 9 ticks

For some reason, I never see missed ticks in "wait_for(0s)".  But we almost
always miss ticks in "wait_for when ready" (usually 7 to 9).  I turned off
ntp but this didn't make any significant difference.  CLOCK_MONOTONIC behaves
similarly to CLOCK_REALTIME with regard to missing ticks..

dave@atlas:~/gnu/gcc/objdir/hppa-linux-gnu/libstdc++-v3/testsuite$ ./ntime
CLOCK_REALTIME - "Systemwide realtime clock.":
        precision: 1000000ns
        value    : -112725112.1719177549
CLOCK_MONOTONIC - "Represents monotonic time. Cannot be set.":
        precision: 1000000ns
        value    : -112725112.68986

Didn't need to adjust VERIFY conditions.

Test times get much longer at 250 and 100 HZ since clock tick depends on HZ.

[Bug libstdc++/98678] 30_threads/future/members/poll.cc execution test FAILs

[danglin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98678

John David Anglin <danglin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #58500|0                           |1
        is obsolete|                            |

--- Comment #13 from John David Anglin <danglin at gcc dot gnu.org> ---
Created attachment 58515
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=58515&action=edit
Really fix poll.cc on hppa-linux

hppa-linux doesn't have high resolution timer support.  Timer resolutions
range from 1 to 10ms depending on CONFIG_HZ.  The ready loop takes about
5ns per call.  As a result, we need a lot of iterations to obtain reasonable
accuracy in the ready loop.  I reduced the number of iteration in the
wait_until_sys_epoch and wait_until_steady_epoch loops to 1000 so test
would run in a reasonable time.  The results for these two loops are
currently not checked.

Here is output on 1GHz c8000 with HZ=250:
dave@atlas:~/gnu/gcc/objdir/hppa-linux-gnu/libstdc++-v3/testsuite$ ./poll.exe
wait_for(0s): 4919957362ns for 100000000 calls, avg 49.1996ns per call
wait_until(system_clock minimum): 22053828859ns for 100000000 calls, avg
220.538ns per call
wait_until(steady_clock minimum): 22277782883ns for 100000000 calls, avg
222.778ns per call
wait_until(system_clock epoch): 3999965336ns for 1000 calls, avg 3.99997e+06ns
per call
wait_until(steady_clock epoch: 3999965335ns for 1000 calls, avg 3.99997e+06ns
per call
wait_for when ready: 519995494ns for 100000000 calls, avg 5.19995ns per call