[PATCH] libstdc++-v3: Set 26_numerics/random/n.b.d./op./values.cc timeout-factor to 3

[PATCH] libstdc++-v3: Set 26_numerics/random/n.b.d./op./values.cc timeout-factor to 3

[Hans-Peter Nilsson]

Also, how about shortening those test-suite file-paths?
They're path sort-of overlong for any git (and ChangeLog)
commit-line limit. 1/2 :-)

Ok to commit (without renaming)?

--- 8< ---

I happened to run my cris-elf regression-tester and some
large job in parallel when this test was running.  According
to "time" it takes 4:39.07 for the GNU cris-elf simulator on
this yesteryear machine.  The default timeout is 360
seconds.  Let's "up" the time by a factor 3.

libstdc++-v3:
	* testsuite/26_numerics/random/negative_binomial_distribution/operators/values.cc:
	Set dg-timeout-factor to 3.
---
 .../random/negative_binomial_distribution/operators/values.cc    | 1 +
 1 file changed, 1 insertion(+)

diff --git a/libstdc++-v3/testsuite/26_numerics/random/negative_binomial_distribution/operators/values.cc b/libstdc++-v3/testsuite/26_numerics/random/negative_binomial_distribution/operators/values.cc
index dda6f43b2540..89a2b3dc4b76 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/negative_binomial_distribution/operators/values.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/negative_binomial_distribution/operators/values.cc
@@ -1,4 +1,5 @@
 // { dg-do run { target c++11 } }
+// { dg-timeout-factor 3 }
 // { dg-require-cstdint "" }
 // { dg-require-cmath "" }
 //
-- 
2.30.2

Re: [PATCH] libstdc++-v3: Set 26_numerics/random/n.b.d./op./values.cc timeout-factor to 3

[Jonathan Wakely]

[-- Attachment #1: Type: text/plain, Size: 1257 bytes --]

On Fri, 20 May 2022 at 03:20, Hans-Peter Nilsson via Libstdc++
<libstdc++@gcc.gnu.org> wrote:
>
> Also, how about shortening those test-suite file-paths?
> They're path sort-of overlong for any git (and ChangeLog)
> commit-line limit. 1/2 :-)

Yes, they're silly. I like what libc++ does, i.e. name the testsuite
directories after the [stable.name] tags in the standard. They have
std/rand/rand.dis/rand.dist.bern/rand.dist.bern.negbin (for some
reason they use "rand.dis" instead of "rand.dist" for the third
component there, which is not what the standard has,
https://eel.is/c++draft/rand.dist.bern.negbin shows the subclause
hierarchy).

That's still quite unwieldy, so we could remove the repetition and use
26_numerics/rand/dist/bernoulli/negbin/. The problem now is that it
gets quite cryptic, so finding the tests for the
negative_binomial_distribution is a bit harder if you don't know where
to look (but git grep solves that in less than a second).



>
> Ok to commit (without renaming)?

I'm OK with the timeout factor, but we could also solve it differently
so that it doesn't take nearly 5 minutes, as in the attached patch.
The testDiscreteDist function can be parametrized with the number of
iterations to perform. Would you rather do that?

[-- Attachment #2: patch.txt --]
[-- Type: text/plain, Size: 1608 bytes --]

diff --git a/libstdc++-v3/testsuite/26_numerics/random/negative_binomial_distribution/operators/values.cc b/libstdc++-v3/testsuite/26_numerics/random/negative_binomial_distribution/operators/values.cc
index dda6f43b254..a3120401d09 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/negative_binomial_distribution/operators/values.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/negative_binomial_distribution/operators/values.cc
@@ -26,6 +26,14 @@
 #include <functional>
 #include <testsuite_random.h>
 
+// { dg-options "-DSIMULATOR_TEST" { target simulator } }
+
+#ifdef SIMULATOR_TEST
+# define ARGS 100, 1000
+#else
+# define ARGS
+#endif
+
 void test01()
 {
   using namespace __gnu_test;
@@ -34,18 +42,18 @@ void test01()
 
   std::negative_binomial_distribution<> nbd1(5, 0.3);
   auto bnbd1 = std::bind(nbd1, eng);
-  testDiscreteDist(bnbd1, [](int n)
-		   { return negative_binomial_pdf(n, 5, 0.3); } );
+  testDiscreteDist<ARGS>(bnbd1, [](int n)
+			 { return negative_binomial_pdf(n, 5, 0.3); } );
 
   std::negative_binomial_distribution<> nbd2(55, 0.3);
   auto bnbd2 = std::bind(nbd2, eng);
-  testDiscreteDist(bnbd2, [](int n)
-		   { return negative_binomial_pdf(n, 55, 0.3); } );
+  testDiscreteDist<ARGS>(bnbd2, [](int n)
+			 { return negative_binomial_pdf(n, 55, 0.3); } );
 
   std::negative_binomial_distribution<> nbd3(10, 0.75);
   auto bnbd3 = std::bind(nbd3, eng);
-  testDiscreteDist(bnbd3, [](int n)
-		   { return negative_binomial_pdf(n, 10, 0.75); } );
+  testDiscreteDist<ARGS>(bnbd3, [](int n)
+			 { return negative_binomial_pdf(n, 10, 0.75); } );
 }
 
 int main()

Re: [PATCH] libstdc++-v3: Set 26_numerics/random/n.b.d./op./values.cc timeout-factor to 3

[Jonathan Wakely]

On Fri, 20 May 2022 at 10:03, Jonathan Wakely <jwakely@redhat.com> wrote:
>
> On Fri, 20 May 2022 at 03:20, Hans-Peter Nilsson via Libstdc++
> <libstdc++@gcc.gnu.org> wrote:
> >
> > Also, how about shortening those test-suite file-paths?
> > They're path sort-of overlong for any git (and ChangeLog)
> > commit-line limit. 1/2 :-)
>
> Yes, they're silly. I like what libc++ does, i.e. name the testsuite
> directories after the [stable.name] tags in the standard. They have
> std/rand/rand.dis/rand.dist.bern/rand.dist.bern.negbin (for some
> reason they use "rand.dis" instead of "rand.dist" for the third
> component there, which is not what the standard has,
> https://eel.is/c++draft/rand.dist.bern.negbin shows the subclause
> hierarchy).
>
> That's still quite unwieldy, so we could remove the repetition and use
> 26_numerics/rand/dist/bernoulli/negbin/. The problem now is that it
> gets quite cryptic, so finding the tests for the
> negative_binomial_distribution is a bit harder if you don't know where
> to look (but git grep solves that in less than a second).

Or we could forget about mirroring the nesting in the standard and
just use 26_numerics/random/dist/negative_binomial

That's still too long for ChangeLogs though.

Re: [PATCH] libstdc++-v3: Set 26_numerics/random/n.b.d./op./values.cc timeout-factor to 3

[Hans-Peter Nilsson]

> From: Jonathan Wakely <jwakely@redhat.com>
> Date: Fri, 20 May 2022 11:03:40 +0200

> > Ok to commit (without renaming)?
> 
> I'm OK with the timeout factor, but we could also solve it differently
> so that it doesn't take nearly 5 minutes, as in the attached patch.
> The testDiscreteDist function can be parametrized with the number of
> iterations to perform. Would you rather do that?

Yes thanks, down from 4m39s to 2.7 seconds, so very much
preferable!

(To the skeptics: the coverage intended with the test, is
IMHO reached with all non-simulator targets also running
this.  Nothing target-dependent here.)

Also in line with many other depth-level-cousin test-files
named value.cc.  Still many more others seem to be
candidates for such pruning, judging by the time it takes
for a 'RUNTESTFLAGS=--target_board=cris-sim\
conformance.exp=values.cc' to get to *that* values.cc.

Though, some -DSIMULATOR_TEST-adjusted files use dg-options,
others dg-additional-options.  It seems the difference is
that by using dg-options, you lose "-include bits/stdc++.h".
Likely not intended.  If so, should Someone fix that by
preapproval but regtested?

brgds, H-P

Re: [PATCH] libstdc++-v3: Set 26_numerics/random/n.b.d./op./values.cc timeout-factor to 3

[Jonathan Wakely]

[-- Attachment #1: Type: text/plain, Size: 1751 bytes --]

On Fri, 20 May 2022 at 15:31, Hans-Peter Nilsson wrote:
>
> > From: Jonathan Wakely <jwakely@redhat.com>
> > Date: Fri, 20 May 2022 11:03:40 +0200
>
> > > Ok to commit (without renaming)?
> >
> > I'm OK with the timeout factor, but we could also solve it differently
> > so that it doesn't take nearly 5 minutes, as in the attached patch.
> > The testDiscreteDist function can be parametrized with the number of
> > iterations to perform. Would you rather do that?
>
> Yes thanks, down from 4m39s to 2.7 seconds, so very much
> preferable!

Nice, thanks for testing it.

> (To the skeptics: the coverage intended with the test, is
> IMHO reached with all non-simulator targets also running
> this.  Nothing target-dependent here.)

Indeed.

> Also in line with many other depth-level-cousin test-files
> named value.cc.  Still many more others seem to be
> candidates for such pruning, judging by the time it takes
> for a 'RUNTESTFLAGS=--target_board=cris-sim\
> conformance.exp=values.cc' to get to *that* values.cc.
>
> Though, some -DSIMULATOR_TEST-adjusted files use dg-options,
> others dg-additional-options.  It seems the difference is
> that by using dg-options, you lose "-include bits/stdc++.h".
> Likely not intended.  If so, should Someone fix that by
> preapproval but regtested?

Ah good point. I've pushed the attached patch now. This adjusts the
values.cc files for some other distributions, and does so using
dg-additional-options not dg-options. Feel free to change existing
(mis)uses of dg-options to be dg-additional-options.

I've tested it on x86_64-linux, normally and also with
-DSIMULATOR_TEST in the test flags, just to ensure I didn't introduce
a silly syntax error for that case. I haven't tested it on a real sim
though.

[-- Attachment #2: patch.txt --]
[-- Type: text/plain, Size: 10961 bytes --]

commit e3b8b4f7814c54543d9b7ea3ee8cf2cb9cff351d
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Fri May 20 15:44:25 2022

    libstdc++: Reduce <random> test iterations for simulators
    
    Some of these tests take several minutes on a simulator like cris-elf,
    so we can conditionally run fewer iterations. The testDiscreteDist
    helper already supports custom sizes so we just need to make use of that
    when { target simulator } matches.
    
    The relevant code is sufficiently tested on other targets, so we're not
    losing anything by only running a small number of iterators for sims.
    
    libstdc++-v3/ChangeLog:
    
            * testsuite/26_numerics/random/bernoulli_distribution/operators/values.cc:
            Run fewer iterations for simulator targets.
            * testsuite/26_numerics/random/binomial_distribution/operators/values.cc:
            Likewise.
            * testsuite/26_numerics/random/discrete_distribution/operators/values.cc:
            Likewise.
            * testsuite/26_numerics/random/geometric_distribution/operators/values.cc:
            Likewise.
            * testsuite/26_numerics/random/negative_binomial_distribution/operators/values.cc:
            Likewise.
            * testsuite/26_numerics/random/poisson_distribution/operators/values.cc:
            Likewise.
            * testsuite/26_numerics/random/uniform_int_distribution/operators/values.cc:
            Likewise.

diff --git a/libstdc++-v3/testsuite/26_numerics/random/bernoulli_distribution/operators/values.cc b/libstdc++-v3/testsuite/26_numerics/random/bernoulli_distribution/operators/values.cc
index 4100692981c..b2cb86f976b 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/bernoulli_distribution/operators/values.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/bernoulli_distribution/operators/values.cc
@@ -24,6 +24,14 @@
 #include <functional>
 #include <testsuite_random.h>
 
+// { dg-additional-options "-DSIMULATOR_TEST" { target simulator } }
+
+#ifdef SIMULATOR_TEST
+# define ARGS 100, 1000
+#else
+# define ARGS
+#endif
+
 void test01()
 {
   using namespace __gnu_test;
@@ -32,15 +40,15 @@ void test01()
 
   std::bernoulli_distribution bd1(0.25);
   auto bbd1 = std::bind(bd1, eng);
-  testDiscreteDist(bbd1, [](int n) { return bernoulli_pdf(n, 0.25); } );
+  testDiscreteDist<ARGS>(bbd1, [](int n) { return bernoulli_pdf(n, 0.25); } );
 
   std::bernoulli_distribution bd2(0.5);
   auto bbd2 = std::bind(bd2, eng);
-  testDiscreteDist(bbd2, [](int n) { return bernoulli_pdf(n, 0.5); } );
+  testDiscreteDist<ARGS>(bbd2, [](int n) { return bernoulli_pdf(n, 0.5); } );
 
   std::bernoulli_distribution bd3(0.75);
   auto bbd3 = std::bind(bd3, eng);
-  testDiscreteDist(bbd3, [](int n) { return bernoulli_pdf(n, 0.75); } );
+  testDiscreteDist<ARGS>(bbd3, [](int n) { return bernoulli_pdf(n, 0.75); } );
 }
 
 int main()
diff --git a/libstdc++-v3/testsuite/26_numerics/random/binomial_distribution/operators/values.cc b/libstdc++-v3/testsuite/26_numerics/random/binomial_distribution/operators/values.cc
index 96570d59fb3..efa259b7e03 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/binomial_distribution/operators/values.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/binomial_distribution/operators/values.cc
@@ -25,6 +25,14 @@
 #include <functional>
 #include <testsuite_random.h>
 
+// { dg-additional-options "-DSIMULATOR_TEST" { target simulator } }
+
+#ifdef SIMULATOR_TEST
+# define ARGS 100, 1000
+#else
+# define ARGS
+#endif
+
 void test01()
 {
   using namespace __gnu_test;
@@ -33,9 +41,9 @@ void test01()
 
   std::binomial_distribution<> bd1(5, 0.3);
   auto bbd1 = std::bind(bd1, eng);
-  testDiscreteDist(bbd1, [](int n) { return binomial_pdf(n, 5, 0.3); } );
+  testDiscreteDist<ARGS>(bbd1, [](int n) { return binomial_pdf(n, 5, 0.3); } );
 
-  // These tests take a relatively long time on soft-float simulated
+  // These tests take a relatively long time on soft-float simulated targets.
   // targets, so please don't add new tests here, instead add a new file.
 }
 
diff --git a/libstdc++-v3/testsuite/26_numerics/random/discrete_distribution/operators/values.cc b/libstdc++-v3/testsuite/26_numerics/random/discrete_distribution/operators/values.cc
index 1cedd6434ca..8bacb86e173 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/discrete_distribution/operators/values.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/discrete_distribution/operators/values.cc
@@ -24,6 +24,14 @@
 #include <functional>
 #include <testsuite_random.h>
 
+// { dg-additional-options "-DSIMULATOR_TEST" { target simulator } }
+
+#ifdef SIMULATOR_TEST
+# define ARGS 100, 1000
+#else
+# define ARGS
+#endif
+
 void test01()
 {
   using namespace __gnu_test;
diff --git a/libstdc++-v3/testsuite/26_numerics/random/geometric_distribution/operators/values.cc b/libstdc++-v3/testsuite/26_numerics/random/geometric_distribution/operators/values.cc
index 65e0e79217b..41a83b1377b 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/geometric_distribution/operators/values.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/geometric_distribution/operators/values.cc
@@ -24,6 +24,14 @@
 #include <functional>
 #include <testsuite_random.h>
 
+// { dg-additional-options "-DSIMULATOR_TEST" { target simulator } }
+
+#ifdef SIMULATOR_TEST
+# define ARGS 100, 1000
+#else
+# define ARGS
+#endif
+
 void test01()
 {
   using namespace __gnu_test;
@@ -32,16 +40,16 @@ void test01()
 
   std::geometric_distribution<> gd1(0.5);
   auto bgd1 = std::bind(gd1, eng);
-  testDiscreteDist(bgd1, [](int n) { return geometric_pdf(n, 0.5); } );
+  testDiscreteDist<ARGS>(bgd1, [](int n) { return geometric_pdf(n, 0.5); } );
 
   std::geometric_distribution<> gd2(0.75);
   auto bgd2 = std::bind(gd2, eng);
-  testDiscreteDist(bgd2, [](int n) { return geometric_pdf(n, 0.75); } );
+  testDiscreteDist<ARGS>(bgd2, [](int n) { return geometric_pdf(n, 0.75); } );
 
   // libstdc++/48114
   std::geometric_distribution<> gd3(0.25);
   auto bgd3 = std::bind(gd3, eng);
-  testDiscreteDist(bgd3, [](int n) { return geometric_pdf(n, 0.25); } );
+  testDiscreteDist<ARGS>(bgd3, [](int n) { return geometric_pdf(n, 0.25); } );
 }
 
 int main()
diff --git a/libstdc++-v3/testsuite/26_numerics/random/negative_binomial_distribution/operators/values.cc b/libstdc++-v3/testsuite/26_numerics/random/negative_binomial_distribution/operators/values.cc
index dda6f43b254..9856b888577 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/negative_binomial_distribution/operators/values.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/negative_binomial_distribution/operators/values.cc
@@ -26,6 +26,14 @@
 #include <functional>
 #include <testsuite_random.h>
 
+// { dg-additional-options "-DSIMULATOR_TEST" { target simulator } }
+
+#ifdef SIMULATOR_TEST
+# define ARGS 100, 1000
+#else
+# define ARGS
+#endif
+
 void test01()
 {
   using namespace __gnu_test;
@@ -34,18 +42,18 @@ void test01()
 
   std::negative_binomial_distribution<> nbd1(5, 0.3);
   auto bnbd1 = std::bind(nbd1, eng);
-  testDiscreteDist(bnbd1, [](int n)
-		   { return negative_binomial_pdf(n, 5, 0.3); } );
+  testDiscreteDist<ARGS>(bnbd1, [](int n)
+			 { return negative_binomial_pdf(n, 5, 0.3); } );
 
   std::negative_binomial_distribution<> nbd2(55, 0.3);
   auto bnbd2 = std::bind(nbd2, eng);
-  testDiscreteDist(bnbd2, [](int n)
-		   { return negative_binomial_pdf(n, 55, 0.3); } );
+  testDiscreteDist<ARGS>(bnbd2, [](int n)
+			 { return negative_binomial_pdf(n, 55, 0.3); } );
 
   std::negative_binomial_distribution<> nbd3(10, 0.75);
   auto bnbd3 = std::bind(nbd3, eng);
-  testDiscreteDist(bnbd3, [](int n)
-		   { return negative_binomial_pdf(n, 10, 0.75); } );
+  testDiscreteDist<ARGS>(bnbd3, [](int n)
+			 { return negative_binomial_pdf(n, 10, 0.75); } );
 }
 
 int main()
diff --git a/libstdc++-v3/testsuite/26_numerics/random/poisson_distribution/operators/values.cc b/libstdc++-v3/testsuite/26_numerics/random/poisson_distribution/operators/values.cc
index 169d720d596..05e8c9f9eb1 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/poisson_distribution/operators/values.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/poisson_distribution/operators/values.cc
@@ -1,4 +1,3 @@
-// { dg-options "-DSIMULATOR_TEST" { target simulator } }
 // { dg-do run { target c++11 } }
 // { dg-require-cstdint "" }
 // { dg-require-cmath "" }
@@ -26,6 +25,14 @@
 #include <functional>
 #include <testsuite_random.h>
 
+// { dg-additional-options "-DSIMULATOR_TEST" { target simulator } }
+
+#ifdef SIMULATOR_TEST
+# define ARGS 100, 1000
+#else
+# define ARGS
+#endif
+
 void test01()
 {
   using namespace __gnu_test;
@@ -34,15 +41,15 @@ void test01()
 
   std::poisson_distribution<> pd1(3.0);
   auto bpd1 = std::bind(pd1, eng);
-  testDiscreteDist(bpd1, [](int n) { return poisson_pdf(n, 3.0); } );
+  testDiscreteDist<ARGS>(bpd1, [](int n) { return poisson_pdf(n, 3.0); } );
 
   std::poisson_distribution<> pd2(15.0);
   auto bpd2 = std::bind(pd2, eng);
-  testDiscreteDist(bpd2, [](int n) { return poisson_pdf(n, 15.0); } );
+  testDiscreteDist<ARGS>(bpd2, [](int n) { return poisson_pdf(n, 15.0); } );
 
   std::poisson_distribution<> pd3(30.0);
   auto bpd3 = std::bind(pd3, eng);
-  testDiscreteDist(bpd3, [](int n) { return poisson_pdf(n, 30.0); } );
+  testDiscreteDist<ARGS>(bpd3, [](int n) { return poisson_pdf(n, 30.0); } );
 
   // This can take extremely long on simulators, timing out the test.
 #ifndef SIMULATOR_TEST
diff --git a/libstdc++-v3/testsuite/26_numerics/random/uniform_int_distribution/operators/values.cc b/libstdc++-v3/testsuite/26_numerics/random/uniform_int_distribution/operators/values.cc
index c1e4b4944cc..ee1ea7ebe5f 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/uniform_int_distribution/operators/values.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/uniform_int_distribution/operators/values.cc
@@ -24,6 +24,14 @@
 #include <functional>
 #include <testsuite_random.h>
 
+// { dg-additional-options "-DSIMULATOR_TEST" { target simulator } }
+
+#ifdef SIMULATOR_TEST
+# define ARGS 100, 1000
+#else
+# define ARGS
+#endif
+
 void test01()
 {
   using namespace __gnu_test;
@@ -32,15 +40,15 @@ void test01()
 
   std::uniform_int_distribution<> uid1(0, 2);
   auto buid1 = std::bind(uid1, eng);
-  testDiscreteDist(buid1, [](int n) { return uniform_int_pdf(n, 0, 2); } );
+  testDiscreteDist<ARGS>(buid1, [](int n) { return uniform_int_pdf(n, 0, 2); } );
 
   std::uniform_int_distribution<> uid2(3, 7);
   auto buid2 = std::bind(uid2, eng);
-  testDiscreteDist(buid2, [](int n) { return uniform_int_pdf(n, 3, 7); } );
+  testDiscreteDist<ARGS>(buid2, [](int n) { return uniform_int_pdf(n, 3, 7); } );
 
   std::uniform_int_distribution<> uid3(1, 20);
   auto buid3 = std::bind(uid3, eng);
-  testDiscreteDist(buid3, [](int n) { return uniform_int_pdf(n, 1, 20); } );
+  testDiscreteDist<ARGS>(buid3, [](int n) { return uniform_int_pdf(n, 1, 20); } );
 }
 
 int main()