[PATCH] testsuite: Disable slow and unneeded test variants

[PATCH] testsuite: Disable slow and unneeded test variants

[Dimitar Dimitrov]

The issue in PR112344 is triggered only at -O2, so there is little value
in running the test at lower optimization levels.  At the same time the
generated code at low and code-size optimization levels is taking a long
time to execute because it loops a few billion iterations.

On the PRU simulator target the non-optimized test variants take more
than 10 minutes, thus failing due to timeout.  Even a native x86_64
takes a few seconds to run the non-optimized variants.

Let's not waste cycles and run only the test configurations which
triggered the issue described in the PR.

On native x86_64 Linux:
$ time make check-gcc-c -j10 RUNTESTFLAGS="dg-torture.exp=pr112344.c"
------------------------------------------
Time    Previously  With this patch
------------------------------------------
real    0m4,786s    0m1,694s
user    0m7,031s    0m4,013s
sys     0m3,300s    0m3,234s

With PRU simulator:
$ time make -j10 check-gcc-c RUNTESTFLAGS="--target_board=pru-sim dg-torture.exp=pr112344.c"
------------------------------------------
Time    Previously   With this patch
------------------------------------------
real    11m32,740s   0m1,897s
user    11m34,301s   0m4,012s
sys     0m2,178s     0m2,133s


Ok for trunk?

	PR middle-end/112344

gcc/testsuite/ChangeLog:

	* gcc.dg/torture/pr112344.c: Run only
	for expensive speed optimizations.

Signed-off-by: Dimitar Dimitrov <dimitar@dinux.eu>
---
 gcc/testsuite/gcc.dg/torture/pr112344.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/gcc/testsuite/gcc.dg/torture/pr112344.c b/gcc/testsuite/gcc.dg/torture/pr112344.c
index c52d2c8304b..abcef51428f 100644
--- a/gcc/testsuite/gcc.dg/torture/pr112344.c
+++ b/gcc/testsuite/gcc.dg/torture/pr112344.c
@@ -1,6 +1,8 @@
 /* { dg-do run } */
 /* { dg-require-effective-target int32plus } */
 
+/* { dg-skip-if "triggered by expensive speed optimizations" { *-*-* } { "-O0" "-O1" "-Os" "-Oz" } { "" } } */
+
 int
 main ()
 {
-- 
2.43.0

Re: [PATCH] testsuite: Disable slow and unneeded test variants

[Jakub Jelinek]

On Fri, Feb 16, 2024 at 07:52:17PM +0200, Dimitar Dimitrov wrote:
> The issue in PR112344 is triggered only at -O2, so there is little value
> in running the test at lower optimization levels.  At the same time the

That is generally not true.
We had hundreds of cases in the history where a test written for one bug
let us discover a different bug later on, often at different optimization
level etc.

If the test is too slow, perhaps the dg-skip-if should take the
run_expensive_tests effective target into account, like:
/* { dg-skip-if "" { ! run_expensive_tests }  { "*" } { "-O0" "-O1" } } */

But guess another question is if the bug can be reproduced with fewer
iterations...

	Jakub

Re: [PATCH] testsuite: Disable slow and unneeded test variants

[Dimitar Dimitrov]

On Fri, Feb 16, 2024 at 07:06:57PM +0100, Jakub Jelinek wrote:
> On Fri, Feb 16, 2024 at 07:52:17PM +0200, Dimitar Dimitrov wrote:
> > The issue in PR112344 is triggered only at -O2, so there is little value
> > in running the test at lower optimization levels.  At the same time the
> 
> That is generally not true.
> We had hundreds of cases in the history where a test written for one bug
> let us discover a different bug later on, often at different optimization
> level etc.
> 
> If the test is too slow, perhaps the dg-skip-if should take the
> run_expensive_tests effective target into account, like:
> /* { dg-skip-if "" { ! run_expensive_tests }  { "*" } { "-O0" "-O1" } } */

Thank you, this is reasonable.  I'll send V2 with that change.

Curiously, -Os also needs to be added to the slow variants, along with
-O0 and -O1.

> 
> But guess another question is if the bug can be reproduced with fewer
> iterations...

I could not think of a way to do it.

Regards,
Dimitar

> 
> 	Jakub
> 

Re: [PATCH] testsuite: Disable slow and unneeded test variants

[Jakub Jelinek]

On Sat, Feb 17, 2024 at 09:59:12PM +0200, Dimitar Dimitrov wrote:
> On Fri, Feb 16, 2024 at 07:06:57PM +0100, Jakub Jelinek wrote:
> > On Fri, Feb 16, 2024 at 07:52:17PM +0200, Dimitar Dimitrov wrote:
> > > The issue in PR112344 is triggered only at -O2, so there is little value
> > > in running the test at lower optimization levels.  At the same time the
> > 
> > That is generally not true.
> > We had hundreds of cases in the history where a test written for one bug
> > let us discover a different bug later on, often at different optimization
> > level etc.
> > 
> > If the test is too slow, perhaps the dg-skip-if should take the
> > run_expensive_tests effective target into account, like:
> > /* { dg-skip-if "" { ! run_expensive_tests }  { "*" } { "-O0" "-O1" } } */
> 
> Thank you, this is reasonable.  I'll send V2 with that change.
> 
> Curiously, -Os also needs to be added to the slow variants, along with
> -O0 and -O1.

Ok then.

> > But guess another question is if the bug can be reproduced with fewer
> > iterations...
> 
> I could not think of a way to do it.

Yeah, I haven't managed to reduce the number of iterations and still trigger
the bug.

	Jakub