[PATCH] libstdc++ testsuite: Increase lwg3464.cc timeout factors to 20

[PATCH] libstdc++ testsuite: Increase lwg3464.cc timeout factors to 20

[Hans-Peter Nilsson]

These tests have always been failing for my cris-elf
autotester running a simulator; they take about 20 minutes
each, compared to the timeout of 720 seconds, doubled
because they timed out in another simulator setup.

They are the *only* libstdc++ tests that timeout for my
setup so I thought this'd be best fixed in the testsuite
rather than a local timeout setting (in e.g. the baseboard
file).  And, better make it an increase that counts.  Or,
maybe they're actually needlessly excessive? FWIW, running
the whole libstdc++ testsuite for this target takes 2h40min.

Ok?

	* testsuite/27_io/basic_istream/get/char/lwg3464.cc: Increase
	timeout factor to 20.
	* testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc: Likewise.
---
 libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc  | 2 +-
 .../testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc        | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc b/libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc
index f426ff7dd867..69612375cba0 100644
--- a/libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc
+++ b/libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc
@@ -16,7 +16,7 @@
 // <http://www.gnu.org/licenses/>.
 
 // { dg-do run { target { ! lp64 } } }
-// { dg-timeout-factor 2 }
+// { dg-timeout-factor 20 }
 
 #include <istream>
 #include <streambuf>
diff --git a/libstdc++-v3/testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc b/libstdc++-v3/testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc
index 4be6beb2310f..5f07c6d9880d 100644
--- a/libstdc++-v3/testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc
+++ b/libstdc++-v3/testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc
@@ -16,7 +16,7 @@
 // <http://www.gnu.org/licenses/>.
 
 // { dg-do run { target { ! lp64 } } }
-// { dg-timeout-factor 2 }
+// { dg-timeout-factor 20 }
 
 #include <istream>
 #include <streambuf>
-- 
2.30.2

Re: [PATCH] libstdc++ testsuite: Increase lwg3464.cc timeout factors to 20

[Jonathan Wakely]

On Sun, 30 Jan 2022, 01:37 Hans-Peter Nilsson via Libstdc++, <
libstdc++@gcc.gnu.org> wrote:

> These tests have always been failing for my cris-elf
> autotester running a simulator; they take about 20 minutes
> each, compared to the timeout of 720 seconds, doubled
> because they timed out in another simulator setup.
>
> They are the *only* libstdc++ tests that timeout for my
> setup so I thought this'd be best fixed in the testsuite
> rather than a local timeout setting (in e.g. the baseboard
> file).  And, better make it an increase that counts.  Or,
> maybe they're actually needlessly excessive?


They are testing behaviour when a counter overflows, so they have to read
that many bytes. Making them do less work would not test that condition.

But there is nothing target-specific in that code, so it should be fine to
disable them for simulators. They're already disabled for LP64 because
overflowing the 64-bit counter would take forever.

I think that would be better than letting them potentially run for 40
minutes even on real hardware.

[PATCH] libstdc++ testsuite: Don't run lwg3464.cc tests on simulators to 20

[Hans-Peter Nilsson]

(I had to add separation between your reply and mine in the
quoted parts; you may be aware.)

> From: Jonathan Wakely <jwakely.gcc@gmail.com>
> Date: Sun, 30 Jan 2022 09:32:08 +0100
> On Sun, 30 Jan 2022, 01:37 Hans-Peter Nilsson via Libstdc++, <libstdc++@gcc.gnu.org<mailto:libstdc%2B%2B@gcc.gnu.org>> wrote:
> > These tests have always been failing for my cris-elf
> > autotester running a simulator; they take about 20 minutes
> > each, compared to the timeout of 720 seconds, doubled
> > because they timed out in another simulator setup.
> >
> > They are the *only* libstdc++ tests that timeout for my
> > setup so I thought this'd be best fixed in the testsuite
> > rather than a local timeout setting (in e.g. the baseboard
> > file).  And, better make it an increase that counts.  Or,
> > maybe they're actually needlessly excessive?
> 
> They are testing behaviour when a counter overflows, so
> they have to read that many bytes. Making them do less
> work would not test that condition.
> 
> But there is nothing target-specific in that code, so it
> should be fine to disable them for simulators. They're
> already disabled for LP64 because overflowing the 64-bit
> counter would take forever.
> 
> I think that would be better than letting them potentially
> run for 40 minutes even on real hardware.

Ok.  Then this becomes obvious (except from the dejagnu
syntax, but that has an obvious progress path).  Also tested
that it still doesn't run nor gets some dejagnu error on
native x86_64-linux-gnu.

Committed.

8<--------->8
These tests have always been failing for my autotester running a
cris-elf simulator; when unrestrained they take about 20 minutes each,
compared to the (doubled) timeout of 720 seconds, of a total 2h40min
for the whole of the libstdc++-v3 testsuite.  The tests cover counter
overflow and are already disabled for LP64 targets.

	* testsuite/27_io/basic_istream/get/char/lwg3464.cc: Don't run on
	simulator targets.
	* testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc: Likewise.
---
 libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc  | 2 +-
 .../testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc        | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc b/libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc
index f426ff7dd867..5d4de3d6cb54 100644
--- a/libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc
+++ b/libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc
@@ -15,7 +15,7 @@
 // with this library; see the file COPYING3.  If not see
 // <http://www.gnu.org/licenses/>.
 
-// { dg-do run { target { ! lp64 } } }
+// { dg-do run { target { { ! lp64 } && { ! simulator } } } }
 // { dg-timeout-factor 2 }
 
 #include <istream>
diff --git a/libstdc++-v3/testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc b/libstdc++-v3/testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc
index 4be6beb2310f..e2349df8c7d6 100644
--- a/libstdc++-v3/testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc
+++ b/libstdc++-v3/testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc
@@ -15,7 +15,7 @@
 // with this library; see the file COPYING3.  If not see
 // <http://www.gnu.org/licenses/>.
 
-// { dg-do run { target { ! lp64 } } }
+// { dg-do run { target { { ! lp64 } && { ! simulator } } } }
 // { dg-timeout-factor 2 }
 
 #include <istream>
-- 
2.30.2

Re: [PATCH] libstdc++ testsuite: Don't run lwg3464.cc tests on simulators to 20

[Jonathan Wakely]

On Sun, 30 Jan 2022 at 16:54, Hans-Peter Nilsson <hp@axis.com> wrote:
>
> (I had to add separation between your reply and mine in the
> quoted parts; you may be aware.)
>
> > From: Jonathan Wakely <jwakely.gcc@gmail.com>
> > Date: Sun, 30 Jan 2022 09:32:08 +0100
> > On Sun, 30 Jan 2022, 01:37 Hans-Peter Nilsson via Libstdc++, <libstdc++@gcc.gnu.org<mailto:libstdc%2B%2B@gcc.gnu.org>> wrote:
> > > These tests have always been failing for my cris-elf
> > > autotester running a simulator; they take about 20 minutes
> > > each, compared to the timeout of 720 seconds, doubled
> > > because they timed out in another simulator setup.
> > >
> > > They are the *only* libstdc++ tests that timeout for my
> > > setup so I thought this'd be best fixed in the testsuite
> > > rather than a local timeout setting (in e.g. the baseboard
> > > file).  And, better make it an increase that counts.  Or,
> > > maybe they're actually needlessly excessive?
> >
> > They are testing behaviour when a counter overflows, so
> > they have to read that many bytes. Making them do less
> > work would not test that condition.
> >
> > But there is nothing target-specific in that code, so it
> > should be fine to disable them for simulators. They're
> > already disabled for LP64 because overflowing the 64-bit
> > counter would take forever.
> >
> > I think that would be better than letting them potentially
> > run for 40 minutes even on real hardware.
>
> Ok.  Then this becomes obvious (except from the dejagnu
> syntax, but that has an obvious progress path).  Also tested
> that it still doesn't run nor gets some dejagnu error on
> native x86_64-linux-gnu.
>
> Committed.

Thanks!


>
> 8<--------->8
> These tests have always been failing for my autotester running a
> cris-elf simulator; when unrestrained they take about 20 minutes each,
> compared to the (doubled) timeout of 720 seconds, of a total 2h40min
> for the whole of the libstdc++-v3 testsuite.  The tests cover counter
> overflow and are already disabled for LP64 targets.
>
>         * testsuite/27_io/basic_istream/get/char/lwg3464.cc: Don't run on
>         simulator targets.
>         * testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc: Likewise.
> ---
>  libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc  | 2 +-
>  .../testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc        | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc b/libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc
> index f426ff7dd867..5d4de3d6cb54 100644
> --- a/libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc
> +++ b/libstdc++-v3/testsuite/27_io/basic_istream/get/char/lwg3464.cc
> @@ -15,7 +15,7 @@
>  // with this library; see the file COPYING3.  If not see
>  // <http://www.gnu.org/licenses/>.
>
> -// { dg-do run { target { ! lp64 } } }
> +// { dg-do run { target { { ! lp64 } && { ! simulator } } } }
>  // { dg-timeout-factor 2 }
>
>  #include <istream>
> diff --git a/libstdc++-v3/testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc b/libstdc++-v3/testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc
> index 4be6beb2310f..e2349df8c7d6 100644
> --- a/libstdc++-v3/testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc
> +++ b/libstdc++-v3/testsuite/27_io/basic_istream/get/wchar_t/lwg3464.cc
> @@ -15,7 +15,7 @@
>  // with this library; see the file COPYING3.  If not see
>  // <http://www.gnu.org/licenses/>.
>
> -// { dg-do run { target { ! lp64 } } }
> +// { dg-do run { target { { ! lp64 } && { ! simulator } } } }
>  // { dg-timeout-factor 2 }
>
>  #include <istream>
> --
> 2.30.2
>
>

Re: [PATCH] libstdc++ testsuite: Don't run lwg3464.cc tests on simulators to 20

[Hans-Peter Nilsson]

> From: Jonathan Wakely <jwakely.gcc@gmail.com>
> Date: Sun, 30 Jan 2022 18:06:28 +0100
> On Sun, 30 Jan 2022 at 16:54, Hans-Peter Nilsson <hp@axis.com> wrote:
> > > From: Jonathan Wakely <jwakely.gcc@gmail.com>
> > > But there is nothing target-specific in that code, so it
> > > should be fine to disable them for simulators. They're
> > > already disabled for LP64 because overflowing the 64-bit
> > > counter would take forever.
> > >
> > > I think that would be better than letting them potentially
> > > run for 40 minutes even on real hardware.
> >
> > Ok.  Then this becomes obvious (except from the dejagnu
> > syntax, but that has an obvious progress path).  Also tested
> > that it still doesn't run nor gets some dejagnu error on
> > native x86_64-linux-gnu.
> >
> > Committed.
> 
> Thanks!

No reason, but thank *you* for the review and for the
biggest time win in test-cycle ever.  No, I didn't look, but
7% (720s*2)/5.5h - the total time; surely is unbeatable.

brgds, H-P