public inbox for gcc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug modula2/114886] New: gm2 testsuite arbitrarily reduces timeouts
@ 2024-04-29  9:42 ro at gcc dot gnu.org
  2024-04-29  9:43 ` [Bug modula2/114886] " ro at gcc dot gnu.org
                   ` (5 more replies)
  0 siblings, 6 replies; 7+ messages in thread
From: ro at gcc dot gnu.org @ 2024-04-29  9:42 UTC (permalink / raw)
  To: gcc-bugs

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114886

            Bug ID: 114886
           Summary: gm2 testsuite arbitrarily reduces timeouts
           Product: gcc
           Version: 14.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: modula2
          Assignee: gaius at gcc dot gnu.org
          Reporter: ro at gcc dot gnu.org
  Target Milestone: ---

When bootstrapping the gcc-14 branch on sparc-sun-solaris2.11, quite a number
of gm2 tests FAILed due to timeouts.

I had seen this before and could trace this to the gm2 testsuite reducing way
beyond the default of 300 seconds (sometimes as low as 10 seconds).  I think
this is badly misguided for a couple of reasons:

* If a test completes (PASS or FAIL) within the reduced reduced timeout, it
  will do so with the default timeout, too.

* If a test takes either longer than the default timeout (either because
  compilation of execution is particularly slow or because some step loops
  indefinitely), reducing the timeout would give a result earlier.  However,
  I've rarely if ever seen such a case in all my testing.

* If a test takes longer than the reduced timeout, it will fail while it would
  pass just fine with the default timeout.

* Using hardcoded timeouts is always wrong: while the values currenly used
  may work one some particular target, it will cause failures on systems that
  are either slower or under higher load.  Besides, it is possible to globally
  change the derfault in some expect snippet (like

  global board_info
  set board_info(unix,gcc,timeout) 600

  (even differently per multilib if desired).

IMO this whole approach is completely misguided, has very little benefit even
in the best case, and demonstrably introduces failures that wouldn't exist
without
it.

I'm attaching my current hack for reference, but it's completely inappropriate 
as is.  It only identifies some (all?) places that need changing.

^ permalink raw reply	[flat|nested] 7+ messages in thread
end of thread, other threads:[~2024-06-04  7:13 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-29  9:42 [Bug modula2/114886] New: gm2 testsuite arbitrarily reduces timeouts ro at gcc dot gnu.org
2024-04-29  9:43 ` [Bug modula2/114886] " ro at gcc dot gnu.org
2024-04-30  7:30 ` ro at gcc dot gnu.org
2024-04-30 11:50 ` cvs-commit at gcc dot gnu.org
2024-04-30 11:53 ` ro at gcc dot gnu.org
2024-06-04  7:12 ` cvs-commit at gcc dot gnu.org
2024-06-04  7:13 ` ro at gcc dot gnu.org

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).