[Bug libbacktrace/112396] New: "make check" should not error out, even when some checks failed

[Bug libbacktrace/112396] New: "make check" should not error out, even when some checks failed

[fxcoudert at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112396

            Bug ID: 112396
           Summary: "make check" should not error out, even when some
                    checks failed
           Product: gcc
           Version: 14.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libbacktrace
          Assignee: unassigned at gcc dot gnu.org
          Reporter: fxcoudert at gcc dot gnu.org
                CC: ian at gcc dot gnu.org
  Target Milestone: ---

The behaviour in GCC is pretty consistent: parts of the testsuite do not error
out in make, even when individual tests fail.

The one exception is libbacktrace, which does error out when one or more test
failed:

make[4]: *** [test-suite.log] Error 1
make[3]: *** [check-TESTS] Error 2
make[2]: *** [check-am] Error 2
make[2]: Target `check' not remade because of errors.
make[1]: *** [check-target-libbacktrace] Error 2

This means that if, like me, you forgot to run "make check" with -k, you get
stopped in the middle of testing. To be nice to users, and for consistency, it
would be nice if this behaviour was changed.

[Bug libbacktrace/112396] "make check" should not error out, even when some checks failed

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112396

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
https://gcc.gnu.org/install/test.html

Even mentions -k option for a long time.

[Bug libbacktrace/112396] "make check" should not error out, even when some checks failed

[egallager at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112396

Eric Gallager <egallager at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |egallager at gcc dot gnu.org

--- Comment #2 from Eric Gallager <egallager at gcc dot gnu.org> ---
(In reply to Andrew Pinski from comment #1)
> https://gcc.gnu.org/install/test.html
> 
> Even mentions -k option for a long time.

Still, it's kind of inconvenient having to do it that way, though. I think this
is an aspect of the testsuite worth changing.

[Bug libbacktrace/112396] "make check" should not error out, even when some checks failed

[ian at airs dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112396

Ian Lance Taylor <ian at airs dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ian at airs dot com

--- Comment #3 from Ian Lance Taylor <ian at airs dot com> ---
libbacktrace is just using automake and its testsuite support.  Anybody know if
there is a way to tell automake not to exit 1 on test failure?