[Bug testsuite/103324] New: RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite

[Bug testsuite/103324] New: RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite

[egallager at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324

            Bug ID: 103324
           Summary: RFE: Add a `make quickcheck` or `make smoketest`
                    Makefile target to allow only running a portion of the
                    testsuite
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Severity: enhancement
          Priority: P3
         Component: testsuite
          Assignee: unassigned at gcc dot gnu.org
          Reporter: egallager at gcc dot gnu.org
                CC: mikestump at comcast dot net, ro at CeBiTec dot Uni-Bielefeld.DE
  Target Milestone: ---

The GCC testsuite is very big, and running it can take a long time. It would be
nice if there were alternate `check` or `test` targets that could be used to
run just a random sampling of the testsuite, or just a subset of the testsuite
that is known to run quickly, for cases when I am feeling impatient and only
need to verify that basic functionality works, without checking for every
possible regression. I have seen other people mentioning that patches "seem to
work ok after smoketesting" so it would be nice if their method of smoketesting
were actually available for the rest of us to use.

[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2021-11-18
     Ever confirmed|0                           |1
             Status|UNCONFIRMED                 |NEW

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Confirmed,


This what have been running since 2010 (and was there for a few years before
that):
        cd check-check-dir; $(RUNTEST) TMPDIR=`pwd` --all --tools gcc
$(DGFLAGS) \
          --srcdir=$(SRC)/gcc/testsuite execute.exp=20000112-1.c

This was for an out of tree already built toolchain testing even. If this
fails, I don't run the full testsuite. (this is not what I run for my upstream
testing though, only for my internal toolchain testing).

[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
I'm usually running pieces that affect the area I am patching like vect.exp for
vectorizer stuff.  Generally a smoke test would be dg-torture.exp (runs C and
C++ pieces) and execute.exp (C, ObjC, Go and Fortran).

Thus,

make check RUNTESTFLAGS="execute.exp"
make check RUNTESTFLAGS="dg-torture.exp"

for crosses compile.exp might be more light weight than execute.exp.  In
reality
bootstrap itself should be smoke test enough ...

[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite

[egallager at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324

--- Comment #3 from Eric Gallager <egallager at gcc dot gnu.org> ---
https://gcc.gnu.org/install/test.html says "To get a list of the possible *.exp
files, pipe the output of ‘make check’ into a file and look at the ‘Running …
.exp’ lines." ...has anyone stored their output from doing so recently? I don't
really want to run the entire testsuite just to generate this list...

[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite

[egallager at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324

--- Comment #4 from Eric Gallager <egallager at gcc dot gnu.org> ---
Another thing that would be useful would be to have (more) comments in the
source code saying stuff like "/* this codepath is tested by <testcasefilename>
*/" or something... although I guess it could be a problem keeping them in
sync...

[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite

[egallager at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324

--- Comment #5 from Eric Gallager <egallager at gcc dot gnu.org> ---
(In reply to Eric Gallager from comment #3)
> https://gcc.gnu.org/install/test.html says "To get a list of the possible
> *.exp files, pipe the output of ‘make check’ into a file and look at the
> ‘Running … .exp’ lines." ...has anyone stored their output from doing so
> recently? I don't really want to run the entire testsuite just to generate
> this list...

So, now I'm running the testsuite anyways for other reasons, and one more thing
to note is that using any sort of parallelism when running the testsuite (which
is pretty much a must these days) makes picking out the ‘Running … .exp’ lines
more difficult than necessary...

[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite

[egallager at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324

--- Comment #6 from Eric Gallager <egallager at gcc dot gnu.org> ---
(In reply to Eric Gallager from comment #5)
> (In reply to Eric Gallager from comment #3)
> > https://gcc.gnu.org/install/test.html says "To get a list of the possible
> > *.exp files, pipe the output of ‘make check’ into a file and look at the
> > ‘Running … .exp’ lines." ...has anyone stored their output from doing so
> > recently? I don't really want to run the entire testsuite just to generate
> > this list...
> 
> So, now I'm running the testsuite anyways for other reasons, and one more
> thing to note is that using any sort of parallelism when running the
> testsuite (which is pretty much a must these days) makes picking out the
> ‘Running … .exp’ lines more difficult than necessary...

In any case, here's my current list of "Running … .exp ..." lines (sorted and
uniq-ed):

Running ../../../../libatomic/testsuite/libatomic.c/c.exp ...
Running ../../../../libgomp/testsuite/libgomp.c++/c++.exp ...
Running ../../../../libgomp/testsuite/libgomp.c/c.exp ...
Running ../../../../libgomp/testsuite/libgomp.fortran/fortran.exp ...
Running ../../../../libgomp/testsuite/libgomp.graphite/graphite.exp ...
Running ../../../../libgomp/testsuite/libgomp.oacc-c++/c++.exp ...
Running ../../../../libgomp/testsuite/libgomp.oacc-c/c.exp ...
Running ../../../../libgomp/testsuite/libgomp.oacc-fortran/fortran.exp ...
Running ../../../../libitm/testsuite/libitm.c++/c++.exp ...
Running ../../../../libitm/testsuite/libitm.c/c.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/analyzer/analyzer.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/asan/asan.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/bprob/bprob.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/charset/charset.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/compat/compat.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/compat/struct-layout-1.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/coroutines/coroutines.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/coroutines/torture/coro-torture.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/debug/debug.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/debug/dwarf2/dwarf2.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/dfp/dfp.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/dg.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/gcov/gcov.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/goacc-gomp/goacc-gomp.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/goacc/goacc.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/gomp/gomp.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/graphite/graphite.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/guality/guality.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/hwasan/hwasan.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/lto/lto.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/modules/modules.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/pch/pch.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/plugin/plugin.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/simulate-thread/simulate-thread.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/special/ecos.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/tls/tls.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/tm/tm.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/torture/dg-torture.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/torture/stackalign/stackalign.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/tree-prof/tree-prof.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/tsan/tsan.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/ubsan/ubsan.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/vect/vect.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.old-deja/old-deja.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/aarch64/aarch64.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/aarch64/advsimd-intrinsics/advsimd-intrinsics.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/aarch64/sve/aarch64-sve.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/aarch64/sve/acle/aarch64-sve-acle-asm.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/aarch64/sve/acle/aarch64-sve-acle.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/aarch64/sve2/acle/aarch64-sve2-acle-asm.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/arm/arm.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/arm/mve.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/i386/i386.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/msp430/msp430.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/nios2/nios2.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/powerpc/powerpc.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/riscv/riscv.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/s390/s390.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/xtensa/xtensa.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.c-torture/compile/compile.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.c-torture/execute/execute.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg-selftests/dg-final.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/analyzer/analyzer.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/analyzer/torture/analyzer-torture.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/asan/asan.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/atomic/atomic.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/autopar/autopar.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/charset/charset.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/compat/compat.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/compat/struct-layout-1.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/cpp/cpp.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/cpp/trad/trad.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/debug/btf/btf.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/debug/ctf/ctf.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/debug/debug.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/debug/dwarf2/dwarf2.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/dfp/dfp.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/dg.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/fixed-point/fixed-point.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/format/format.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/goacc-gomp/goacc-gomp.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/goacc/goacc.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/gomp/gomp.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/graphite/graphite.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/guality/guality.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/hwasan/hwasan.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/ipa/ipa.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/lto/lto.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/noncompile/noncompile.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/params/params.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/pch/pch.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/plugin/plugin.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/rtl/rtl.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/sancov/sancov.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/special/mips-abi.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/special/special.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/sso/sso.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/tls/tls.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/tm/tm.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/torture/dg-torture.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/torture/tls/tls.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/tree-prof/tree-prof.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/tree-ssa/tree-ssa.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/tsan/tsan.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/ubsan/ubsan.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/vect/complex/complex.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/vect/costmodel/i386/i386-costmodel-vect.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/vect/costmodel/ppc/ppc-costmodel-vect.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/vect/costmodel/x86_64/x86_64-costmodel-vect.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/vect/vect.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/vmx/vmx.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/vxworks/vxworks.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.dg/weak/weak.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/acker1.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/arm-isr.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/bprob.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/dectest.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/dhry.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/gcov.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/godump.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/help.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/linkage.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/matrix1.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/mg-2.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/mg.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/options.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/output.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/outputs.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/sieve.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.misc-tests/sort2.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.src/maintainers.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/aapcs64/aapcs64.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/aarch64.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/acle/acle.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/advsimd-intrinsics/advsimd-intrinsics.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/cpunative/aarch64-cpunative.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/fp16/fp16.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/simd/simd.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/sls-mitigation/sls-mitigation.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/sve/aarch64-sve.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/sve/acle/aarch64-sve-acle-asm.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/sve/acle/aarch64-sve-acle.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/sve/pcs/aarch64-sve-pcs.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/sve2/aarch64-sve2.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/sve2/acle/aarch64-sve2-acle-asm.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/sve2/acle/aarch64-sve2-acle.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/aarch64/torture/aarch64-torture.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/alpha/alpha.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/arc/arc.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/arm/aapcs/aapcs.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/arm/acle/acle.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/arm/arm.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/arm/cmse/cmse.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/arm/lto/lto.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/arm/multilib.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/arm/mve/mve.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/arm/pure-code/pure-code.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/arm/simd/simd.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/avr/avr.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/avr/mmcu/avr-mmcu.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/avr/torture/avr-torture.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/bfin/bfin.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/bfin/builtins/bfin-builtins.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/bpf/bpf.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/cris/cris.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/cris/torture/cris-torture.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/csky/csky.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/csky/fpuv3/fpuv3.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/epiphany/epiphany.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/frv/frv.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/gcn/gcn.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/h8300/h8300.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/hppa/hppa.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/i386/i386.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/i386/iamcu/abi-iamcu.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/i386/math-torture/math-torture.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/i386/stackalign/stackalign.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/ia64/ia64.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/m68k/m68k.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/microblaze/microblaze.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/mips/inter/mips16-inter.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/mips/mips-nonpic/mips-nonpic.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/mips/mips.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/msp430/msp430.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/nds32/nds32.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/nios2/nios2.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/nvptx/nvptx.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/or1k/or1k.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/powerpc/bfp/bfp.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/powerpc/dfp/dfp.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/powerpc/powerpc.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/powerpc/ppc-fortran/ppc-fortran.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/powerpc/vsu/vsu.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/pru/pru.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/riscv/riscv.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/rl78/rl78.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/rx/rx.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/s390/s390.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/sh/sh.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/sh/torture/sh-torture.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/sparc/sparc.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/tic6x/builtins/c6x-builtins.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/tic6x/tic6x.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/v850/v850.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/vax/vax.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/visium/visium.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/x86_64/abi/abi-x86_64.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/x86_64/abi/avx/abi-avx.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/x86_64/abi/avx512f/abi-avx512f.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/x86_64/abi/avx512fp16/abi-avx512fp16-xmm.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/x86_64/abi/avx512fp16/m256h/abi-avx512fp16-ymm.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/x86_64/abi/avx512fp16/m512h/abi-avx512fp16-zmm.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/x86_64/abi/callabi/callabi.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/x86_64/abi/ms-sysv/ms-sysv.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/xstormy16/xstormy16.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.target/xtensa/xtensa.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gcc.test-framework/test-framework.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gdc.dg/asan/asan.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gdc.dg/debug/debug.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gdc.dg/debug/dwarf2/dwarf2.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gdc.dg/dg.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gdc.dg/extern-c++/extern-c++.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gdc.dg/lto/lto.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gdc.dg/torture/torture.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gdc.dg/ubsan/ubsan.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gdc.test/compilable/compilable.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gdc.test/fail_compilation/fail_compilation.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gdc.test/runnable/runnable.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gdc.test/runnable_cxx/runnable_cxx.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/analyzer/analyzer.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/asan/asan.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/c-interop/c-interop.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/coarray/caf.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/debug/debug.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/dg.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/goacc-gomp/goacc-gomp.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/goacc/goacc.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/gomp/gomp.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/graphite/graphite.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/guality/guality.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/ieee/ieee.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/lto/lto.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/prof/prof.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/ubsan/ubsan.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.dg/vect/vect.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.fortran-torture/compile/compile.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/gfortran.fortran-torture/execute/execute.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gnat.dg/dg.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gnat.dg/gcov/gcov.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gnat.dg/specs/specs.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/gnat.dg/sso/sso.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/jit.dg/jit.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/obj-c++.dg/attributes/attributes.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/obj-c++.dg/dg.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/obj-c++.dg/lto/lto.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/obj-c++.dg/plugin/plugin.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/obj-c++.dg/property/property.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/obj-c++.dg/strings/strings.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/obj-c++.dg/tls/tls.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/obj-c++.dg/torture/dg-torture.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/obj-c++.dg/torture/strings/strings.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/obj-c++.dg/torture/tls/tls.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/objc.dg/attributes/attributes.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/objc.dg/dg.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/objc.dg/gnu-encoding/gnu-encoding.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/objc.dg/lto/lto.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/objc.dg/pch/pch.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/objc.dg/plugin/plugin.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/objc.dg/property/property.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/objc.dg/special/special.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/objc.dg/strings/strings.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/objc.dg/tls/tls.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/objc.dg/torture/dg-torture.exp ...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/objc.dg/torture/strings/strings.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/objc.dg/torture/tls/tls.exp ...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/objc/compile/compile.exp
...
Running
/Users/ericgallager/gcc_newgit/gcc/testsuite/objc/execute/exceptions/exceptions.exp
...
Running /Users/ericgallager/gcc_newgit/gcc/testsuite/objc/execute/execute.exp
...
Running /Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.aa/aa.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.allocations/allocations.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.betterc/betterc.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.config/config.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.cycles/cycles.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.druntime/druntime.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.druntime_shared/druntime_shared.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.exceptions/exceptions.exp
...
Running /Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.gc/gc.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.hash/hash.exp ...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.imports/imports.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.init_fini/init_fini.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.lifetime/lifetime.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.phobos/phobos.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.phobos_shared/phobos_shared.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.shared/shared.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.thread/thread.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.traits/traits.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.typeinfo/typeinfo.exp
...
Running
/Users/ericgallager/gcc_newgit/libphobos/testsuite/libphobos.unittest/unittest.exp
...
Running
/Users/ericgallager/gcc_newgit/libstdc++-v3/testsuite/libstdc++-abi/abi.exp ...
Running
/Users/ericgallager/gcc_newgit/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp
...
Running
/Users/ericgallager/gcc_newgit/libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp
...
Running
/Users/ericgallager/gcc_newgit/libstdc++-v3/testsuite/libstdc++-xmethods/xmethods.exp
...

[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324

--- Comment #7 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Eric Gallager from comment #5)
> So, now I'm running the testsuite anyways for other reasons, and one more
> thing to note is that using any sort of parallelism when running the
> testsuite (which is pretty much a must these days) makes picking out the
> ‘Running … .exp’ lines more difficult than necessary...

Surely only if you try to get them while the tests are still running? After
they finish, all the output is flattened out into the .log files.

[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite

[sam at gentoo dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324

--- Comment #8 from Sam James <sam at gentoo dot org> ---
Using make synchronisation can help a bit:
https://www.gnu.org/software/make/manual/html_node/Parallel-Output.html. It's
made our build logs in Gentoo a lot more readable for GCC, FWIW.

As for the bug request: yes, this would be most helpful in trying to quickly
ascertain if things work well enough as a non-GCC developer. Otherwise I'm
stuck trying to diff failures and runs taking hours at a time. Not a complaint
about your workflow, but saying there's a good usecase for some partial "really
should pass" tests.

[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite

[egallager at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324

--- Comment #9 from Eric Gallager <egallager at gcc dot gnu.org> ---
(In reply to Sam James from comment #8)
> Using make synchronisation can help a bit:
> https://www.gnu.org/software/make/manual/html_node/Parallel-Output.html.
> It's made our build logs in Gentoo a lot more readable for GCC, FWIW.

So, I'm finally getting around to trying this, and it makes it seem as if the
testsuite is hanging while waiting for output to be synchronized...

[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite

[egallager at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324

--- Comment #10 from Eric Gallager <egallager at gcc dot gnu.org> ---
(In reply to Richard Biener from comment #2)
> Thus,
> 
> make check RUNTESTFLAGS="execute.exp"
> make check RUNTESTFLAGS="dg-torture.exp"


Just confirming that the proper way to combine these would be:
make check RUNTESTFLAGS="execute.exp dg-torture.exp"
...correct?
(The docs said something about being whitespace-delimited, but I wasn't quite
sure if that just applied to the globbing part, or also to the .exp
filenames...)

[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite

[egallager at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324

--- Comment #11 from Eric Gallager <egallager at gcc dot gnu.org> ---
(In reply to Eric Gallager from comment #10)
> (In reply to Richard Biener from comment #2)
> > Thus,
> > 
> > make check RUNTESTFLAGS="execute.exp"
> > make check RUNTESTFLAGS="dg-torture.exp"
> 
> 
> Just confirming that the proper way to combine these would be:
> make check RUNTESTFLAGS="execute.exp dg-torture.exp"
> ...correct?
> (The docs said something about being whitespace-delimited, but I wasn't
> quite sure if that just applied to the globbing part, or also to the .exp
> filenames...)

(ok, from testing, it appears that that's correct...)