* [PATCH] parallelize g++ testing a bit more @ 2011-06-17 18:01 Nathan Froyd 2011-06-17 18:38 ` Jason Merrill 2011-06-18 2:06 ` Mike Stump 0 siblings, 2 replies; 10+ messages in thread From: Nathan Froyd @ 2011-06-17 18:01 UTC (permalink / raw) To: gcc-patches; +Cc: jason I've done a lot of g++-only testsuite runs lately and I noticed that it didn't parallelize all that well. The patch below adds a couple more .exp files to the parallel infrastructure. dg-torture.exp is the big one; it takes about as much time as old-deja.exp. Other valid candidates are lto.exp and debug.exp, but the patch cuts g++ testing time in half as-is, so I felt it was a sufficient stopping point. OK to commit? -Nathan gcc/cp/ * Make-lang.in (check_g++_parallelize): Add more .exp files. diff --git a/gcc/cp/Make-lang.in b/gcc/cp/Make-lang.in index 45efd67..95bae37 100644 --- a/gcc/cp/Make-lang.in +++ b/gcc/cp/Make-lang.in @@ -154,7 +154,7 @@ check-c++-subtargets : check-g++-subtargets lang_checks += check-g++ lang_checks_parallelized += check-g++ # For description see comment above check_gcc_parallelize in gcc/Makefile.in. -check_g++_parallelize = old-deja.exp dg.exp +check_g++_parallelize = old-deja.exp dg.exp dg-torture.exp struct-layout-1.exp #\f # Install hooks: ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] parallelize g++ testing a bit more 2011-06-17 18:01 [PATCH] parallelize g++ testing a bit more Nathan Froyd @ 2011-06-17 18:38 ` Jason Merrill 2011-06-18 2:06 ` Mike Stump 1 sibling, 0 replies; 10+ messages in thread From: Jason Merrill @ 2011-06-17 18:38 UTC (permalink / raw) To: Nathan Froyd; +Cc: gcc-patches OK. Jason ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] parallelize g++ testing a bit more 2011-06-17 18:01 [PATCH] parallelize g++ testing a bit more Nathan Froyd 2011-06-17 18:38 ` Jason Merrill @ 2011-06-18 2:06 ` Mike Stump 2011-06-20 11:11 ` Rainer Orth 2011-06-20 13:33 ` Jason Merrill 1 sibling, 2 replies; 10+ messages in thread From: Mike Stump @ 2011-06-18 2:06 UTC (permalink / raw) To: Nathan Froyd; +Cc: gcc-patches, jason On Jun 17, 2011, at 10:47 AM, Nathan Froyd wrote: > I've done a lot of g++-only testsuite runs lately I think it is reasonable to have even more of them, say, if you have 16 cores and just test c++... I wonder what the scaling is like as we approach larger N. :-) ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] parallelize g++ testing a bit more 2011-06-18 2:06 ` Mike Stump @ 2011-06-20 11:11 ` Rainer Orth 2011-06-20 13:41 ` Mike Stump 2011-06-20 13:33 ` Jason Merrill 1 sibling, 1 reply; 10+ messages in thread From: Rainer Orth @ 2011-06-20 11:11 UTC (permalink / raw) To: Mike Stump; +Cc: Nathan Froyd, gcc-patches, jason Mike Stump <mikestump@comcast.net> writes: > On Jun 17, 2011, at 10:47 AM, Nathan Froyd wrote: >> I've done a lot of g++-only testsuite runs lately > > I think it is reasonable to have even more of them, say, if you have > 16 cores and just test c++... I wonder what the scaling is like as we > approach larger N. :-) I've got a patch to do this, prompted by the use of UltraSPARC-T2 machines with 8 cores/8 strands which are quite slow on their own: [build, testsuite, v3] Increase gcc, g++, gfortran and libstdc++-v3 testsuite parallelism http://gcc.gnu.org/ml/gcc-patches/2010-07/msg01633.html but haven't yet incorporated Jakub's comments. Rainer -- ----------------------------------------------------------------------------- Rainer Orth, Center for Biotechnology, Bielefeld University ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] parallelize g++ testing a bit more 2011-06-20 11:11 ` Rainer Orth @ 2011-06-20 13:41 ` Mike Stump 0 siblings, 0 replies; 10+ messages in thread From: Mike Stump @ 2011-06-20 13:41 UTC (permalink / raw) To: Rainer Orth; +Cc: Nathan Froyd, gcc-patches, jason On Jun 20, 2011, at 4:06 AM, Rainer Orth wrote: > I've got a patch to do this, prompted by the use of UltraSPARC-T2 > machines with 8 cores/8 strands which are quite slow on their own: > > [build, testsuite, v3] Increase gcc, g++, gfortran and libstdc++-v3 testsuite parallelism > http://gcc.gnu.org/ml/gcc-patches/2010-07/msg01633.html > > but haven't yet incorporated Jakub's comments. Yeah, I would not even consider a patch that was count based. :-) Ideally, what you want to do is to bin pack the completed in N seconds in B bins, where B is related to the physical number of cores. The, there is _no_ extra overhead for subdivision on 1 core, and on 10,000 core, it still works, though, one is going to need slightly better 1 to B code. Anyone know of a bin packer command line argument? We don't care if it is slow, because we can bin pack every 10th run, and start with idealized times. I would have contributed that sort of change a while ago, but didn't spot a bin packer. I also did up code to pull from a single work list, but, dejagnu isn't structured to do that. :-( That would be the best solution, but is much more work. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] parallelize g++ testing a bit more 2011-06-18 2:06 ` Mike Stump 2011-06-20 11:11 ` Rainer Orth @ 2011-06-20 13:33 ` Jason Merrill 2011-06-20 14:02 ` Jakub Jelinek 1 sibling, 1 reply; 10+ messages in thread From: Jason Merrill @ 2011-06-20 13:33 UTC (permalink / raw) To: Mike Stump; +Cc: Nathan Froyd, gcc-patches On 06/17/2011 08:20 PM, Mike Stump wrote: > On Jun 17, 2011, at 10:47 AM, Nathan Froyd wrote: >> I've done a lot of g++-only testsuite runs lately > > I think it is reasonable to have even more of them, say, if you have 16 cores and just test c++... I wonder what the scaling is like as we approach larger N. :-) In my test runs, one of the libstdc++ batches (normal3) takes longer than any of the g++ batches, so breaking that up would be more effective for me. :) Jason ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] parallelize g++ testing a bit more 2011-06-20 13:33 ` Jason Merrill @ 2011-06-20 14:02 ` Jakub Jelinek 2011-06-20 22:03 ` [PATCH] parallelize gcc, gfortran and libstdc++ testing some more Jakub Jelinek 0 siblings, 1 reply; 10+ messages in thread From: Jakub Jelinek @ 2011-06-20 14:02 UTC (permalink / raw) To: Jason Merrill; +Cc: Mike Stump, Nathan Froyd, gcc-patches On Mon, Jun 20, 2011 at 09:28:56AM -0400, Jason Merrill wrote: > On 06/17/2011 08:20 PM, Mike Stump wrote: > >On Jun 17, 2011, at 10:47 AM, Nathan Froyd wrote: > >>I've done a lot of g++-only testsuite runs lately > > > >I think it is reasonable to have even more of them, say, if you have 16 cores and just test c++... I wonder what the scaling is like as we approach larger N. :-) > > In my test runs, one of the libstdc++ batches (normal3) takes longer > than any of the g++ batches, so breaking that up would be more > effective for me. :) Yeah, certainly, libstdc++ needs more parallelization most. But, as I wrote already to Rainer in Autumn, the methodology for splitting things up should be on a fast box look at the time spent in each of the parts as reported by dejagnu in the log files, and split it to make the jobs roughly even sized with not overcomplicated patterns. Here are some numbers from my last x86_64-linux regtest. From these numbers, I'd say the first parallelization changes should be split gcc/execute.exp testing from 2 to 4 parallel jobs, gcc/compile.exp from one to 2, gfortran/dg.exp from 3 to 6 and libstdc++-v3/conformance.exp from 4 to 8 or 10. Everything else is much less important. gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/bprob/bprob.exp completed in 23 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/charset/charset.exp completed in 9 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/compat/compat.exp completed in 49 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/compat/struct-layout-1.exp completed in 271 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/debug/debug.exp completed in 156 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/debug/dwarf2/dwarf2.exp completed in 7 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/dfp/dfp.exp completed in 25 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/gcov/gcov.exp completed in 3 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/gomp/gomp.exp completed in 16 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/graphite/graphite.exp completed in 0 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/guality/guality.exp completed in 11 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/lto/lto.exp completed in 176 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/pch/pch.exp completed in 18 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/plugin/plugin.exp completed in 5 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/special/ecos.exp completed in 2 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/tls/tls.exp completed in 1 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/torture/dg-torture.exp completed in 469 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/torture/stackalign/stackalign.exp completed in 52 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/tree-prof/tree-prof.exp completed in 22 seconds gcc/testsuite/g++/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/vect/vect.exp completed in 1 seconds gcc/testsuite/g++1/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.old-deja/old-deja.exp completed in 662 seconds gcc/testsuite/g++2/g++.log.sep:testcase /usr/src/gcc/gcc/testsuite/g++.dg/dg.exp completed in 704 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp completed in 227 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/autopar/autopar.exp completed in 9 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/charset/charset.exp completed in 4 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/compat/compat.exp completed in 87 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/cpp/cpp.exp completed in 31 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/cpp/trad/trad.exp completed in 5 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/debug/debug.exp completed in 201 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/debug/dwarf2/dwarf2.exp completed in 6 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/dfp/dfp.exp completed in 28 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/fixed-point/fixed-point.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/format/format.exp completed in 25 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/gomp/gomp.exp completed in 16 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/graphite/graphite.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/guality/guality.exp completed in 272 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/ipa/ipa.exp completed in 5 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/lto/lto.exp completed in 115 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/matrix/matrix.exp completed in 4 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/noncompile/noncompile.exp completed in 24 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/pch/pch.exp completed in 52 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/plugin/plugin.exp completed in 4 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/special/mips-abi.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/special/special.exp completed in 2 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/tls/tls.exp completed in 3 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp completed in 33 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/tree-prof/tree-prof.exp completed in 8 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/tree-ssa/tree-ssa.exp completed in 51 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/vect/costmodel/i386/i386-costmodel-vect.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/vect/costmodel/ppc/ppc-costmodel-vect.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/vect/costmodel/spu/spu-costmodel-vect.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/vect/costmodel/x86_64/x86_64-costmodel-vect.exp completed in 1 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/vect/vect.exp completed in 248 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/vmx/vmx.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/vxworks/vxworks.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/weak/weak.exp completed in 1 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/acker1.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/arm-isr.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/bprob.exp completed in 1 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/dectest.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/dhry.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/gcov.exp completed in 2 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/help.exp completed in 1 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/linkage.exp completed in 1 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/matrix1.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/mg-2.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/mg.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/options.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/sieve.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.misc-tests/sort2.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/alpha/alpha.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/arm/aapcs/aapcs.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/arm/arm.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/arm/neon/neon.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/avr/avr.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/avr/torture/avr-torture.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/bfin/bfin.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/bfin/builtins/bfin-builtins.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/cris/cris.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/cris/torture/cris-torture.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/frv/frv.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/i386/math-torture/math-torture.exp completed in 15 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/ia64/ia64.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/m68k/m68k.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/microblaze/microblaze.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/mips/inter/mips16-inter.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/mips/mips.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/mips/mips-nonpic/mips-nonpic.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/powerpc/powerpc.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/rx/rx.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/s390/s390.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/sh/sh.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/sparc/sparc.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/spu/ea/ea.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/spu/spu.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/x86_64/abi/abi-x86_64.exp completed in 35 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/x86_64/abi/avx/abi-avx.exp completed in 9 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/x86_64/abi/callabi/callabi.exp completed in 1 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/xstormy16/xstormy16.exp completed in 0 seconds gcc/testsuite/gcc/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.test-framework/test-framework.exp completed in 0 seconds gcc/testsuite/gcc1/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.c-torture/execute/execute.exp completed in 1139 seconds gcc/testsuite/gcc2/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.c-torture/execute/execute.exp completed in 1582 seconds gcc/testsuite/gcc3/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.c-torture/compile/compile.exp completed in 1675 seconds gcc/testsuite/gcc4/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/dg.exp completed in 579 seconds gcc/testsuite/gcc5/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp completed in 495 seconds gcc/testsuite/gcc5/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/torture/dg-torture.exp completed in 662 seconds gcc/testsuite/gcc6/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp completed in 412 seconds gcc/testsuite/gcc6/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/compat/struct-layout-1.exp completed in 204 seconds gcc/testsuite/gcc6/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp completed in 300 seconds gcc/testsuite/gcc6/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/i386/i386.exp completed in 384 seconds gcc/testsuite/gcc6/gcc.log.sep:testcase /usr/src/gcc/gcc/testsuite/gcc.target/i386/stackalign/stackalign.exp completed in 0 seconds gcc/testsuite/gfortran/gfortran.log.sep:testcase /usr/src/gcc/gcc/testsuite/gfortran.dg/coarray/caf.exp completed in 12 seconds gcc/testsuite/gfortran/gfortran.log.sep:testcase /usr/src/gcc/gcc/testsuite/gfortran.dg/debug/debug.exp completed in 8 seconds gcc/testsuite/gfortran/gfortran.log.sep:testcase /usr/src/gcc/gcc/testsuite/gfortran.dg/gomp/gomp.exp completed in 24 seconds gcc/testsuite/gfortran/gfortran.log.sep:testcase /usr/src/gcc/gcc/testsuite/gfortran.dg/graphite/graphite.exp completed in 0 seconds gcc/testsuite/gfortran/gfortran.log.sep:testcase /usr/src/gcc/gcc/testsuite/gfortran.dg/guality/guality.exp completed in 16 seconds gcc/testsuite/gfortran/gfortran.log.sep:testcase /usr/src/gcc/gcc/testsuite/gfortran.dg/lto/lto.exp completed in 72 seconds gcc/testsuite/gfortran/gfortran.log.sep:testcase /usr/src/gcc/gcc/testsuite/gfortran.dg/vect/vect.exp completed in 64 seconds gcc/testsuite/gfortran/gfortran.log.sep:testcase /usr/src/gcc/gcc/testsuite/gfortran.fortran-torture/compile/compile.exp completed in 104 seconds gcc/testsuite/gfortran/gfortran.log.sep:testcase /usr/src/gcc/gcc/testsuite/gfortran.fortran-torture/execute/execute.exp completed in 605 seconds gcc/testsuite/gfortran1/gfortran.log.sep:testcase /usr/src/gcc/gcc/testsuite/gfortran.dg/dg.exp completed in 1184 seconds gcc/testsuite/gfortran2/gfortran.log.sep:testcase /usr/src/gcc/gcc/testsuite/gfortran.dg/dg.exp completed in 1409 seconds gcc/testsuite/gfortran3/gfortran.log.sep:testcase /usr/src/gcc/gcc/testsuite/gfortran.dg/dg.exp completed in 1464 seconds gcc/testsuite/go/go.log.sep:testcase /usr/src/gcc/gcc/testsuite/go.dg/dg.exp completed in 0 seconds gcc/testsuite/go/go.log.sep:testcase /usr/src/gcc/gcc/testsuite/go.go-torture/execute/execute.exp completed in 143 seconds gcc/testsuite/go1/go.log.sep:testcase /usr/src/gcc/gcc/testsuite/go.test/go-test.exp completed in 251 seconds gcc/testsuite/go2/go.log.sep:testcase /usr/src/gcc/gcc/testsuite/go.test/go-test.exp completed in 243 seconds gcc/testsuite/go3/go.log.sep:testcase /usr/src/gcc/gcc/testsuite/go.test/go-test.exp completed in 452 seconds gcc/testsuite/go4/go.log.sep:testcase /usr/src/gcc/gcc/testsuite/go.test/go-test.exp completed in 104 seconds gcc/testsuite/objc/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc/compile/compile.exp completed in 7 seconds gcc/testsuite/objc/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc.dg/attributes/attributes.exp completed in 4 seconds gcc/testsuite/objc/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc.dg/dg.exp completed in 60 seconds gcc/testsuite/objc/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc.dg/lto/lto.exp completed in 5 seconds gcc/testsuite/objc/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc.dg/pch/pch.exp completed in 4 seconds gcc/testsuite/objc/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc.dg/property/property.exp completed in 23 seconds gcc/testsuite/objc/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc.dg/special/special.exp completed in 2 seconds gcc/testsuite/objc/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc.dg/strings/strings.exp completed in 1 seconds gcc/testsuite/objc/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc.dg/tls/tls.exp completed in 1 seconds gcc/testsuite/objc/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc.dg/torture/dg-torture.exp completed in 11 seconds gcc/testsuite/objc/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc.dg/torture/strings/strings.exp completed in 35 seconds gcc/testsuite/objc/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc.dg/torture/tls/tls.exp completed in 12 seconds gcc/testsuite/objc1/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc.dg/gnu-encoding/gnu-encoding.exp completed in 52 seconds gcc/testsuite/objc2/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc/execute/execute.exp completed in 225 seconds gcc/testsuite/objc3/objc.log.sep:testcase /usr/src/gcc/gcc/testsuite/objc/execute/exceptions/exceptions.exp completed in 35 seconds x86_64-unknown-linux-gnu/libstdc++-v3/testsuite/libstdc++.log.sep:testcase /usr/src/gcc/libstdc++-v3/testsuite/libstdc++-abi/abi.exp completed in 37 seconds x86_64-unknown-linux-gnu/libstdc++-v3/testsuite/normal0/libstdc++.log.sep:testcase /usr/src/gcc/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp completed in 1284 seconds x86_64-unknown-linux-gnu/libstdc++-v3/testsuite/normal1/libstdc++.log.sep:testcase /usr/src/gcc/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp completed in 1174 seconds x86_64-unknown-linux-gnu/libstdc++-v3/testsuite/normal2/libstdc++.log.sep:testcase /usr/src/gcc/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp completed in 1778 seconds x86_64-unknown-linux-gnu/libstdc++-v3/testsuite/normal3/libstdc++.log.sep:testcase /usr/src/gcc/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp completed in 1516 seconds Jakub ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH] parallelize gcc, gfortran and libstdc++ testing some more 2011-06-20 14:02 ` Jakub Jelinek @ 2011-06-20 22:03 ` Jakub Jelinek 2011-06-21 7:09 ` Jason Merrill 2011-06-22 13:55 ` Jeff Law 0 siblings, 2 replies; 10+ messages in thread From: Jakub Jelinek @ 2011-06-20 22:03 UTC (permalink / raw) To: Mike Stump, Rainer Orth Cc: Jason Merrill, Nathan Froyd, gcc-patches, libstdc++ On Mon, Jun 20, 2011 at 03:54:59PM +0200, Jakub Jelinek wrote: > On Mon, Jun 20, 2011 at 09:28:56AM -0400, Jason Merrill wrote: > > On 06/17/2011 08:20 PM, Mike Stump wrote: > > >On Jun 17, 2011, at 10:47 AM, Nathan Froyd wrote: > > >>I've done a lot of g++-only testsuite runs lately > > > > > >I think it is reasonable to have even more of them, say, if you have 16 cores and just test c++... I wonder what the scaling is like as we approach larger N. :-) > > > > In my test runs, one of the libstdc++ batches (normal3) takes longer > > than any of the g++ batches, so breaking that up would be more > > effective for me. :) > > Yeah, certainly, libstdc++ needs more parallelization most. > But, as I wrote already to Rainer in Autumn, the methodology > for splitting things up should be on a fast box look at > the time spent in each of the parts as reported by dejagnu > in the log files, and split it to make the jobs roughly even > sized with not overcomplicated patterns. So, here is a patch that parallelizes the longest test chunks on x86_64-linux some more. libstdc++ testing is now split into 10 jobs, roughly equally sized (used to be 4), check-gcc testing is now split into 10 jobs too (used to be 7, but the new division attempts to be more balanced, fastest job takes for me 8 minutes and longest job takes 11), and check-gfortran is now run in 7 jobs instead of 4. This should make these more comparable e.g. to check-g++ chunks etc. Regtested on x86_64-linux, verified the merged *.sum files are identical. Ok for trunk? 2011-06-20 Jakub Jelinek <jakub@redhat.com> * Makefile.in (dg_target_exps): Set. (check_gcc_parallelize): Parallelize gcc testing into 10 jobs instead of 7, try to divide it more evenly. * Make-lang.in (check_gfortran_parallelize): Parallelize dg.exp into 6 jobs instead of 3. * testsuite/Makefile.am (check_DEJAGNU_normal_targets): Add check-DEJAGNUnormal[4-9]. (check-DEJAGNU): Split into 10 jobs for parallel testing instead of 4. * testsuite/Makefile.in: Regenerated. --- gcc/Makefile.in.jj 2011-06-15 11:54:43.000000000 +0200 +++ gcc/Makefile.in 2011-06-20 19:12:06.593450989 +0200 @@ -501,6 +501,10 @@ xm_include_list=@xm_include_list@ xm_defines=@xm_defines@ lang_checks=check-gcc lang_checks_parallelized=check-gcc +dg_target_exps:=alpha.exp,arm.exp,avr.exp,bfin.exp,cris.exp,frv.exp +dg_target_exps:=$(dg_target_exps),i386.exp,ia64.exp,m68k.exp,microblaze.exp +dg_target_exps:=$(dg_target_exps),mips.exp,powerpc.exp,rx.exp,s390.exp,sh.exp +dg_target_exps:=$(dg_target_exps),sparc.exp,spu.exp,xstormy16.exp # This lists a couple of test files that take most time during check-gcc. # When doing parallelized check-gcc, these can run in parallel with the # remaining tests. Each word in this variable stands for work for one @@ -514,10 +518,14 @@ lang_checks_parallelized=check-gcc # */ prefixed to it in runtest_file_p, it is usually desirable to include # a subdirectory name. check_gcc_parallelize=execute.exp=execute/2* \ - execute.exp=execute/\[013-9a-zA-Z\]* \ - compile.exp dg.exp \ - dg-torture.exp,builtins.exp \ - struct-layout-1.exp,unsorted.exp,stackalign.exp,i386.exp + execute.exp=execute/\[013-9a-fA-F\]* \ + execute.exp=execute/\[pP\]*,dg.exp \ + execute.exp=execute/\[g-oq-zG-OQ-Z\]*,compile.exp=compile/2* \ + compile.exp=compile/\[9pP\]*,builtins.exp \ + compile.exp=compile/\[013-8a-oq-zA-OQ-Z\]* \ + dg-torture.exp,ieee.exp \ + vect.exp,guality.exp,unsorted.exp \ + struct-layout-1.exp,stackalign.exp,$(dg_target_exps) lang_opt_files=@lang_opt_files@ $(srcdir)/c-family/c.opt $(srcdir)/common.opt lang_specs_files=@lang_specs_files@ lang_tree_files=@lang_tree_files@ --- gcc/fortran/Make-lang.in.jj 2011-05-24 23:59:28.000000000 +0200 +++ gcc/fortran/Make-lang.in 2011-06-20 20:29:29.986808197 +0200 @@ -171,9 +171,12 @@ check-fortran-subtargets : check-gfortra lang_checks += check-gfortran lang_checks_parallelized += check-gfortran # For description see comment above check_gcc_parallelize in gcc/Makefile.in. -check_gfortran_parallelize = dg.exp=gfortran.dg/\[a-cA-C\]* \ - dg.exp=gfortran.dg/\[d-mD-M\]* \ - dg.exp=gfortran.dg/\[n-zN-Z0-9\]* +check_gfortran_parallelize = dg.exp=gfortran.dg/\[adAD\]* \ + dg.exp=gfortran.dg/\[bcBC\]* \ + dg.exp=gfortran.dg/\[nopNOP\]* \ + dg.exp=gfortran.dg/\[isuvISUV\]* \ + dg.exp=gfortran.dg/\[efhkqrxzEFHKQRXZ\]* \ + dg.exp=gfortran.dg/\[0-9gjlmtwyGJLMTWY\]* # GFORTRAN documentation. GFORTRAN_TEXI = \ --- libstdc++-v3/testsuite/Makefile.am.jj 2011-03-03 10:37:23.000000000 +0100 +++ libstdc++-v3/testsuite/Makefile.am 2011-06-20 18:11:22.734808299 +0200 @@ -100,7 +100,7 @@ new-abi-baseline: @test ! -f $*/site.exp || mv $*/site.exp $*/site.bak @mv $*/site.exp.tmp $*/site.exp -check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3) +check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9) $(check_DEJAGNU_normal_targets): check-DEJAGNUnormal%: normal%/site.exp # Run the testsuite in normal mode. @@ -109,22 +109,17 @@ check-DEJAGNU $(check_DEJAGNU_normal_tar RANLIB="$(RANLIB)"; export RANLIB; \ if [ -z "$*$(filter-out --target_board=%, $(RUNTESTFLAGS))" ] \ && [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \ - $(MAKE) $(AM_MAKEFLAGS) check-DEJAGNUnormal0 check-DEJAGNUnormal1 \ - check-DEJAGNUnormal2 check-DEJAGNUnormal3; \ - for idx in 0 1 2 3; do \ + $(MAKE) $(AM_MAKEFLAGS) $(check_DEJAGNU_normal_targets); \ + for idx in 0 1 2 3 4 5 6 7 8 9; do \ mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \ mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \ done; \ mv -f libstdc++.sum libstdc++.sum.sep; \ mv -f libstdc++.log libstdc++.log.sep; \ $(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh \ - libstdc++.sum.sep normal0/libstdc++.sum.sep \ - normal1/libstdc++.sum.sep normal2/libstdc++.sum.sep \ - normal3/libstdc++.sum.sep > libstdc++.sum; \ + libstdc++.sum.sep normal[0-9]/libstdc++.sum.sep > libstdc++.sum; \ $(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh -L \ - libstdc++.log.sep normal0/libstdc++.log.sep \ - normal1/libstdc++.log.sep normal2/libstdc++.log.sep \ - normal3/libstdc++.log.sep > libstdc++.log; \ + libstdc++.log.sep normal[0-9]/libstdc++.log.sep > libstdc++.log; \ exit 0; \ fi; \ srcdir=`$(am__cd) $(srcdir) && pwd`; export srcdir; \ @@ -140,13 +135,25 @@ check-DEJAGNU $(check_DEJAGNU_normal_tar $(RUNTESTFLAGS) abi.exp; \ else echo "WARNING: could not find \`runtest'" 1>&2; :;\ fi; \ - dirs="`cd $$srcdir; echo [013-9][0-9]_*/* [ab]* de* [ep]*/*`";; \ + dirs="`cd $$srcdir; echo [013-9][0-9]_*/*`";; \ normal1) \ - dirs="`cd $$srcdir; echo 2[0-2]_*/*`";; \ + dirs="`cd $$srcdir; echo [ab]* de* [ep]*/*`";; \ normal2) \ - dirs="`cd $$srcdir; echo 2[4-9]_*/*`";; \ + dirs="`cd $$srcdir; echo 2[01]_*/*`";; \ normal3) \ - dirs="`cd $$srcdir; echo 23_*/* t*/*`";; \ + dirs="`cd $$srcdir; echo 22_*/*`";; \ + normal4) \ + dirs="`cd $$srcdir; echo 23_*/[a-km-tw-z]*`";; \ + normal5) \ + dirs="`cd $$srcdir; echo 23_*/[luv]*`";; \ + normal6) \ + dirs="`cd $$srcdir; echo 2[459]_*/*`";; \ + normal7) \ + dirs="`cd $$srcdir; echo 2[68]_*/*`";; \ + normal8) \ + dirs="`cd $$srcdir; echo 27_*/*`";; \ + normal9) \ + dirs="`cd $$srcdir; echo t*/*`";; \ esac; \ if [ -n "$*" ]; then cd "$*"; fi; \ if $(SHELL) -c "$$runtest --version" > /dev/null 2>&1; then \ --- libstdc++-v3/testsuite/Makefile.in.jj 2011-03-08 11:39:33.199392326 +0100 +++ libstdc++-v3/testsuite/Makefile.in 2011-06-20 18:11:48.176808266 +0200 @@ -287,7 +287,7 @@ lists_of_files = \ extract_symvers = $(glibcxx_builddir)/scripts/extract_symvers baseline_subdir := $(shell $(CXX) --print-multi-dir) -check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3) +check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9) # Runs the testsuite, but in compile only mode. # Can be used to test sources with non-GNU FE's at various warning @@ -545,22 +545,17 @@ check-DEJAGNU $(check_DEJAGNU_normal_tar RANLIB="$(RANLIB)"; export RANLIB; \ if [ -z "$*$(filter-out --target_board=%, $(RUNTESTFLAGS))" ] \ && [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \ - $(MAKE) $(AM_MAKEFLAGS) check-DEJAGNUnormal0 check-DEJAGNUnormal1 \ - check-DEJAGNUnormal2 check-DEJAGNUnormal3; \ - for idx in 0 1 2 3; do \ + $(MAKE) $(AM_MAKEFLAGS) $(check_DEJAGNU_normal_targets); \ + for idx in 0 1 2 3 4 5 6 7 8 9; do \ mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \ mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \ done; \ mv -f libstdc++.sum libstdc++.sum.sep; \ mv -f libstdc++.log libstdc++.log.sep; \ $(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh \ - libstdc++.sum.sep normal0/libstdc++.sum.sep \ - normal1/libstdc++.sum.sep normal2/libstdc++.sum.sep \ - normal3/libstdc++.sum.sep > libstdc++.sum; \ + libstdc++.sum.sep normal[0-9]/libstdc++.sum.sep > libstdc++.sum; \ $(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh -L \ - libstdc++.log.sep normal0/libstdc++.log.sep \ - normal1/libstdc++.log.sep normal2/libstdc++.log.sep \ - normal3/libstdc++.log.sep > libstdc++.log; \ + libstdc++.log.sep normal[0-9]/libstdc++.log.sep > libstdc++.log; \ exit 0; \ fi; \ srcdir=`$(am__cd) $(srcdir) && pwd`; export srcdir; \ @@ -576,13 +571,25 @@ check-DEJAGNU $(check_DEJAGNU_normal_tar $(RUNTESTFLAGS) abi.exp; \ else echo "WARNING: could not find \`runtest'" 1>&2; :;\ fi; \ - dirs="`cd $$srcdir; echo [013-9][0-9]_*/* [ab]* de* [ep]*/*`";; \ + dirs="`cd $$srcdir; echo [013-9][0-9]_*/*`";; \ normal1) \ - dirs="`cd $$srcdir; echo 2[0-2]_*/*`";; \ + dirs="`cd $$srcdir; echo [ab]* de* [ep]*/*`";; \ normal2) \ - dirs="`cd $$srcdir; echo 2[4-9]_*/*`";; \ + dirs="`cd $$srcdir; echo 2[01]_*/*`";; \ normal3) \ - dirs="`cd $$srcdir; echo 23_*/* t*/*`";; \ + dirs="`cd $$srcdir; echo 22_*/*`";; \ + normal4) \ + dirs="`cd $$srcdir; echo 23_*/[a-km-tw-z]*`";; \ + normal5) \ + dirs="`cd $$srcdir; echo 23_*/[luv]*`";; \ + normal6) \ + dirs="`cd $$srcdir; echo 2[459]_*/*`";; \ + normal7) \ + dirs="`cd $$srcdir; echo 2[68]_*/*`";; \ + normal8) \ + dirs="`cd $$srcdir; echo 27_*/*`";; \ + normal9) \ + dirs="`cd $$srcdir; echo t*/*`";; \ esac; \ if [ -n "$*" ]; then cd "$*"; fi; \ if $(SHELL) -c "$$runtest --version" > /dev/null 2>&1; then \ Jakub ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] parallelize gcc, gfortran and libstdc++ testing some more 2011-06-20 22:03 ` [PATCH] parallelize gcc, gfortran and libstdc++ testing some more Jakub Jelinek @ 2011-06-21 7:09 ` Jason Merrill 2011-06-22 13:55 ` Jeff Law 1 sibling, 0 replies; 10+ messages in thread From: Jason Merrill @ 2011-06-21 7:09 UTC (permalink / raw) To: Jakub Jelinek Cc: Mike Stump, Rainer Orth, Nathan Froyd, gcc-patches, libstdc++ OK. Jason ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] parallelize gcc, gfortran and libstdc++ testing some more 2011-06-20 22:03 ` [PATCH] parallelize gcc, gfortran and libstdc++ testing some more Jakub Jelinek 2011-06-21 7:09 ` Jason Merrill @ 2011-06-22 13:55 ` Jeff Law 1 sibling, 0 replies; 10+ messages in thread From: Jeff Law @ 2011-06-22 13:55 UTC (permalink / raw) To: Jakub Jelinek Cc: Mike Stump, Rainer Orth, Jason Merrill, Nathan Froyd, gcc-patches, libstdc++ -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06/20/11 15:04, Jakub Jelinek wrote: > On Mon, Jun 20, 2011 at 03:54:59PM +0200, Jakub Jelinek wrote: >> On Mon, Jun 20, 2011 at 09:28:56AM -0400, Jason Merrill wrote: >>> On 06/17/2011 08:20 PM, Mike Stump wrote: >>>> On Jun 17, 2011, at 10:47 AM, Nathan Froyd wrote: >>>>> I've done a lot of g++-only testsuite runs lately >>>> >>>> I think it is reasonable to have even more of them, say, if you have 16 cores and just test c++... I wonder what the scaling is like as we approach larger N. :-) >>> >>> In my test runs, one of the libstdc++ batches (normal3) takes longer >>> than any of the g++ batches, so breaking that up would be more >>> effective for me. :) >> >> Yeah, certainly, libstdc++ needs more parallelization most. >> But, as I wrote already to Rainer in Autumn, the methodology >> for splitting things up should be on a fast box look at >> the time spent in each of the parts as reported by dejagnu >> in the log files, and split it to make the jobs roughly even >> sized with not overcomplicated patterns. > > So, here is a patch that parallelizes the longest test chunks > on x86_64-linux some more. > > libstdc++ testing is now split into 10 jobs, roughly equally sized > (used to be 4), check-gcc testing is now split into 10 jobs too (used to > be 7, but the new division attempts to be more balanced, fastest > job takes for me 8 minutes and longest job takes 11), and check-gfortran > is now run in 7 jobs instead of 4. This should make these more comparable > e.g. to check-g++ chunks etc. > > Regtested on x86_64-linux, verified the merged *.sum files are identical. > Ok for trunk? > > 2011-06-20 Jakub Jelinek <jakub@redhat.com> > > * Makefile.in (dg_target_exps): Set. > (check_gcc_parallelize): Parallelize gcc testing into 10 jobs > instead of 7, try to divide it more evenly. > > * Make-lang.in (check_gfortran_parallelize): Parallelize dg.exp > into 6 jobs instead of 3. > > * testsuite/Makefile.am (check_DEJAGNU_normal_targets): Add > check-DEJAGNUnormal[4-9]. > (check-DEJAGNU): Split into 10 jobs for parallel testing instead of 4. > * testsuite/Makefile.in: Regenerated. OK. jeff -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOAfOYAAoJEBRtltQi2kC7fh4H/0ku6F+8rZEb0iy0zyr+KHUd 2qvA4BsXPcNs3xjwdqgrpvg9pM+6qXsEB8oEuaCBu7ndT8kCMPfnjaRP3otnl5ju Maceyf0rZMDOUVYiYXYEstLMH+gljpraBU/n6UrvrXLLtZzna5yU6ZLwmPs1ngt/ +QQXWe34gRraEsLo4A9y5pcU69WnsVMdmrVKsRn/EnsLqSxClvmyWSaFrX0i5X4C +NfQC3YQqiG0bjC6acCyfccHGJ+7tlyiafsyR0TDZPfGQLCwnRWNhm+A/GrTjYpY nHbwhZ7sJef3gz9pLRm4qJ0OyMKqZo6c07Z43KdYIaIMQdjsh0CA3xfnd/vHSYQ= =wO5+ -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2011-06-22 13:53 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2011-06-17 18:01 [PATCH] parallelize g++ testing a bit more Nathan Froyd 2011-06-17 18:38 ` Jason Merrill 2011-06-18 2:06 ` Mike Stump 2011-06-20 11:11 ` Rainer Orth 2011-06-20 13:41 ` Mike Stump 2011-06-20 13:33 ` Jason Merrill 2011-06-20 14:02 ` Jakub Jelinek 2011-06-20 22:03 ` [PATCH] parallelize gcc, gfortran and libstdc++ testing some more Jakub Jelinek 2011-06-21 7:09 ` Jason Merrill 2011-06-22 13:55 ` Jeff Law
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).