* parallel testsuite results differ from a single task approach
@ 2012-12-21 4:06 Dennis Clarke
2012-12-21 13:01 ` Tim Prince
0 siblings, 1 reply; 3+ messages in thread
From: Dennis Clarke @ 2012-12-21 4:06 UTC (permalink / raw)
To: gcc-help; +Cc: Jonathan Wakely, Ian Lance Taylor
I gad received some good advice to try using gmake -j X -k check for some X > 1 when running a testsuite in order to speed the process up. So I decided to test that theory with an eye on the results. I was surprised to see that the results when running "gmake -j 8 -k check" differ from a simple "gmake -k check" in the area of go and libmudflap :
1 ) parallel testsuite :
http://gcc.gnu.org/ml/gcc-testresults/2012-12/msg01903.html
2 ) single process testsuite :
http://gcc.gnu.org/ml/gcc-testresults/2012-12/msg01935.html
In parallel we see this for "go" :
=== go tests ===
Running target unix
FAIL: go.test/test/stack.go execution, -O2 -g
=== go Summary ===
# of expected passes 3293
# of unexpected failures 1
# of expected failures 4
# of untested testcases 4
/home/dclarke/pgm/build/gcc-4.7.2_2.6.32-279.14.1.el6.x86_64.004/gcc/testsuite/go/../../gccgo version 4.7.2 (Blastwave.org Inc. time_rfc-3339=2012-12-20 11:23:25+00:00)
While the single process apporach reports :
=== go Summary ===
# of expected passes 3294
# of expected failures 4
# of untested testcases 4
/home/dclarke/pgm/build/gcc-4.7.2_2.6.32-279.14.1.el6.x86_64.004/gcc/testsuite/go/../../gccgo version 4.7.2 (Blastwave.org Inc. time_rfc-3339=2012-12-20 11:23:25+00:00)
Perhaps the go test "go.test/test/stack.go" is somehow sensitive to running on a busy machine? I am only guessing.
The "libmudflap tests" results are wildly different.
So at first glance it appears that the testsuite will in fact product different results if one chooses to use "gmake -j X" for X>1 but I am not sure why. Felt is was worth pointing out as maybe no one else has done the comparison.
Dennis
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: parallel testsuite results differ from a single task approach
2012-12-21 4:06 parallel testsuite results differ from a single task approach Dennis Clarke
@ 2012-12-21 13:01 ` Tim Prince
2012-12-21 14:56 ` Dennis Clarke
0 siblings, 1 reply; 3+ messages in thread
From: Tim Prince @ 2012-12-21 13:01 UTC (permalink / raw)
To: gcc-help
On 12/20/2012 11:06 PM, Dennis Clarke wrote:
> I gad received some good advice to try using gmake -j X -k check for some X > 1 when running a testsuite in order to speed the process up. So I decided to test that theory with an eye on the results. I was surprised to see that the results when running "gmake -j 8 -k check" differ from a simple "gmake -k check" in the area of go and libmudflap :
>
> 1 ) parallel testsuite :
>
> http://gcc.gnu.org/ml/gcc-testresults/2012-12/msg01903.html
>
> 2 ) single process testsuite :
>
> http://gcc.gnu.org/ml/gcc-testresults/2012-12/msg01935.html
>
>
> In parallel we see this for "go" :
>
> === go tests ===
>
>
> Running target unix
> FAIL: go.test/test/stack.go execution, -O2 -g
>
> === go Summary ===
>
> # of expected passes 3293
> # of unexpected failures 1
> # of expected failures 4
> # of untested testcases 4
> /home/dclarke/pgm/build/gcc-4.7.2_2.6.32-279.14.1.el6.x86_64.004/gcc/testsuite/go/../../gccgo version 4.7.2 (Blastwave.org Inc. time_rfc-3339=2012-12-20 11:23:25+00:00)
>
>
>
>
> While the single process apporach reports :
>
>
> === go Summary ===
>
> # of expected passes 3294
> # of expected failures 4
> # of untested testcases 4
> /home/dclarke/pgm/build/gcc-4.7.2_2.6.32-279.14.1.el6.x86_64.004/gcc/testsuite/go/../../gccgo version 4.7.2 (Blastwave.org Inc. time_rfc-3339=2012-12-20 11:23:25+00:00)
>
>
> Perhaps the go test "go.test/test/stack.go" is somehow sensitive to running on a busy machine? I am only guessing.
>
> The "libmudflap tests" results are wildly different.
>
> So at first glance it appears that the testsuite will in fact product different results if one chooses to use "gmake -j X" for X>1 but I am not sure why. Felt is was worth pointing out as maybe no one else has done the comparison.
>
> Dennis
>
>
Did you read any reports from recent years about problems with this? If
a test dupiicates itself while accessing the same file system, it may
create a kind of race condition.
If the additional failures are time-outs, it's to be expected on a "busy
machine." In particular, you may have multiple tests contending for a
single rotating disk. Number of cores, OS, and other factors will make
a large difference. How are we to know what you aren't telling?
--
Tim Prince
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: parallel testsuite results differ from a single task approach
2012-12-21 13:01 ` Tim Prince
@ 2012-12-21 14:56 ` Dennis Clarke
0 siblings, 0 replies; 3+ messages in thread
From: Dennis Clarke @ 2012-12-21 14:56 UTC (permalink / raw)
To: tprince; +Cc: gcc-help
> How are we to know what you aren't telling?
I am generally very detailed in that regard.
I think that I will refrain from running gmake -j X for X > ( number of cores available -2 )
and then the problem may go away.
Otherwise a single "gmake" is safest. Slow. But safe.
dc
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2012-12-21 14:56 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-12-21 4:06 parallel testsuite results differ from a single task approach Dennis Clarke
2012-12-21 13:01 ` Tim Prince
2012-12-21 14:56 ` Dennis Clarke
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).