Re: parallel check output changes?

[Andrew MacLeod <amacleod@redhat.com>]

On 09/23/2014 11:33 AM, Richard Sandiford wrote:
> Segher Boessenkool<segher@kernel.crashing.org>  writes:
>> On Thu, Sep 18, 2014 at 01:44:55PM -0500, Segher Boessenkool wrote:
>>> I am testing a patch that is just
>>>
>>>
>>> diff --git a/contrib/dg-extract-results.py b/contrib/dg-extract-results.py
>>> index cccbfd3..3781423 100644
>>> --- a/contrib/dg-extract-results.py
>>> +++ b/contrib/dg-extract-results.py
>>> @@ -117,7 +117,7 @@ class Prog:
>>>           self.tool_re = re.compile (r'^\t\t=== (.*) tests ===$')
>>>           self.result_re = re.compile (r'^(PASS|XPASS|FAIL|XFAIL|UNRESOLVED'
>>>                                        r'|WARNING|ERROR|UNSUPPORTED|UNTESTED'
>>> -                                     r'|KFAIL):\s*(\S+)')
>>> +                                     r'|KFAIL):\s*(.+)')
>>>           self.completed_re = re.compile (r'.* completed at (.*)')
>>>           # Pieces of text to write at the head of the output.
>>>           # start_line is a pair in which the first element is a datetime
>> Tested that with four runs on powerpc64-linux, four configs each time;
>> test-summary
>> shows the same in all cases.  Many lines have moved compared to without
>> the patch, but that cannot be helped.  Okay for mainline?
>>
>>
>> 2014-09-19  Segher Boessenkool<segher@kernel.crashing.org>
>>
>> contrib/
>> 	* dg-extract-results.py (Prog.result_re): Include options in test name.
> FWIW, the \S+ thing was deliberate.  When one test is run multiple times
> with different options, those options aren't necessarily tried in
> alphabetical order.  The old sh/awk script therefore used just the test
> name as the key and kept tests with the same name in the order that
> they were encountered:
>
> /^(PASS|XPASS|FAIL|XFAIL|UNRESOLVED|WARNING|ERROR|UNSUPPORTED|UNTESTED|KFAIL):/ {
>    testname=\$2
>    # Ugly hack for gfortran.dg/dg.exp
>    if ("$TOOL" == "gfortran" && testname ~ /^gfortran.dg\/g77\//)
>      testname="h"testname
> }
>
> (note the "$2").  This means that the output of the script is in the same
> order as it would be for non-parallel runs.  I was following (or trying
> to follow) that behaviour in the python script.
>
> Your patch instead sorts based on the full test name, including options,
> which means that the output no longer matches what you'd get from a
> non-parallel run.  AFAICT, it also no longer matches what you'd get from
> the .sh version.  That might be OK, just thought I'd mention it.
>
> Thanks,
> Richard
>
Is this suppose to be resolved now?  I'm still seeing some issues with a 
branch cut from mainline from yesterday.   This is from the following 
sequence:

check out revision 215511 , build, make -j16 check, make -j16 check, 
then compare all the .sum files:

PASS: gcc.dg/tls/asm-1.c  (test for errors, line 7)
PASS: gcc.dg/tls/asm-1.c (test for excess errors)
PASS: gcc.dg/tls/debug-1.c (test for excess errors)
PASS: gcc.dg/tls/diag-1.c (test for excess errors)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 4)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 5)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 6)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 7)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 11)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 12)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 13)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 14)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 17)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 18)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 19)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 20)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 22)

and then
PASS: gcc.dg/tls/asm-1.c  (test for errors, line 7)
PASS: gcc.dg/tls/asm-1.c (test for excess errors)
PASS: gcc.dg/tls/debug-1.c (test for excess errors)
PASS: gcc.dg/tls/diag-1.c (test for excess errors)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 11)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 12)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 13)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 14)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 17)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 18)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 19)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 20)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 22)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 4)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 5)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 6)
PASS: gcc.dg/tls/diag-2.c  (test for errors, line 7)

it looks like the first time sorted by line numerically (or just 
happened to leave the run order)  and the second time did the sort 
alphabetically...

Andrew