For help:Unexpected fail about testsuite of GCC

For help:Unexpected fail about testsuite of GCC

[陈龙]

Hi,

 

I have run the  testsuite of GCC and compared with results from a similar configuration in the  gcc-testresults mailing list,  the results just have a little difference,and my test summary is in chapter Summary as below,. Both of the results have many unexpected fails, I want to know why they failed but the g++.log couldn’t affort enough fail case information,they were all almost described like this:'FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps'.So was it right with so many fails? And could you tell me how could I debug the fail case please?Thanks.

  

Test Environment




- x86_64-pc-mingw64 and msys2

- gcc8.1.0




Summary

               === g++ Summary ===

# of expected passes            115078
# of unexpected failures        628
# of unexpected successes       3
# of expected failures          493
# of unresolved testcases       5
# of unsupported tests          5122
/mingw64/bin/c++  version 8.1.0 (x86_64-posix-seh-rev0, Built by MinGW-W64 project)



Best regards

CL

Re: For help:Unexpected fail about testsuite of GCC

[U.Mutlu]

You can get a more detailed output, like in the following posting:
https://gcc.gnu.org/ml/gcc-testresults/2018-07/msg00453.html



陈龙 wrote on 07/04/2018 03:32 AM:
> Hi,
>
>
>
> I have run the  testsuite of GCC and compared with results from a similar configuration in the  gcc-testresults mailing list,  the results just have a little difference,and my test summary is in chapter Summary as below,. Both of the results have many unexpected fails, I want to know why they failed but the g++.log couldn’t affort enough fail case information,they were all almost described like this:'FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps'.So was it right with so many fails? And could you tell me how could I debug the fail case please?Thanks.
>
>
>
> Test Environment
>
>
>
>
> - x86_64-pc-mingw64 and msys2
>
> - gcc8.1.0
>
>
>
>
> Summary
>
>                 === g++ Summary ===
>
> # of expected passes            115078
> # of unexpected failures        628
> # of unexpected successes       3
> # of expected failures          493
> # of unresolved testcases       5
> # of unsupported tests          5122
> /mingw64/bin/c++  version 8.1.0 (x86_64-posix-seh-rev0, Built by MinGW-W64 project)
>
>
>
> Best regards
>
> CL
>
>

Re: For help:Unexpected fail about testsuite of GCC

[Jonathan Wakely]

On Wed, 4 Jul 2018 at 22:21, U.Mutlu <um@mutluit.com> wrote:
>
> You can get a more detailed output, like in the following posting:
> https://gcc.gnu.org/ml/gcc-testresults/2018-07/msg00453.html

Look in the logs, that's where the details are.

Re: For help:Unexpected fail about testsuite of GCC

[U.Mutlu]

As was said in the other posting, you need to create a log file
of the testing, and then you can analyse it (locate the failed test files etc.).
Here's how I do it on Linux.
After it finishes, see the file /tmp/my_tests.log for the details.


my_build_script.sh:
-------------------

nCores=$(getconf _NPROCESSORS_ONLN)
nJobs=$(( $nCores * 2 ))

#...

function do_tests()
{
   echo ""
   echo "do_tests: start"

   make -j $nJobs -k check
   #...

   echo "do_tests: end"
   echo ""
}

do_tests 2>&1 | tee /tmp/my_tests.log





U.Mutlu wrote on 07/04/2018 11:21 PM:
> You can get a more detailed output, like in the following posting:
> https://gcc.gnu.org/ml/gcc-testresults/2018-07/msg00453.html
>
>
>
> 陈龙 wrote on 07/04/2018 03:32 AM:
>> Hi,
>>
>>
>>
>> I have run the  testsuite of GCC and compared with results from a similar
>> configuration in the  gcc-testresults mailing list,  the results just have a
>> little difference,and my test summary is in chapter Summary as below,. Both
>> of the results have many unexpected fails, I want to know why they failed
>> but the g++.log couldn’t affort enough fail case information,they were all
>> almost described like this:'FAIL: g++.dg/pr80481.C  -std=gnu++14
>> scan-assembler-not vmovaps'.So was it right with so many fails? And could
>> you tell me how could I debug the fail case please?Thanks.
>>
>>
>>
>> Test Environment
>>
>>
>>
>>
>> - x86_64-pc-mingw64 and msys2
>>
>> - gcc8.1.0
>>
>>
>>
>>
>> Summary
>>
>>                 === g++ Summary ===
>>
>> # of expected passes            115078
>> # of unexpected failures        628
>> # of unexpected successes       3
>> # of expected failures          493
>> # of unresolved testcases       5
>> # of unsupported tests          5122
>> /mingw64/bin/c++  version 8.1.0 (x86_64-posix-seh-rev0, Built by MinGW-W64
>> project)
>>
>>
>>
>> Best regards
>>
>> CL
>>
>>
>
>
>

Re: For help:Unexpected fail about testsuite of GCC

[U.Mutlu]

Hello CL,

[as said, I'm new to the gcc testsuite, much like you, so take my answer with 
grain of salt, or so to say :-)]


In your original posting you had this failing example:
'FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps'

The test source file pr80481.C is in the source directory under:
   $ find ../gcc_trunk/ -iname "pr80481.C" -print
   ../gcc_trunk/gcc/testsuite/g++.dg/pr80481.C
and is handled by the make-target "check-c++" (see the log file of "make -k 
check"; you have to save that log file for reference & searching...).

So, now in the next step you have to make the following test that will
test just the "check-c++" category of the tests but with the additional
flags "RUNTESTFLAGS="-v -v" as it will give more verbosity in the log file:
   make -k check-c++ RUNTESTFLAGS="-v -v"
And again redirect the output into a log file for analysis.

It takes a long time (maybe one better should use the make -j
to use parallelization, ie. using all CPU cores for the tests).

The log file will give indication under which conditions that test fails.

I'm running this test myself now, it has not finished yet.
Here, you can find additional info (see under the section "Increase the 
verbosity":
https://gcc.gnu.org/wiki/Testing_GCC

Cheers,
Uenal



陈龙 wrote on 07/06/2018 04:34 AM:
> /Hi,Mutlu/
>
> /I'm glad to say you that I have gotten a command with DejaGNU for extend the
> error inforation,and I'm trying a new round of test now,and thank you very
> much for your help these days!/
>
> /Best regards,/
> /CL/
>
>
>
>
>
>
>
> At 2018-07-06 09:40:07, "陈龙" <18116491546@163.com> wrote:
>
>     Hi,
>
>     I already have a log but the fail information wasn't enough,I couldn't
>     locate the bug to fix. Could you give me some mannuals or documents
>     for  direction please?
>
>     Thank you very much for your help!
>
>
>     Best regards,
>     CL
>
>
>
>
>
>
>     At 2018-07-05 22:23:44, "U.Mutlu" <um@mutluit.com <mailto:um@mutluit.com>> wrote:
>     >As was said in the other posting, you need to create a log file
>     >of the testing, and then you can analyse it (locate the failed test files etc.).
>     >Here's how I do it on Linux.
>     >After it finishes, see the file /tmp/my_tests.log for the details.
>     >
>     >
>     >my_build_script.sh:
>     >-------------------
>     >
>     >nCores=$(getconf _NPROCESSORS_ONLN)
>     >nJobs=$(( $nCores * 2 ))
>     >
>     >#...
>     >
>     >function do_tests()
>     >{
>     >   echo ""
>     >   echo "do_tests: start"
>     >
>     >   make -j $nJobs -k check
>     >   #...
>     >
>     >   echo "do_tests: end"
>     >   echo ""
>     >}
>     >
>     >do_tests 2>&1 | tee /tmp/my_tests.log
>     >
>     >
>     >
>     >
>     >
>     >U.Mutlu wrote on 07/04/2018 11:21 PM:
>     >> You can get a more detailed output, like in the following posting:
>     >> https://gcc.gnu.org/ml/gcc-testresults/2018-07/msg00453.html
>     >>
>     >>
>     >>
>     >> 陈龙 wrote on 07/04/2018 03:32 AM:
>     >>> Hi,
>     >>>
>     >>>
>     >>>
>     >>> I have run the  testsuite of GCC and compared with results from a similar
>     >>> configuration in the  gcc-testresults mailing list,  the results just have a
>     >>> little difference,and my test summary is in chapter Summary as below,. Both
>     >>> of the results have many unexpected fails, I want to know why they failed
>     >>> but the g++.log couldn’t affort enough fail case information,they were all
>     >>> almost described like this:'FAIL: g++.dg/pr80481.C  -std=gnu++14
>     >>> scan-assembler-not vmovaps'.So was it right with so many fails? And could
>     >>> you tell me how could I debug the fail case please?Thanks.
>     >>>
>     >>>
>     >>>
>     >>> Test Environment
>     >>>
>     >>>
>     >>>
>     >>>
>     >>> - x86_64-pc-mingw64 and msys2
>     >>>
>     >>> - gcc8.1.0
>     >>>
>     >>>
>     >>>
>     >>>
>     >>> Summary
>     >>>
>     >>>                 === g++ Summary ===
>     >>>
>     >>> # of expected passes            115078
>     >>> # of unexpected failures        628
>     >>> # of unexpected successes       3
>     >>> # of expected failures          493
>     >>> # of unresolved testcases       5
>     >>> # of unsupported tests          5122
>     >>> /mingw64/bin/c++  version 8.1.0 (x86_64-posix-seh-rev0, Built by MinGW-W64
>     >>> project)
>     >>>
>     >>>
>     >>>
>     >>> Best regards
>     >>>
>     >>> CL
>     >>>
>     >>>
>     >>
>     >>
>     >>
>     >
>
> ★ 此邮件已被 *网易公正邮* <http://gzy.mail.163.com/?from=mailread> 保全存证，
> 发件人承诺与您诚信沟通！ ★

Re: For help:Unexpected fail about testsuite of GCC

[U.Mutlu]

> It takes a long time (maybe one better should use the make -j
> to use parallelization, ie. using all CPU cores for the tests).

You must definitely use "make -j ..." as it takes a very long time.
And the log file will be several hundereds MB big (>300 MB).

Re: For help:Unexpected fail about testsuite of GCC

[U.Mutlu]

U.Mutlu wrote on 07/06/2018 12:45 PM:
> Hello CL,
>
> [as said, I'm new to the gcc testsuite, much like you, so take my answer with
> grain of salt, or so to say :-)]
>
>
> In your original posting you had this failing example:
> 'FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps'
>
> The test source file pr80481.C is in the source directory under:
>    $ find ../gcc_trunk/ -iname "pr80481.C" -print
>    ../gcc_trunk/gcc/testsuite/g++.dg/pr80481.C
> and is handled by the make-target "check-c++" (see the log file of "make -k
> check"; you have to save that log file for reference & searching...).
>
> So, now in the next step you have to make the following test that will
> test just the "check-c++" category of the tests but with the additional
> flags "RUNTESTFLAGS="-v -v" as it will give more verbosity in the log file:
>    make -k check-c++ RUNTESTFLAGS="-v -v"
> And again redirect the output into a log file for analysis.
>
> It takes a long time (maybe one better should use the make -j
> to use parallelization, ie. using all CPU cores for the tests).
>
> The log file will give indication under which conditions that test fails.
>
> I'm running this test myself now, it has not finished yet.
> Here, you can find additional info (see under the section "Increase the
> verbosity":
> https://gcc.gnu.org/wiki/Testing_GCC


Ok, here are the first results of this bug hunting session:

The above said FAIL case seems to be related to these errors:

/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/g++.dg/spellcheck-identifiers-2.C:12:10: 
error: 'ssacnf' was not declared in this scope
/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/g++.dg/spellcheck-identifiers-2.C:32:10: 
error: 'sacnf' was not declared in this scope
/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/g++.dg/spellcheck-reswords.C:3:3: 
error: 'forget' was not declared in this scope
/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/g++.dg/spellcheck-reswords.C:9:3: 
error: 'memset' was not declared in this scope

as these errors happen between the following markers in the log:

  363945 Testing g++.dg/pr80481.C,  -std=gnu++14
  ...
  364936 FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps


...to be continued...

Re: For help:Unexpected fail about testsuite of GCC

[Jonathan Wakely]

On Fri, 6 Jul 2018 at 11:46, U.Mutlu <um@mutluit.com> wrote:
>
> Hello CL,
>
> [as said, I'm new to the gcc testsuite, much like you, so take my answer with
> grain of salt, or so to say :-)]
>
>
> In your original posting you had this failing example:
> 'FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps'
>
> The test source file pr80481.C is in the source directory under:
>    $ find ../gcc_trunk/ -iname "pr80481.C" -print
>    ../gcc_trunk/gcc/testsuite/g++.dg/pr80481.C
> and is handled by the make-target "check-c++" (see the log file of "make -k
> check"; you have to save that log file for reference & searching...).

Nope, see my previous email:
https://gcc.gnu.org/ml/gcc-help/2018-07/msg00020.html

> So, now in the next step you have to make the following test that will
> test just the "check-c++" category of the tests but with the additional
> flags "RUNTESTFLAGS="-v -v" as it will give more verbosity in the log file:
>    make -k check-c++ RUNTESTFLAGS="-v -v"
> And again redirect the output into a log file for analysis.

Look in $builddir/gcc/testsuite/g++/g++.log which already has the info
you need, and why it failed.

Re: For help:Unexpected fail about testsuite of GCC

[U.Mutlu]

Jonathan Wakely wrote on 07/06/2018 02:20 PM:
> On Fri, 6 Jul 2018 at 11:46, U.Mutlu <um@mutluit.com> wrote:
>>
>> Hello CL,
>>
>> [as said, I'm new to the gcc testsuite, much like you, so take my answer with
>> grain of salt, or so to say :-)]
>>
>>
>> In your original posting you had this failing example:
>> 'FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps'
>>
>> The test source file pr80481.C is in the source directory under:
>>     $ find ../gcc_trunk/ -iname "pr80481.C" -print
>>     ../gcc_trunk/gcc/testsuite/g++.dg/pr80481.C
>> and is handled by the make-target "check-c++" (see the log file of "make -k
>> check"; you have to save that log file for reference & searching...).
>
> Nope, see my previous email:
> https://gcc.gnu.org/ml/gcc-help/2018-07/msg00020.html
>
>> So, now in the next step you have to make the following test that will
>> test just the "check-c++" category of the tests but with the additional
>> flags "RUNTESTFLAGS="-v -v" as it will give more verbosity in the log file:
>>     make -k check-c++ RUNTESTFLAGS="-v -v"
>> And again redirect the output into a log file for analysis.
>
> Look in $builddir/gcc/testsuite/g++/g++.log which already has the info
> you need, and why it failed.

I see, thx, but there is only the following short log excerpt with
no indication of any "error: " entries for the said FAIL case.
Is this maybe caused by the fact that I used "make -j ...",
ie. is the log file maybe "garbled" by the many threads, and
the "error: " entries are somewhere outside of this range below?


Testing g++.dg/pr80481.C,  -std=gnu++14
replacement dg-process-target: `{ target { i?86-*-* x86_64-*-* }  && { ! 
*-*-solaris* } }'
dg-process-target-1: `{target { i?86-*-* x86_64-*-* }  && { ! *-*-solaris* }}'
replacement dg-process-target: `{target i?86-*-* x86_64-*-*}'
dg-process-target-1: `{target i?86-*-* x86_64-*-*}'
selector_list: ` i?86-*-* x86_64-*-* ' 1
selector_expression: ` i?86-*-* x86_64-*-* ' 1
/data/sw/src/gcc_dev/my_build_dir_for_gcc/x86_64-linux-gnu/./libio/_G_config.h
/data/sw/src/gcc_dev/my_build_dir_for_gcc/x86_64-linux-gnu/libio/_G_config.h
/data/sw/src/gcc_dev/my_build_dir_for_gcc/libio/_G_config.h
/data/sw/src/gcc_dev/libio/_G_config.h
/data/sw/src/gcc_dev/my_build_dir_for_gcc/x86_64-linux-gnu/./libio/iostream.list
/data/sw/src/gcc_dev/my_build_dir_for_gcc/x86_64-linux-gnu/libio/iostream.list
/data/sw/src/gcc_dev/my_build_dir_for_gcc/libio/iostream.list
/data/sw/src/gcc_dev/libio/iostream.list
/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/libio/Makefile.in
/sw/src/gcc_dev/gcc_trunk/gcc/libio/Makefile.in
/sw/src/gcc_dev/gcc_trunk/libio/Makefile.in
/sw/src/gcc_dev/libio/Makefile.in
doing compile
Invoking the compiler as 
/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../xg++ 
-B/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../ 
/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/
Setting timeout to 300
Executing on host: 
/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../xg++ 
-B/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../ 
/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/g++.dg
spawn /data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../xg++ 
-B/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../ 
/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/g++.dg/pr80481.C -f
pid is 23149 -23149
waitres is 23149 exp8 0 0
output is  status 0
Checking pattern "sparc-*-sunos*" with x86_64-pc-linux-gnu
Checking pattern "alpha*-*-*" with x86_64-pc-linux-gnu
Checking pattern "hppa*-*-hpux*" with x86_64-pc-linux-gnu
Checking pattern "sparc-*-sunos*" with x86_64-pc-linux-gnu
Checking pattern "alpha*-*-*" with x86_64-pc-linux-gnu
Checking pattern "hppa*-*-hpux*" with x86_64-pc-linux-gnu
PASS: g++.dg/pr80481.C  -std=gnu++14 (test for excess errors)
FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps

Re:Re: For help:Unexpected fail about testsuite of GCC

[陈龙]

Hi,all
Thanks for the response.
I know we need analyse the log if we want to deal with the failures. But before that ,I want to confirm some things described in my first mail ,I bring them here as below:
 
>> I have run the  testsuite of GCC and compared with results from a similar configuration in the  gcc-testresults mailing list,  the results just have a little difference,and my test summary is in chapter Summary as below,. Both of the results have many unexpected fails, I want to know why they failed but the g++.log couldn’t affort enough fail case information,they were all almost described like this:'FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps'.So was it right with so many fails? And could you tell me how could I debug the fail case please?Thanks.
>>
>>
>>
>> Test Environment
>>
>>
>>
>>
>> - x86_64-pc-mingw64 and msys2
>>
>> - gcc8.1.0
>>
>>
>>
>>
>> Summary
>>
>>                 === g++ Summary ===
>>
>> # of expected passes            115078
>> # of unexpected failures        628
>> # of unexpected successes       3
>> # of expected failures          493
>> # of unresolved testcases       5
>> # of unsupported tests          5122
>> /mingw64/bin/c++  version 8.1.0 (x86_64-posix-seh-rev0, Built by MinGW-W64 project)

please look at the sentence with yellow background , my test got 628 unexpected  failures,which is a little of difference from the result with a similar configuration in the  gcc-testresults mailing list(refer to https://docs.philips.com/:f:/r/personal/long_chen_philips_com/Documents/g++log?csf=1&e=L28vsv). Is that mean my test is right even if there are so many unexpected failures?And could I use the latest gcc version(8.1.0)  before I correct these fails in my environment?



Best regards,
CL




At 2018-07-06 20:54:33, "U.Mutlu" <um@mutluit.com> wrote:
>Jonathan Wakely wrote on 07/06/2018 02:20 PM:
>> On Fri, 6 Jul 2018 at 11:46, U.Mutlu <um@mutluit.com> wrote:
>>>
>>> Hello CL,
>>>
>>> [as said, I'm new to the gcc testsuite, much like you, so take my answer with
>>> grain of salt, or so to say :-)]
>>>
>>>
>>> In your original posting you had this failing example:
>>> 'FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps'
>>>
>>> The test source file pr80481.C is in the source directory under:
>>>     $ find ../gcc_trunk/ -iname "pr80481.C" -print
>>>     ../gcc_trunk/gcc/testsuite/g++.dg/pr80481.C
>>> and is handled by the make-target "check-c++" (see the log file of "make -k
>>> check"; you have to save that log file for reference & searching...).
>>
>> Nope, see my previous email:
>> https://gcc.gnu.org/ml/gcc-help/2018-07/msg00020.html
>>
>>> So, now in the next step you have to make the following test that will
>>> test just the "check-c++" category of the tests but with the additional
>>> flags "RUNTESTFLAGS="-v -v" as it will give more verbosity in the log file:
>>>     make -k check-c++ RUNTESTFLAGS="-v -v"
>>> And again redirect the output into a log file for analysis.
>>
>> Look in $builddir/gcc/testsuite/g++/g++.log which already has the info
>> you need, and why it failed.
>
>I see, thx, but there is only the following short log excerpt with
>no indication of any "error: " entries for the said FAIL case.
>Is this maybe caused by the fact that I used "make -j ...",
>ie. is the log file maybe "garbled" by the many threads, and
>the "error: " entries are somewhere outside of this range below?
>
>
>Testing g++.dg/pr80481.C,  -std=gnu++14
>replacement dg-process-target: `{ target { i?86-*-* x86_64-*-* }  && { ! 
>*-*-solaris* } }'
>dg-process-target-1: `{target { i?86-*-* x86_64-*-* }  && { ! *-*-solaris* }}'
>replacement dg-process-target: `{target i?86-*-* x86_64-*-*}'
>dg-process-target-1: `{target i?86-*-* x86_64-*-*}'
>selector_list: ` i?86-*-* x86_64-*-* ' 1
>selector_expression: ` i?86-*-* x86_64-*-* ' 1
>/data/sw/src/gcc_dev/my_build_dir_for_gcc/x86_64-linux-gnu/./libio/_G_config.h
>/data/sw/src/gcc_dev/my_build_dir_for_gcc/x86_64-linux-gnu/libio/_G_config.h
>/data/sw/src/gcc_dev/my_build_dir_for_gcc/libio/_G_config.h
>/data/sw/src/gcc_dev/libio/_G_config.h
>/data/sw/src/gcc_dev/my_build_dir_for_gcc/x86_64-linux-gnu/./libio/iostream.list
>/data/sw/src/gcc_dev/my_build_dir_for_gcc/x86_64-linux-gnu/libio/iostream.list
>/data/sw/src/gcc_dev/my_build_dir_for_gcc/libio/iostream.list
>/data/sw/src/gcc_dev/libio/iostream.list
>/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/libio/Makefile.in
>/sw/src/gcc_dev/gcc_trunk/gcc/libio/Makefile.in
>/sw/src/gcc_dev/gcc_trunk/libio/Makefile.in
>/sw/src/gcc_dev/libio/Makefile.in
>doing compile
>Invoking the compiler as 
>/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../xg++ 
>-B/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../ 
>/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/
>Setting timeout to 300
>Executing on host: 
>/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../xg++ 
>-B/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../ 
>/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/g++.dg
>spawn /data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../xg++ 
>-B/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../ 
>/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/g++.dg/pr80481.C -f
>pid is 23149 -23149
>waitres is 23149 exp8 0 0
>output is  status 0
>Checking pattern "sparc-*-sunos*" with x86_64-pc-linux-gnu
>Checking pattern "alpha*-*-*" with x86_64-pc-linux-gnu
>Checking pattern "hppa*-*-hpux*" with x86_64-pc-linux-gnu
>Checking pattern "sparc-*-sunos*" with x86_64-pc-linux-gnu
>Checking pattern "alpha*-*-*" with x86_64-pc-linux-gnu
>Checking pattern "hppa*-*-hpux*" with x86_64-pc-linux-gnu
>PASS: g++.dg/pr80481.C  -std=gnu++14 (test for excess errors)
>FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps
>

Re: For help:Unexpected fail about testsuite of GCC

[U.Mutlu]

陈龙 wrote on 07/09/2018 04:19 AM:
> /Sorry ,the site
> //https://docs.philips.com/:f:/r/personal/long_chen_philips_com/Documents/g++log?csf=1&e=L28vsv// have
> a wrong direction,please ignore it, refer to
> //https://gcc.gnu.org/ml/gcc-testresults/2018-05/msg00583.html//./
>

 >     Is that mean my test is right even if there are so many unexpected
 >     failures?And could I use the latest gcc version(8.1.0)  before I correct
 >     these fails in my environment?/

Hi,
I had not the time to look further on the failing tests.
I'm just a user like yourself :-)

If you need just a working compiler then take a precompiled stable version 
from the repository of your linux distribution. It will be an older version, 
not the current 8.1/9 version.

The 8.1/9 version is the "developer version", ie. unfinished work-in-progress
version. This is intented for testers and gcc-developers (patch submitters etc.).

FYI: yesterday an official snapshot of v9 has been released by gccadmin:
https://gcc.gnu.org/ml/gcc/2018-07/msg00119.html

I would say yes, take the latest version, but if your program behaves 
differently than expected, then remember that it could be a compiler issue.

Cheers
U.Mutlu


> /Best regards,/
> /CL/
>
>
>
>
>
>
> At 2018-07-09 10:01:46, "陈龙" <18116491546@163.com> wrote:
>
>     /Hi,all/
>     /Thanks for the response./
>     /I know we need analyse the log if we want to deal with the failures. But
>     before that ,I want to confirm some things described in my first mail ,I
>     bring them here as below:/
>     //
>      >> I have run the  testsuite of GCC and compared with results from a
>     similar configuration in the  gcc-testresults mailing list,  the results
>     just have a little difference,and my test summary is in chapter Summary as
>     below,. Both of the results have many unexpected fails, I want to know why
>     they failed but the g++.log couldn’t affort enough fail case
>     information,they were all almost described like this:'FAIL:
>     g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps'.So was it
>     right with so many fails? And could you tell me how could I debug the fail
>     case please?Thanks.
>      >>
>      >>
>      >>
>      >> Test Environment
>      >>
>      >>
>      >>
>      >>
>      >> - x86_64-pc-mingw64 and msys2
>      >>
>      >> - gcc8.1.0
>      >>
>      >>
>      >>
>      >>
>      >> Summary
>      >>
>      >>                 === g++ Summary ===
>      >>
>      >> # of expected passes            115078
>      >> # of unexpected failures        628
>      >> # of unexpected successes       3
>      >> # of expected failures          493
>      >> # of unresolved testcases       5
>      >> # of unsupported tests          5122
>      >> /mingw64/bin/c++  version 8.1.0 (x86_64-posix-seh-rev0, Built by
>     MinGW-W64 project)
>
>     /please look at the sentence with yellow background , my test got 628
>     unexpected  failures,which is a little of difference from the result with
>     a similar configuration in the  gcc-testresults mailing list(refer to
>     //https://docs.philips.com/:f:/r/personal/long_chen_philips_com/Documents/g++log?csf=1&e=L28vsv/
>     <https://docs.philips.com/:f:/r/personal/long_chen_philips_com/Documents/g++log?csf=1&e=L28vsv%20>/).
>     Is that mean my test is right even if there are so many unexpected
>     failures?And could I use the latest gcc version(8.1.0)  before I correct
>     these fails in my environment?/
>
>     /Best regards,/
>     /CL/
>
>
>     At 2018-07-06 20:54:33, "U.Mutlu" <um@mutluit.com <mailto:um@mutluit.com>> wrote:
>     >Jonathan Wakely wrote on 07/06/2018 02:20 PM:
>     >> On Fri, 6 Jul 2018 at 11:46, U.Mutlu <um@mutluit.com <mailto:um@mutluit.com>> wrote:
>     >>>
>     >>> Hello CL,
>     >>>
>     >>> [as said, I'm new to the gcc testsuite, much like you, so take my answer with
>     >>> grain of salt, or so to say :-)]
>     >>>
>     >>>
>     >>> In your original posting you had this failing example:
>     >>> 'FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps'
>     >>>
>     >>> The test source file pr80481.C is in the source directory under:
>     >>>     $ find ../gcc_trunk/ -iname "pr80481.C" -print
>     >>>     ../gcc_trunk/gcc/testsuite/g++.dg/pr80481.C
>     >>> and is handled by the make-target "check-c++" (see the log file of "make -k
>     >>> check"; you have to save that log file for reference & searching...).
>     >>
>     >> Nope, see my previous email:
>     >> https://gcc.gnu.org/ml/gcc-help/2018-07/msg00020.html
>     >>
>     >>> So, now in the next step you have to make the following test that will
>     >>> test just the "check-c++" category of the tests but with the additional
>     >>> flags "RUNTESTFLAGS="-v -v" as it will give more verbosity in the log file:
>     >>>     make -k check-c++ RUNTESTFLAGS="-v -v"
>     >>> And again redirect the output into a log file for analysis.
>     >>
>     >> Look in $builddir/gcc/testsuite/g++/g++.log which already has the info
>     >> you need, and why it failed.
>     >
>     >I see, thx, but there is only the following short log excerpt with
>     >no indication of any "error: " entries for the said FAIL case.
>     >Is this maybe caused by the fact that I used "make -j ...",
>     >ie. is the log file maybe "garbled" by the many threads, and
>     >the "error: " entries are somewhere outside of this range below?
>     >
>     >
>     >Testing g++.dg/pr80481.C,  -std=gnu++14
>     >replacement dg-process-target: `{ target { i?86-*-* x86_64-*-* }  && { !
>     >*-*-solaris* } }'
>     >dg-process-target-1: `{target { i?86-*-* x86_64-*-* }  && { ! *-*-solaris* }}'
>     >replacement dg-process-target: `{target i?86-*-* x86_64-*-*}'
>     >dg-process-target-1: `{target i?86-*-* x86_64-*-*}'
>     >selector_list: ` i?86-*-* x86_64-*-* ' 1
>     >selector_expression: ` i?86-*-* x86_64-*-* ' 1
>     >/data/sw/src/gcc_dev/my_build_dir_for_gcc/x86_64-linux-gnu/./libio/_G_config.h
>     >/data/sw/src/gcc_dev/my_build_dir_for_gcc/x86_64-linux-gnu/libio/_G_config.h
>     >/data/sw/src/gcc_dev/my_build_dir_for_gcc/libio/_G_config.h
>     >/data/sw/src/gcc_dev/libio/_G_config.h
>     >/data/sw/src/gcc_dev/my_build_dir_for_gcc/x86_64-linux-gnu/./libio/iostream.list
>     >/data/sw/src/gcc_dev/my_build_dir_for_gcc/x86_64-linux-gnu/libio/iostream.list
>     >/data/sw/src/gcc_dev/my_build_dir_for_gcc/libio/iostream.list
>     >/data/sw/src/gcc_dev/libio/iostream.list
>     >/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/libio/Makefile.in
>     >/sw/src/gcc_dev/gcc_trunk/gcc/libio/Makefile.in
>     >/sw/src/gcc_dev/gcc_trunk/libio/Makefile.in
>     >/sw/src/gcc_dev/libio/Makefile.in
>     >doing compile
>     >Invoking the compiler as
>     >/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../xg++
>     >-B/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../
>     >/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/
>     >Setting timeout to 300
>     >Executing on host:
>     >/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../xg++
>     >-B/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../
>     >/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/g++.dg
>     >spawn /data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../xg++
>     >-B/data/sw/src/gcc_dev/my_build_dir_for_gcc/gcc/testsuite/g++3/../../
>     >/sw/src/gcc_dev/gcc_trunk/gcc/testsuite/g++.dg/pr80481.C -f
>     >pid is 23149 -23149
>     >waitres is 23149 exp8 0 0
>     >output is  status 0
>     >Checking pattern "sparc-*-sunos*" with x86_64-pc-linux-gnu
>     >Checking pattern "alpha*-*-*" with x86_64-pc-linux-gnu
>     >Checking pattern "hppa*-*-hpux*" with x86_64-pc-linux-gnu
>     >Checking pattern "sparc-*-sunos*" with x86_64-pc-linux-gnu
>     >Checking pattern "alpha*-*-*" with x86_64-pc-linux-gnu
>     >Checking pattern "hppa*-*-hpux*" with x86_64-pc-linux-gnu
>     >PASS: g++.dg/pr80481.C  -std=gnu++14 (test for excess errors)
>     >FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps
>     >
>

Re: For help:Unexpected fail about testsuite of GCC

[U.Mutlu]

U.Mutlu wrote on 07/09/2018 11:48 AM:
> 陈龙 wrote on 07/09/2018 04:19 AM:
>> /Sorry ,the site
>> //https://docs.philips.com/:f:/r/personal/long_chen_philips_com/Documents/g++log?csf=1&e=L28vsv//
>> have
>> a wrong direction,please ignore it, refer to
>> //https://gcc.gnu.org/ml/gcc-testresults/2018-05/msg00583.html//./
>>
>
>  >     Is that mean my test is right even if there are so many unexpected
>  >     failures?And could I use the latest gcc version(8.1.0)  before I correct
>  >     these fails in my environment?/
>
> Hi,
> I had not the time to look further on the failing tests.
> I'm just a user like yourself :-)
>
> If you need just a working compiler then take a precompiled stable version
> from the repository of your linux distribution. It will be an older version,
> not the current 8.1/9 version.
>
> The 8.1/9 version is the "developer version", ie. unfinished work-in-progress
> version. This is intented for testers and gcc-developers (patch submitters etc.).
>
> FYI: yesterday an official snapshot of v9 has been released by gccadmin:
> https://gcc.gnu.org/ml/gcc/2018-07/msg00119.html
>
> I would say yes, take the latest version, but if your program behaves
> differently than expected, then remember that it could be a compiler issue.
>
> Cheers
> U.Mutlu

Correction:
   GCC 8.1 released [2018-05-02]
   GCC 9 has not officially been released yet, IMO, just pre-releases, ie. 
snapshots/betas/svn/git.
see also https://gcc.gnu.org/

Re:Re: For help:Unexpected fail about testsuite of GCC

[陈龙]

Hi,


It is a great pleasure for me to got your help a while ago.Thanks.


I got new questions about gcc version8.1.0 testsuite failure cases when I analyzed the log, there were some description I couldn't understand, please look at it below:


Testing g++.dg/pr80481.C,  -std=gnu++98
replacement dg-process-target: `{ target { i?86-*-* x86_64-*-* }  && { ! *-*-solaris* } }'
dg-process-target-1: `{target { i?86-*-* x86_64-*-* }  && { ! *-*-solaris* }}'
replacement dg-process-target: `{target i?86-*-* x86_64-*-*}'
dg-process-target-1: `{target i?86-*-* x86_64-*-*}'
selector_list: ` i?86-*-* x86_64-*-* ' 1
selector_expression: ` i?86-*-* x86_64-*-* ' 1
C:/msys64/mingw64/././libio/_G_config.h
/c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/C:/msys64/../libio/_G_config.h
/c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/C:/../../libio/_G_config.h
/c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/../../../libio/_G_config.h
/c/Users/.../Desktop/work/test/gcc-8.1.0/../../../../libio/_G_config.h
C:/msys64/mingw64/././libio/iostream.list
/c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/C:/msys64/../libio/iostream.list
/c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/C:/../../libio/iostream.list
/c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/../../../libio/iostream.list
/c/Users/.../Desktop/work/test/gcc-8.1.0/../../../../libio/iostream.list
./testsuite/./libio/Makefile.in
/c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/../libio/Makefile.in
/c/Users/.../Desktop/work/test/gcc-8.1.0/../../libio/Makefile.in
/c/Users/.../Desktop/work/test/../../../libio/Makefile.in
/c/Users/.../Desktop/work/../../../../libio/Makefile.in
Choosing /mingw64/bin/c++
doing compile
Invoking the compiler as c++ ./testsuite/g++.dg/pr80481.C  -fno-diagnostics-show-caret -fdiagnostics-color=never   -fmessage-length=0 -std=gnu++98 -Ofast -funroll-loops -fopenmp -march=knl -ffat-lto-objects  -S -o pr80481.s
Setting timeout to 300
Executing on host: c++ ./testsuite/g++.dg/pr80481.C  -fno-diagnostics-show-caret -fdiagnostics-color=never   -fmessage-length=0 -std=gnu++98 -Ofast -funroll-loops -fopenmp -march=knl -ffat-lto-objects  -S -o pr80481.s    (timeout = 300)
spawn -ignore SIGHUP c++ ./testsuite/g++.dg/pr80481.C -fno-diagnostics-show-caret -fdiagnostics-color=never -fmessage-length=0 -std=gnu++98 -Ofast -funroll-loops -fopenmp -march=knl -ffat-lto-objects -S -o pr80481.s
pid is 172120 -172120
pid is -1
waitres is 172120 exp8 0 0
output is  status 0
Checking pattern "sparc-*-sunos*" with x86_64-pc-mingw64
Checking pattern "alpha*-*-*" with x86_64-pc-mingw64
Checking pattern "hppa*-*-hpux*" with x86_64-pc-mingw64
Checking pattern "sparc-*-sunos*" with x86_64-pc-mingw64
Checking pattern "alpha*-*-*" with x86_64-pc-mingw64
Checking pattern "hppa*-*-hpux*" with x86_64-pc-mingw64


PASS: g++.dg/pr80481.C  -std=gnu++98 (test for excess errors)
FAIL: g++.dg/pr80481.C  -std=gnu++98  scan-assembler-not vmovaps


Q1:Does the first highlight part mean target process is a list including i?86 and x86_64? but the second highlight part show some differernt processes pattern like 'sparc','alpha','hppa',so how could I confirm which architecture the testsuite use? 


Q2:The third highlight part shows different results which under the same running condition in general from the log,why is it?



Thanks very much for help!



Best regards,
CL






At 2018-07-06 20:20:32, "Jonathan Wakely" <jwakely.gcc@gmail.com> wrote:
>On Fri, 6 Jul 2018 at 11:46, U.Mutlu <um@mutluit.com> wrote:
>>
>> Hello CL,
>>
>> [as said, I'm new to the gcc testsuite, much like you, so take my answer with
>> grain of salt, or so to say :-)]
>>
>>
>> In your original posting you had this failing example:
>> 'FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps'
>>
>> The test source file pr80481.C is in the source directory under:
>>    $ find ../gcc_trunk/ -iname "pr80481.C" -print
>>    ../gcc_trunk/gcc/testsuite/g++.dg/pr80481.C
>> and is handled by the make-target "check-c++" (see the log file of "make -k
>> check"; you have to save that log file for reference & searching...).
>
>Nope, see my previous email:
>https://gcc.gnu.org/ml/gcc-help/2018-07/msg00020.html
>
>> So, now in the next step you have to make the following test that will
>> test just the "check-c++" category of the tests but with the additional
>> flags "RUNTESTFLAGS="-v -v" as it will give more verbosity in the log file:
>>    make -k check-c++ RUNTESTFLAGS="-v -v"
>> And again redirect the output into a log file for analysis.
>
>Look in $builddir/gcc/testsuite/g++/g++.log which already has the info
>you need, and why it failed.

Re: Re: For help:Unexpected fail about testsuite of GCC

[Jonathan Wakely]

On Tue, 24 Jul 2018 at 06:53, 陈龙 <18116491546@163.com> wrote:
>
> Hi,
>
>
> It is a great pleasure for me to got your help a while ago.Thanks.
>
>
> I got new questions about gcc version8.1.0 testsuite failure cases when I analyzed the log, there were some description I couldn't understand, please look at it below:
>
>
> Testing g++.dg/pr80481.C,  -std=gnu++98
> replacement dg-process-target: `{ target { i?86-*-* x86_64-*-* }  && { ! *-*-solaris* } }'
> dg-process-target-1: `{target { i?86-*-* x86_64-*-* }  && { ! *-*-solaris* }}'
> replacement dg-process-target: `{target i?86-*-* x86_64-*-*}'
> dg-process-target-1: `{target i?86-*-* x86_64-*-*}'
> selector_list: ` i?86-*-* x86_64-*-* ' 1
> selector_expression: ` i?86-*-* x86_64-*-* ' 1
> C:/msys64/mingw64/././libio/_G_config.h
> /c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/C:/msys64/../libio/_G_config.h
> /c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/C:/../../libio/_G_config.h
> /c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/../../../libio/_G_config.h
> /c/Users/.../Desktop/work/test/gcc-8.1.0/../../../../libio/_G_config.h
> C:/msys64/mingw64/././libio/iostream.list
> /c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/C:/msys64/../libio/iostream.list
> /c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/C:/../../libio/iostream.list
> /c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/../../../libio/iostream.list
> /c/Users/.../Desktop/work/test/gcc-8.1.0/../../../../libio/iostream.list
> ./testsuite/./libio/Makefile.in
> /c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/../libio/Makefile.in
> /c/Users/.../Desktop/work/test/gcc-8.1.0/../../libio/Makefile.in
> /c/Users/.../Desktop/work/test/../../../libio/Makefile.in
> /c/Users/.../Desktop/work/../../../../libio/Makefile.in
> Choosing /mingw64/bin/c++
> doing compile
> Invoking the compiler as c++ ./testsuite/g++.dg/pr80481.C  -fno-diagnostics-show-caret -fdiagnostics-color=never   -fmessage-length=0 -std=gnu++98 -Ofast -funroll-loops -fopenmp -march=knl -ffat-lto-objects  -S -o pr80481.s
> Setting timeout to 300
> Executing on host: c++ ./testsuite/g++.dg/pr80481.C  -fno-diagnostics-show-caret -fdiagnostics-color=never   -fmessage-length=0 -std=gnu++98 -Ofast -funroll-loops -fopenmp -march=knl -ffat-lto-objects  -S -o pr80481.s    (timeout = 300)
> spawn -ignore SIGHUP c++ ./testsuite/g++.dg/pr80481.C -fno-diagnostics-show-caret -fdiagnostics-color=never -fmessage-length=0 -std=gnu++98 -Ofast -funroll-loops -fopenmp -march=knl -ffat-lto-objects -S -o pr80481.s
> pid is 172120 -172120
> pid is -1
> waitres is 172120 exp8 0 0
> output is  status 0
> Checking pattern "sparc-*-sunos*" with x86_64-pc-mingw64
> Checking pattern "alpha*-*-*" with x86_64-pc-mingw64
> Checking pattern "hppa*-*-hpux*" with x86_64-pc-mingw64
> Checking pattern "sparc-*-sunos*" with x86_64-pc-mingw64
> Checking pattern "alpha*-*-*" with x86_64-pc-mingw64
> Checking pattern "hppa*-*-hpux*" with x86_64-pc-mingw64

> PASS: g++.dg/pr80481.C  -std=gnu++98 (test for excess errors)
> FAIL: g++.dg/pr80481.C  -std=gnu++98  scan-assembler-not vmovaps
>
>
> Q1:Does the first highlight part mean target process is a list including i?86 and x86_64?

I don't see any highlighting, this is a plain text email.

Those patterns come from the test file, not from your system. If your
system matches one of the patterns then the test will run.


> but the second highlight part show some differernt processes pattern like 'sparc','alpha','hppa',so how could I confirm which architecture the testsuite use?

Your system is x86_64-pc-mingw64.

The testsuite is comparing your target triplet (x86_64-pc-ming64) to
some other patterns to see if it matches. It doesn't mean your machine
is a sparc, alpha or hppa machine.


> Q2:The third highlight part shows different results which under the same running condition in general from the log,why is it?

I don't understand what you're asking.

The log shows that the "test for excess errors" part got a PASS,
meaning it compiled without errors. But the "scan-assembler-not
vmovaps" part got a FAIL, meaning that checking the generated code
failed. It is expected that the generated code does not contain the
vmovaps instruction, and apparently for your target that instruction
gets used.

Re:Re: Re: For help:Unexpected fail about testsuite of GCC

[陈龙]

 Sorry for my unclear expression .

 I have gotten much help from your description and now I know how to read the log correctly,thank you very much.


Best regards,
CL





At 2018-07-24 18:44:52, "Jonathan Wakely" <jwakely.gcc@gmail.com> wrote:
>On Tue, 24 Jul 2018 at 06:53, 陈龙 <18116491546@163.com> wrote:
>>
>> Hi,
>>
>>
>> It is a great pleasure for me to got your help a while ago.Thanks.
>>
>>
>> I got new questions about gcc version8.1.0 testsuite failure cases when I analyzed the log, there were some description I couldn't understand, please look at it below:
>>
>>
>> Testing g++.dg/pr80481.C,  -std=gnu++98
>> replacement dg-process-target: `{ target { i?86-*-* x86_64-*-* }  && { ! *-*-solaris* } }'
>> dg-process-target-1: `{target { i?86-*-* x86_64-*-* }  && { ! *-*-solaris* }}'
>> replacement dg-process-target: `{target i?86-*-* x86_64-*-*}'
>> dg-process-target-1: `{target i?86-*-* x86_64-*-*}'
>> selector_list: ` i?86-*-* x86_64-*-* ' 1
>> selector_expression: ` i?86-*-* x86_64-*-* ' 1
>> C:/msys64/mingw64/././libio/_G_config.h
>> /c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/C:/msys64/../libio/_G_config.h
>> /c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/C:/../../libio/_G_config.h
>> /c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/../../../libio/_G_config.h
>> /c/Users/.../Desktop/work/test/gcc-8.1.0/../../../../libio/_G_config.h
>> C:/msys64/mingw64/././libio/iostream.list
>> /c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/C:/msys64/../libio/iostream.list
>> /c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/C:/../../libio/iostream.list
>> /c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/../../../libio/iostream.list
>> /c/Users/.../Desktop/work/test/gcc-8.1.0/../../../../libio/iostream.list
>> ./testsuite/./libio/Makefile.in
>> /c/Users/.../Desktop/work/test/gcc-8.1.0/gcc/../libio/Makefile.in
>> /c/Users/.../Desktop/work/test/gcc-8.1.0/../../libio/Makefile.in
>> /c/Users/.../Desktop/work/test/../../../libio/Makefile.in
>> /c/Users/.../Desktop/work/../../../../libio/Makefile.in
>> Choosing /mingw64/bin/c++
>> doing compile
>> Invoking the compiler as c++ ./testsuite/g++.dg/pr80481.C  -fno-diagnostics-show-caret -fdiagnostics-color=never   -fmessage-length=0 -std=gnu++98 -Ofast -funroll-loops -fopenmp -march=knl -ffat-lto-objects  -S -o pr80481.s
>> Setting timeout to 300
>> Executing on host: c++ ./testsuite/g++.dg/pr80481.C  -fno-diagnostics-show-caret -fdiagnostics-color=never   -fmessage-length=0 -std=gnu++98 -Ofast -funroll-loops -fopenmp -march=knl -ffat-lto-objects  -S -o pr80481.s    (timeout = 300)
>> spawn -ignore SIGHUP c++ ./testsuite/g++.dg/pr80481.C -fno-diagnostics-show-caret -fdiagnostics-color=never -fmessage-length=0 -std=gnu++98 -Ofast -funroll-loops -fopenmp -march=knl -ffat-lto-objects -S -o pr80481.s
>> pid is 172120 -172120
>> pid is -1
>> waitres is 172120 exp8 0 0
>> output is  status 0
>> Checking pattern "sparc-*-sunos*" with x86_64-pc-mingw64
>> Checking pattern "alpha*-*-*" with x86_64-pc-mingw64
>> Checking pattern "hppa*-*-hpux*" with x86_64-pc-mingw64
>> Checking pattern "sparc-*-sunos*" with x86_64-pc-mingw64
>> Checking pattern "alpha*-*-*" with x86_64-pc-mingw64
>> Checking pattern "hppa*-*-hpux*" with x86_64-pc-mingw64
>
>> PASS: g++.dg/pr80481.C  -std=gnu++98 (test for excess errors)
>> FAIL: g++.dg/pr80481.C  -std=gnu++98  scan-assembler-not vmovaps
>>
>>
>> Q1:Does the first highlight part mean target process is a list including i?86 and x86_64?
>
>I don't see any highlighting, this is a plain text email.
>
>Those patterns come from the test file, not from your system. If your
>system matches one of the patterns then the test will run.
>
>
>> but the second highlight part show some differernt processes pattern like 'sparc','alpha','hppa',so how could I confirm which architecture the testsuite use?
>
>Your system is x86_64-pc-mingw64.
>
>The testsuite is comparing your target triplet (x86_64-pc-ming64) to
>some other patterns to see if it matches. It doesn't mean your machine
>is a sparc, alpha or hppa machine.
>
>
>> Q2:The third highlight part shows different results which under the same running condition in general from the log,why is it?
>
>I don't understand what you're asking.
>
>The log shows that the "test for excess errors" part got a PASS,
>meaning it compiled without errors. But the "scan-assembler-not
>vmovaps" part got a FAIL, meaning that checking the generated code
>failed. It is expected that the generated code does not contain the
>vmovaps instruction, and apparently for your target that instruction
>gets used.

Re: For help:Unexpected fail about testsuite of GCC

[Jonathan Wakely]

On Thu, 5 Jul 2018 at 15:23, U.Mutlu <um@mutluit.com> wrote:
>
> As was said in the other posting, you need to create a log file
> of the testing, and then you can analyse it (locate the failed test files etc.).
> Here's how I do it on Linux.

No it isn't. All you've done is capture the information printed to
standard output. That tells you what failed, but not why.

The testsuite creates the logs itself, you don't create them.

find `find $builddir -name testsuite -type d` -type f -name '*.log'

Re: For help:Unexpected fail about testsuite of GCC

[Jonathan Wakely]

On Wed, 4 Jul 2018 at 02:32, 陈龙 <18116491546@163.com> wrote:
>
> Hi,
>
>
>
> I have run the  testsuite of GCC and compared with results from a similar configuration in the  gcc-testresults mailing list,  the results just have a little difference,and my test summary is in chapter Summary as below,. Both of the results have many unexpected fails, I want to know why they failed but the g++.log couldn’t affort enough fail case information,they were all almost described like this:'FAIL: g++.dg/pr80481.C  -std=gnu++14  scan-assembler-not vmovaps'.

That means the test checks the generated assembly code and fails if
the vmovaps instruction appears.