[Bug testsuite/115460] New: gcc test results deviations

[Bug testsuite/115460] New: gcc test results deviations

[sadineniharish8446 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115460

            Bug ID: 115460
           Summary: gcc test results deviations
           Product: gcc
           Version: 13.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: testsuite
          Assignee: unassigned at gcc dot gnu.org
          Reporter: sadineniharish8446 at gmail dot com
  Target Milestone: ---

I'am using gcc-13.2.0, I have given two consecutive builds with make check-gcc
and compared test results, i have observed following deviations: 

Result from first test build:
 WARNING: program timed out                                            
 FAIL: gcc.target/i386/avx512bw-pr94509-2.c (test for excess errors)   
 UNRESOLVED: gcc.target/i386/avx512bw-pr94509-2.c compilation failed to

Result from second test build: 
 PASS: gcc.target/i386/avx512bw-pr94509-2.c (test for excess errors)
 PASS: gcc.target/i386/avx512bw-pr94509-2.c execution test

In yocto we are using gcc-13.2.0 and after running gcc testsuite and comparing
results we have observed a lot of test result deviations :
for example, test passed in first iteration is moving to unresolved in the next
iteration.

 ptestresult.gcc.gcc.dg/20061026.c (test for excess errors): PASS -> UNRESOLVED
        ptestresult.gcc.gcc.dg/20081223-1.c  (test for errors, line 5): PASS ->
UNRESOLVED
        ptestresult.gcc.gcc.dg/Warray-bounds-6.c (test for excess errors): PASS
-> UNRESOLVED
        ptestresult.gcc.gcc.dg/asm-4.c (test for excess errors): PASS ->
UNRESOLVED
        ptestresult.gcc.gcc.dg/attr-access-read-write-2.c  (test for warnings,
line 18): PASS -> UNRESOLVED
        ptestresult.gcc.gcc.dg/bitfld-10.c  (test for errors, line 8): PASS ->
UNRESOLVED
        ptestresult.gcc.gcc.dg/c2x-complit-4.c: UNRESOLVED -> UNSUPPORTED
        ptestresult.gcc.gcc.dg/c2x-float-5.c: UNRESOLVED -> UNSUPPORTED
        ptestresult.gcc.gcc.dg/c2x-stdarg-5.c (test for excess errors): PASS ->
UNRESOLVED
        ptestresult.gcc.gcc.dg/c99-condexpr-1.c (test for excess errors): PASS
-> UNRESOLVED
        ptestresult.gcc.gcc.dg/const-float80-ped.c  (test for warnings, line
5): PASS -> UNRESOLVED


 1)What can be the reason for this?
 2)How to resolve this kind of issues?

 thanks,
 Harish sadineni

[Bug testsuite/115460] gcc test results deviations

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115460

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2024-06-12
             Status|UNCONFIRMED                 |WAITING

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Without the full .log of the testsuite runs, it is hard to debug what is going
wrong. And I personally have not seen this. The only time I have (recently)
seen a timeout compiling has been when the compiler is miscompiled.

[Bug testsuite/115460] gcc test results deviations

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115460

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Note you want to look into the gcc.log file to understand why things are
turning into UNRESOLVED. the gcc.sum file is only a summary of what is going
on.

Also depending on the board you are using or how you are running the testsuite,
it could be that the ssh connection has been broken due to the machine
crashing. So you really should look into the gcc.log file to understand the
exact failure here.