[Bug testsuite/102122] New: contrib/compare_tests reports different result with identical sum input

[Bug testsuite/102122] New: contrib/compare_tests reports different result with identical sum input

[nickhuang99 at hotmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102122

            Bug ID: 102122
           Summary: contrib/compare_tests reports different result with
                    identical sum input
           Product: gcc
           Version: 11.1.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: testsuite
          Assignee: unassigned at gcc dot gnu.org
          Reporter: nickhuang99 at hotmail dot com
  Target Milestone: ---

Created attachment 51373
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51373&action=edit
identical sum file

if I compare two identical sum input, compare_tests reports:

Tests that now fail, but worked before (4 tests):

: gcc.dg/Wvla-parameter-2.c pr????? (test for warnings, line 40)
: gcc.dg/Wvla-parameter-2.c pr????? (test for warnings, line 42)
: gcc.dg/Wvla-parameter-2.c pr????? (test for warnings, line 44)
: gcc.dg/Wvla-parameter-2.c pr????? (test for warnings, line 46)

Tests that now work, but didn't before (4 tests):

: gcc.dg/Wvla-parameter-2.c pr????? (test for warnings, line 40)
: gcc.dg/Wvla-parameter-2.c pr????? (test for warnings, line 42)
: gcc.dg/Wvla-parameter-2.c pr????? (test for warnings, line 44)
: gcc.dg/Wvla-parameter-2.c pr????? (test for warnings, line 46)


What I suspect is the "XFAIL:" part confuses report.

[Bug testsuite/102122] contrib/compare_tests reports different result with identical sum input

[nickhuang99 at hotmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102122

--- Comment #1 from qingzhe huang <nickhuang99 at hotmail dot com> ---
I can see now this is maybe a testcase "gcc.dg/Wvla-parameter-2.c" issue: Do we
expect XFAIL and PASS happen at the same time?

For example, the input can be simplified as following:

XFAIL: gcc.dg/Wvla-parameter-2.c pr????? (test for warnings, line 40)
PASS: gcc.dg/Wvla-parameter-2.c pr????? (test for warnings, line 40)

I guess compare_tests doesn't expect such case can occur, right?

[Bug testsuite/102122] gcc.dg/Wvla-parameter-2.c tests don't have unique name to it

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102122

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2021-08-30
             Status|UNCONFIRMED                 |NEW
           Keywords|xfail                       |
            Summary|contrib/compare_tests       |gcc.dg/Wvla-parameter-2.c
                   |reports different result    |tests don't have unique
                   |with identical sum input    |name to it

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Right, the test summary line should be unique.

So the bug is with gcc.dg/Wvla-parameter-2.c.


See old PR 32076, PR 64221 and maybe a few others.

[Bug testsuite/102122] gcc.dg/Wvla-parameter-2.c tests don't have unique name to it

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102122

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |DUPLICATE

--- Comment #3 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
This was already fixed and is a dup of bug 100412.

*** This bug has been marked as a duplicate of bug 100412 ***