DejaGnu/GCC testsuite behavior regarding multiple 'dg-do'

[Thomas Schwinge <thomas@codesourcery.com>]

Hi!

Comparing DejaGnu/GCC testsuite '*.sum' files between two systems ("old"
vs. "new") that ought to return identical results, I found that they
didn't:

    @@ -75032,6 +75023,7 @@ PASS: g++.dg/expr/bitfield4.C  -std=c++98 (test for excess errors)
     PASS: g++.dg/expr/bitfield5.C  -std=c++14  (test for warnings, line 12)
     PASS: g++.dg/expr/bitfield5.C  -std=c++14  (test for warnings, line 16)
     PASS: g++.dg/expr/bitfield5.C  -std=c++14 (test for excess errors)
    +PASS: g++.dg/expr/bitfield5.C  -std=c++14 execution test
     PASS: g++.dg/expr/bitfield5.C  -std=c++17  (test for errors, line 12)
     PASS: g++.dg/expr/bitfield5.C  -std=c++17  (test for errors, line 16)
     PASS: g++.dg/expr/bitfield5.C  -std=c++17 (test for excess errors)
    @@ -75041,6 +75033,7 @@ PASS: g++.dg/expr/bitfield5.C  -std=c++2a (test for excess errors)
     PASS: g++.dg/expr/bitfield5.C  -std=c++98  (test for warnings, line 12)
     PASS: g++.dg/expr/bitfield5.C  -std=c++98  (test for warnings, line 16)
     PASS: g++.dg/expr/bitfield5.C  -std=c++98 (test for excess errors)
    +PASS: g++.dg/expr/bitfield5.C  -std=c++98 execution test
     PASS: g++.dg/expr/bitfield6.C  -std=c++14  (test for warnings, line 10)
     PASS: g++.dg/expr/bitfield6.C  -std=c++14 (test for excess errors)
     PASS: g++.dg/expr/bitfield6.C  -std=c++17  (test for errors, line 10)

..., and several more like that.

'g++.dg/expr/bitfield5.C':

    // PR c++/30274
    // { dg-do run { target c++14_down } }
    // { dg-do compile { target c++17 } }
    [...]
      s.x++; // { dg-warning "5:use of an operand of type .bool. in .operator\\+\\+. is deprecated" "" { target { ! c++17 } } }
      // { dg-error "forbidden" "" { target c++17 } .-1 }
    [...]

So, this is meant to be an execution test for C++14 and older, and a
compile test for C++17 and newer.  This pattern seems to be recognized on
the "new" system, but not on the "old" system, where execution tests
aren't being run, only compile tests.

The "old" system, powerpc64le-unknown-linux-gnu Ubuntu 14.04, has DejaGnu
1.5-3ubuntu1.  Also reproduced on a x86_64-pc-linux-gnu Ubuntu 12.10
system with DejaGnu 1.5-3.

The "new" system, powerpc64le-unknown-linux-gnu Ubuntu 18.04, has DejaGnu
1.6.1-1.  Also reproduced on a x86_64-pc-linux-gnu Ubuntu 18.04 system
with DejaGnu 1.6.1-1.  And: also reproduced on the "old" system, when
instead of system-provided DejaGnu 1.5-3ubuntu1 manually running with
Ubuntu 18.04 DejaGnu 1.6.1-1.

Relevant (but haven't verified) seems to be the following DejaGnu commit:

    commit 569f8718b534a2cd9511a7d640352eb0126ff492
    Author: Dominik Vogt <vogt@linux.vnet.ibm.com>
    Date:   Mon Mar 28 17:31:07 2016 +1100

            * dg.exp (dg-do): Do not change the previously selected action if
            a de-selected dg-do is encountered.

Discussed in
<http://mid.mail-archive.com/20160308132910.GA30210@linux.vnet.ibm.com>,
and this indeed got included for DejaGnu 1.6.

Looking through the GCC testuite for test case files with more than one
'dg-do', the confusing thing is that this pattern has already been used
for a very long time.  See, for example, commit
5bdf05c8743e7486521ce3a3981ac3e6e7850ad0 (r143350, 2009-01-13):

    -/* { dg-do run { target powerpc*-*-* } } */
    +/* { dg-do run { target { powerpc*-*-* && vmx_hw } } } */
    +/* { dg-do compile { target { powerpc*-*-* && { ! vmx_hw } } } } */

I have not found any evidence in DejaGnu master branch that this not
working would've been a "recent" DejaGnu regression (and then fixed for
DejaGnu 1.6) -- so do we have to assume that this never worked as
intended back then?

I have not found an easy way to express different 'do-what-keyword'
within one test case file that work with old and new DejaGnu.

Per our "Prerequisites for GCC" installation documentation, we currently
require DejaGnu 1.4.4.  Advancing that to 1.6 is probably out of
question, given that it has "just" been released (four years ago).

As the failure mode with old DejaGnu is "benign" (only causes missing
execution testing), we could simply move on, and accept non-reproducible
results between different DejaGnu versions?  Kind of lame...  ;-|


Grüße
 Thomas
-----------------
Mentor Graphics (Deutschland) GmbH, Arnulfstraße 201, 80634 München / Germany
Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Alexander Walter