Re: Problems with strub tests

[Hans-Peter Nilsson <hp@bitrange.com>]

On Tue, 19 Dec 2023, Jeff Law wrote:
> 
> So the strub tests in c-c++-common are problematical.  They get run twice,
> once for C, once for C++.  Yet the name of the test is the same in both runs.
> (by the name, I mean the name emitted into the dejagnu summary and log files).
> 
> Thus if you have a test in there which passes in one context, but fails in the
> other, comparison tools like contrib/compare_tests may erroneously report the
> tests as both a test which now fails, but passed before and a test which now
> passes but failed before.
> 
> It looks like some of the strub tests are currently known to fail with C++ and
> are triggering this problem
> 
> 
> Ideally we'd include the c or c++ in the test name depending on which context
> its being run within.  That would be sufficient to resolve these problems and
> avoid them in the future.  It would also be sufficient to get all the tests to
> the point where their behavior is the same for both languages.
> 
> Not sure if the latter is reasonably in the cards or not.  If it's not likely
> to land soon, any change you could look at the framework for c-c++-common and
> get the names unique across the two times they're run?
> 
> A third option would be to change the compare_tests tool to somehow
> distinguish between the C and C++ tests.  Not sure how feasible that is.

How about including the name of the .sum file in the key?

(They're gcc.sum and g++.sum thus different.  This is what 
contrib/regression/btest-gcc.sh does.  On the other hand, that 
prunes the name of the test at the first space.  Don't copy 
that bit. :)

Also not sure how feasible that is.

brgds, H-P