Spurious libstdc++ testsuite failures because of truncated buffered output

Spurious libstdc++ testsuite failures because of truncated buffered output

[Diego Novillo]

I am getting several failures in decimal/mixed-mode_neg.cc:

FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 195)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 196)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 197)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 198)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 199)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 200)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 201)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 202)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 203)

This test expects syntax errors in all these lines, and the compiler
is indeed flagging them but the output from the compiler is so big
that dejagnu truncates it.  The testsuite never sees the error
messages starting at line 195 (I can see the output truncated in
libstdc++.log).

Is there a way to increase dejagnu's buffering?  This does not happen,
if I simply run the build in a shallower build tree, but that is not a
viable alternative.

Any suggestions on how to address this, short of butchering this test?


Thanks.  Diego.

Re: Spurious libstdc++ testsuite failures because of truncated buffered output

[Paolo Carlini]

Hi
> I am getting several failures in decimal/mixed-mode_neg.cc:
>
> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 195)
> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 196)
> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 197)
> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 198)
> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 199)
> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 200)
> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 201)
> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 202)
> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 203)
>
> This test expects syntax errors in all these lines, and the compiler
> is indeed flagging them but the output from the compiler is so big
> that dejagnu truncates it.  The testsuite never sees the error
> messages starting at line 195 (I can see the output truncated in
> libstdc++.log).
>
> Is there a way to increase dejagnu's buffering?  This does not happen,
> if I simply run the build in a shallower build tree, but that is not a
> viable alternative.
Honestly, I know very little about the dejagnu buffering, but I'm having 
a quick look to the issue and I'm rather surprised that you are seeing 
it for these particular errors and nowhere else in the testsuite: here, 
on the libstdc++.log of x86_64-linux, I see just 9 candidates listed, 
and I'm pretty sure that in terms of error messages we have much, much, 
longer examples, when templates are involved in particular. What I'm 
missing?

Paolo.

Re: Spurious libstdc++ testsuite failures because of truncated buffered output

[Diego Novillo]

On Fri, Mar 18, 2011 at 11:30, Paolo Carlini <paolo.carlini@oracle.com> wrote:
> Hi
>>
>> I am getting several failures in decimal/mixed-mode_neg.cc:
>>
>> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 195)
>> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 196)
>> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 197)
>> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 198)
>> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 199)
>> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 200)
>> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 201)
>> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 202)
>> FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 203)
>>
>> This test expects syntax errors in all these lines, and the compiler
>> is indeed flagging them but the output from the compiler is so big
>> that dejagnu truncates it.  The testsuite never sees the error
>> messages starting at line 195 (I can see the output truncated in
>> libstdc++.log).
>>
>> Is there a way to increase dejagnu's buffering?  This does not happen,
>> if I simply run the build in a shallower build tree, but that is not a
>> viable alternative.
>
> Honestly, I know very little about the dejagnu buffering, but I'm having a
> quick look to the issue and I'm rather surprised that you are seeing it for
> these particular errors and nowhere else in the testsuite: here, on the
> libstdc++.log of x86_64-linux, I see just 9 candidates listed, and I'm
> pretty sure that in terms of error messages we have much, much, longer
> examples, when templates are involved in particular. What I'm missing?

I don't really know why the others don't fail, but I will take a look.
 In this particular case, the output is truncated after 1,032,134
bytes are printed.  So I can only speculate that no other tests
generate more than this number of bytes in output.  I'll take a look.


Diego.

Re: Spurious libstdc++ testsuite failures because of truncated buffered output

[Paolo Carlini]

On 03/18/2011 04:35 PM, Diego Novillo wrote:
> I don't really know why the others don't fail, but I will take a look.
>   In this particular case, the output is truncated after 1,032,134
> bytes are printed.  So I can only speculate that no other tests
> generate more than this number of bytes in output.  I'll take a look.
For sure if it's the *overall* number of bytes that matters, working 
around the issue is simple, because the testcase can be trivially split 
to 14 separate tests, one for each operation.

Paolo.

Re: Spurious libstdc++ testsuite failures because of truncated buffered output

[Diego Novillo]

On Fri, Mar 18, 2011 at 11:40, Paolo Carlini <paolo.carlini@oracle.com> wrote:
> On 03/18/2011 04:35 PM, Diego Novillo wrote:
>>
>> I don't really know why the others don't fail, but I will take a look.
>>  In this particular case, the output is truncated after 1,032,134
>> bytes are printed.  So I can only speculate that no other tests
>> generate more than this number of bytes in output.  I'll take a look.
>
> For sure if it's the *overall* number of bytes that matters, working around
> the issue is simple, because the testcase can be trivially split to 14
> separate tests, one for each operation.

Yeah, I was thinking of splitting the test in several files.  I'll
send a patch later, if you think that's a reasonable thing to do.


Thanks.  Diego.

Re: Spurious libstdc++ testsuite failures because of truncated buffered output

[Andreas Schwab]

Diego Novillo <dnovillo@google.com> writes:

> Is there a way to increase dejagnu's buffering?

expect(1):

       match_max [-d] [-i spawn_id] [size]
             defines the size of the buffer  (in  bytes)  used  internally  by
             expect.  With no size argument, the current size is returned.

             With  the -d flag, the default size is set.  (The initial default
             is 2000.)  With the -i flag, the size is set for the named  spawn
             id, otherwise it is set for the current process.

Andreas.

-- 
Andreas Schwab, schwab@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84  5EC7 45C6 250E 6F00 984E
"And now for something completely different."

Re: Spurious libstdc++ testsuite failures because of truncated buffered output

[Joseph S. Myers]

On Fri, 18 Mar 2011, Diego Novillo wrote:

> I don't really know why the others don't fail, but I will take a look.
>  In this particular case, the output is truncated after 1,032,134
> bytes are printed.  So I can only speculate that no other tests
> generate more than this number of bytes in output.  I'll take a look.

There are a couple of places in DejaGnu's remote.exp that compare numbers 
of bytes with 512000; try increasing those to a much bigger figure, say 
5120000.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Spurious libstdc++ testsuite failures because of truncated buffered output

[Diego Novillo]

On Fri, Mar 18, 2011 at 11:55, Andreas Schwab <schwab@redhat.com> wrote:
> Diego Novillo <dnovillo@google.com> writes:
>
>> Is there a way to increase dejagnu's buffering?
>
> expect(1):
>
>       match_max [-d] [-i spawn_id] [size]
>             defines the size of the buffer  (in  bytes)  used  internally  by
>             expect.  With no size argument, the current size is returned.
>
>             With  the -d flag, the default size is set.  (The initial default
>             is 2000.)  With the -i flag, the size is set for the named  spawn
>             id, otherwise it is set for the current process.

Thanks, but I don't think match_max affects this.  It's the output
from the compiler that is being truncated.  I modified conformance.exp
with a match_max value 10x the limit I need, but the failures are
still there:

Index: testsuite/libstdc++-dg/conformance.exp
===================================================================
--- testsuite/libstdc++-dg/conformance.exp      (revision 171068)
+++ testsuite/libstdc++-dg/conformance.exp      (working copy)
@@ -21,6 +21,8 @@
 # Initialization.
 dg-init

+match_max -d 10000000
+
 # Build the support objects.
 v3-build_support

Running <bld>/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp ...
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 196)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 197)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 198)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 199)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 200)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 201)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 202)
FAIL: decimal/mixed-mode_neg.cc  (test for errors, line 203)

The output is clearly truncated, right before the first 'PASS' result, I see:

[ ... ]/x86_64-unknown-linux-gnu/libstdc++-v3/include/decimal/decimal.h:83:3:
note: std::decimal::decimal32&
std::decimal::decimal32::operator/=(long int)^M
/usr/local/google/hom      <<<< TRUNCATED
PASS: decimal/mixed-mode_neg.cc  (test for errors, line 43)
PASS: decimal/mixed-mode_neg.cc  (test for errors, line 44)


Diego.

Re: Spurious libstdc++ testsuite failures because of truncated buffered output

[Diego Novillo]

On Fri, Mar 18, 2011 at 11:43, Joseph S. Myers <joseph@codesourcery.com> wrote:

> There are a couple of places in DejaGnu's remote.exp that compare numbers
> of bytes with 512000; try increasing those to a much bigger figure, say
> 5120000.

Thanks.  Given that my output is truncated at ~1Mb, I'm not sure if
this will help.  Even if it did, I really cannot touch remote.exp, as
this is installed in a system location that is not feasible for us to
modify.

I think my best bet will be to simply split this test case.


Diego.

Re: Spurious libstdc++ testsuite failures because of truncated buffered output

[Andreas Schwab]

Diego Novillo <dnovillo@google.com> writes:

> Thanks, but I don't think match_max affects this.  It's the output
> from the compiler that is being truncated.

I don't see that problem on powerpc-linux (and I couldn't find any such
failure on gcc-testresults).

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Spurious libstdc++ testsuite failures because of truncated buffered output

[Diego Novillo]

On Fri, Mar 18, 2011 at 14:05, Andreas Schwab <schwab@linux-m68k.org> wrote:
> Diego Novillo <dnovillo@google.com> writes:
>
>> Thanks, but I don't think match_max affects this.  It's the output
>> from the compiler that is being truncated.
>
> I don't see that problem on powerpc-linux (and I couldn't find any such
> failure on gcc-testresults).

It only happens in our builds that have very deep directory trees.
The extra long path names contribute to filling up whatever buffer
that dejagnu uses for output.  I believe that if you manage to build
inside a tree this long, you'll be able to reproduce the problem:

$ pwd -P | wc -c
236


Diego.