Re: testsuite under wine

[Jacob Bachmeyer <jcb62281@gmail.com>]

NightStrike wrote:
> On Sun, Dec 18, 2022 at 11:29 PM Jacob Bachmeyer <jcb62281@gmail.com> wrote:
>   
>> NightStrike wrote:
>>     
>>> On Sat, Dec 17, 2022 at 10:44 PM Jacob Bachmeyer <jcb62281@gmail.com> wrote:
>>>
>>>       
>>>> [...]
>>>> This is either a testsuite problem or an environment problem.  The GNU
>>>> Fortran I/O module certainly has interesting behavior here.  Try setting
>>>> TERM=dumb in the environment while running the testsuite.  If that fixes
>>>> the problem, it may be appropriate to add "set ::env(TERM) dumb" to the
>>>> tool init file for GNU Fortran.
>>>>
>>>>         
>>> Setting TERM doesn't help.  Wine tries to emulate the windows console,
>>> which requires outputting this stuff.  It does so any time there's a
>>> pty, and I believe that Deja creates a pty when running the tests.
>>>
>>>       
>> That is a bug in Wine:  the escapes should be suppressed entirely if the
>> terminal does not support them---and the "dumb" terminal does not
>> support them.
>>     
>
> I think it's a paradigm difference.  I'm just guessing here, but Wine
> runs in a terminal that doesn't know about TERM.

Terminals never know about TERM.  TERM exists to inform programs what 
terminal type they are running with.  X11 terminal emulators *set* TERM 
in the environment given to their child processes to identify themselves 
for this reason.

> It's mimicking the "cmd.exe" that you'd run on a native Windows system (I think... I'd welcome corrections from those more knowledgeable).

Yes, but Wine must emulate a console window with the terminal that it 
has, which is not necessarily a VT100-alike, therefore it should be 
checking the TERM variable at the minimum, and preferably using terminfo 
to adapt whatever escape sequences the terminal in use actually expects, 
or gracefully degrade when the terminal simply does not support the 
needed features.

(Minor technical note:  while cmd.exe is a console program, it does not 
actually provide the console; that is a different part of the NT system, 
but I am not certain exactly what.  Wine is trying to emulate the 
Windows console, not cmd.exe.)

> In theory, the effect would be the same if I set up a remote target board to ssh to a windows system and get dropped into a cmd.exe shell (I used to run this way... it'd take about a week to run the whole testsuite. It sucked...)
>   

Ah yes, Windows has some serious performance problems related to 
spawning processes.  :-D  I once cross-compiled GNU libc from a Windows 
(98SE) host while setting up my first GNU/Linux system.  It took about 
three days and the machine needed a universal Windows fix-it (reboot) 
afterwards due to memory leaks resulting in swap thrashing that 
continued even after all application programs had been closed---just a 
desktop, disk light on almost solidly, mouse pointer lagging mouse 
movement visibly.  Later, on the same hardware but using a Linux kernel, 
rebuilding glibc took the expected two hours.  Also, the same hardware 
semi-regularly locked up solid (even the mouse pointer would not move) 
with Windows... and not even once with GNU/Linux/XFree86.  [...snip 
extended rant...]

> [...]
>>> Note that there are other problems, too.  It seems that when Deja is
>>> matching against "\n", it doesn't handle the different line endings of
>>> Windows correctly in a cross environment.  Is there a way that I can
>>> set how to interpret \n in a target-board file?  This affects fortran
>>> and other language tests also.
>>>       
>> No---problems related to line endings are bugs in the testsuite.  This
>> caveat is documented in *Note: (dejagnu)Writing a test case.  The manual
>> explains:  "Note that terminal settings may result in the insertion of
>> additional `\r' characters, usually translating `\n' to `\r\n'."
>>
>> At the terminal layer, POSIX can *also* use "\r\n" sequences, since some
>> terminals historically needed them, even though the standard line ending
>> *within* a POSIX system is "\n" by itself.  Because a pty simply
>> presents the "terminal" side of the interface to the controlling
>> program, Expect can receive "\r\n" when the subprocess emits "\n"; the
>> translation is performed by the kernel terminal driver and DejaGnu
>> testsuites must be prepared to receive (and discard) excess carriage
>> returns in the general case.
>>     
>
> Here's one that tries to handle different line endings (most tests do
> not do this):
>
> gfortran.dg/parameter_array_dummy.f90
>
> which uses:
> ! { dg-output " *1 aa(\n|\r\n|\r)" }
> ! { dg-output " *2 ab(\n|\r\n|\r)" }
> ! { dg-output " *3 aaab(\n|\r\n|\r)" }
> ! { dg-output " *4 abaa(\n|\r\n|\r)" }
> ! { dg-output " *5 ababab(\n|\r\n|\r)" }
>
> But this results in:
>
> FAIL: gfortran.dg/parameter_array_dummy.f90   -O0  output pattern test
> [...]
>
> The problem being that we are getting "0x0d 0x0d 0x0a", or \r\r\n.
>   

These are testsuite bugs.

This detail also (finally) tells me that the framework /is/ involved 
here after all, so there are possibilities for enhancing lib/dg.exp to 
reduce these issues.  One possibility could be to use "$" in dg-output 
patterns as an end-of-line marker, internally substituted with a known 
EOL pattern for the platform being tested.  (Not yet implemented, and 
testsuites using it would not be able to run with older framework 
versions.)  Another solution is to note that dg-output matching does 
*not* use Expect and is actually batch-oriented:  the test program is 
run (and its output gathered) using ${tool}_load, and then the patterns 
are matched against that returned output.  This means that the Expect 
caveats described in the manual for normal tests do not apply here, and 
a simple EOL subpattern is usable:  "[\r\n]+" (using that subpattern 
with Expect *will* cause subtle timing-related bugs)

> (Other examples fail differently, for instance there's a Rust test
> that outputs \r\r\n\n... but let's start with this one).
>   

I believe that dg-output could reliably match that with an "[\r\n]+" 
subpattern, too.  :-)

One last detail that might help you:  ${tool}_load has a chance to apply 
regsub(n) to the output before returning it.  If the testsuite should 
ignore escape sequences, and you can match them with a Tcl regexp, you 
could use a similar conditional as my previous suggestion to prune the 
output before dg.exp examines it.


-- Jacob