Re: testsuite under wine

[Jacob Bachmeyer <jcb62281@gmail.com>]

[Bringing the DejaGnu list back into the discussion...]

Jacek Caban wrote:
> Hi all,
>
>
> I'm responsible for Wine changes that cause your problems. I'm also 
> CCing Eric, who is Wine console expert, maybe he has better ideas. 
> Eric, see [1] if you're interested in the context.
>
>
> Recent Wine versions implement Windows pseudo-consoles, see [2] for a 
> bit more details. It's generally Microsoft's semi-recent API that's 
> intended to be more compatible with POSIX than what was present in 
> previous versions of Windows. In theory, with that implemented, we 
> could just plug tty fds that we get from Unix and have Unix consoles 
> working using those Windows APIs.

I would expect that to work very well with MinGW.

> In practice, it's not quite as good as promised and we need some 
> tweaks to make it work nicely. We could improve those tweaks, but 
> there are architectural limitations that will come to play sooner or 
> later.

Of course; "not quite as good as promised" seems to be a typical 
experience with Microsoft and their APIs.

> > I think that the long-term solution is that Wine should properly honor
> > the TERM environment variable and not produce escape codes that the
> > declared terminal does not support.
>
>
> I think that it would not be enough. The way Windows consoles work is 
> that we manage complete internal screen buffer and emit output that 
> synchronizes the buffer with Unix terminal inside conhost.exe process. 
> It means that its output heavily processed and may be very different 
> from what application writes to its console handle. While escape codes 
> discussed in this thread are the most prominent difference (and that 
> part could, in theory, be improved on our side), there are more 
> differences. For example, if application writes "\rA\rB\rC", conhost 
> will process it, update its internal buffer which changes just one 
> character and cursor position, and emit sequence to update it in Unix 
> terminal, which could be just "\rC" (or even "C" if cursor was already 
> at the beginning of the line). Another example would be long lines: 
> conhost will emit additional EOLs instead of depending on embedder to 
> wrap the line.

So conhost is essentially a Wine-specific screen(1) in that sense, 
except that it translates Windows screen buffer manipulations instead of 
VT100 escape codes?  As I understand ncurses also implements most of 
this; perhaps simply delegating output to ncurses would solve the 
problem?  If output were simply delegated to ncurses, (as I understand) 
setting TERM=dumb should be effective to eliminate escape codes from the 
output, since the "dumb" terminal does not support them.

Alternately, could we have a "transparent" mode in conhost where most 
processing is bypassed?  Setting TERM=dumb in the environment could 
reasonably activate this mode, or some other Wine-specific setting could 
be used.  (maybe "WINETERM=raw"?)

> I'm not really familiar with DejaGnu, but if you want to match 
> application output, that's probably not what you're looking for.

DejaGnu is mostly designed to drive text-mode interactive programs 
through simulated user interactions, for which Expect is extremely 
useful, although the tests at issue here are simpler than this general 
case.  Here, the compiler under test is used to build a test program, 
which is then run, its output collected, and that collected output 
compared to an expected pattern.  In Windows terms (as I remember), this 
is equivalent to writing out a batch file that compiles a test program, 
then runs the test program with output redirected to a temporary file, 
running that batch file, and reading/checking the output file.  POSIX 
allows DejaGnu to dispense with the batch file and the output temporary 
file.

(If you think these issues are fun, wait until someone tries to test a 
MinGW port of GDB...)

> The reason the previous workaround of compiling Wine without ncurses 
> worked is that if made Wine treat tty stdin/stdout in a way very 
> similar to regular pipes, so no extra processing was performed. This 
> was more of a side effect than a design choice. It should be possible 
> to provide some way to achieve that with the new Wine architecture. 
> I'm attaching an example of Wine patch that would allow it. With that 
> patch, you may disable conhost.exe (via winecfg or 
> WINEDLLOVERRIDES=conhost.exe=d environment variable) and achieve 
> something similar to previous workaround.

If Wine makes proper use of ncurses, then setting TERM=dumb should have 
the effect of eliminating escape codes from the output.  Why does it not 
do so?

> Long term, I think that it would be best to get rid of console 
> behaviour expectations by using Windows build of DejaGnu. My 
> understanding is that it requires Cygwin, so the stack would be: 
> Windows DejaGnu on Cygwin on Wine on Linux. This would make all 
> similar mismatches in expectations non-existent. Cygwin is tricky to 
> run on Wine, there are a few known problems like [3], but we're 
> getting there.

That could also introduce new problems, since it would be relying on 
Expect's Windows port.  (DejaGnu happens to work on Windows because 
Expect is portable to Windows, or at least Cygwin.)  There are some 
ongoing efforts to reduce gratuitous POSIX dependencies in DejaGnu, i.e. 
to use Tcl's portability aids where appropriate, mostly because using 
[file join $a $b $c] (for example) clearly indicates that $a, $b, and $c 
are file names, while simply [join [list $a $b $c] "/"] or "$a/$b/$c" 
says nothing about the meaning of the operation.

> Jacek
>
>
> [1] https://gcc.gnu.org/pipermail/fortran/2022-December/058645.html
>
> [2] 
> https://devblogs.microsoft.com/commandline/windows-command-line-introducing-the-windows-pseudo-console-conpty/ 
>
>
> [3] https://bugs.winehq.org/show_bug.cgi?id=47808


-- Jacob