From: Jerry D <jvdelisle2@gmail.com>
To: jcb62281@gmail.com, NightStrike <nightstrike@gmail.com>
Cc: Jacek Caban <jacek@codeweavers.com>,
fortran@gcc.gnu.org, Eric Pouech <eric.pouech@orange.fr>,
"gcc@gcc.gnu.org" <gcc@gcc.gnu.org>,
dejagnu@gnu.org
Subject: Re: testsuite under wine
Date: Thu, 5 Jan 2023 19:44:41 -0800 [thread overview]
Message-ID: <0f560ae5-f829-b91c-b16c-0cad672a0b31@gmail.com> (raw)
In-Reply-To: <63B7967B.60502@gmail.com>
On 1/5/23 7:33 PM, Jacob Bachmeyer via Fortran wrote:
> NightStrike wrote:
>> On Fri, Dec 23, 2022 at 11:00 PM Jacob Bachmeyer <jcb62281@gmail.com>
>> wrote:
>>> NightStrike wrote:
>>>> On Wed, Dec 21, 2022 at 11:37 PM Jacob Bachmeyer
>>>> <jcb62281@gmail.com> wrote:
>>>>> [...]
>>>> So at least we know for sure that this particular instance of extra
>>>> characters is coming from Wine. Maybe Wine can be smart enough to
>>>> only translate \n into \r\n instead of translating \r\n into \r\r\n.
>>>> Jacek / Eric, comments here? I'm happy to try another patch, the
>>>> first one was great.
>>>>
>>> I doubt that Wine is doing that translation. MinGW libc produces output
>>> conformant to Windows conventions, so printf("\n") on a text handle
>>> emits "\r\n", which Wine passes along. POSIX convention is that "\n" is
>>> translated to "\r\n" in the kernel terminal driver upon output, so the
>>> kernel translates the "\n" in the "\r\n" into /another/ "\r\n", yielding
>>> "\r\r\n" at the pty master end. This is why DejaGnu testsuites must be
>>> prepared to discard excess carriage returns. The first CR came from
>>> MinGW libc; the second CR came from the kernel terminal driver; the LF
>>> was ultimately passed through.
>>
>> Jacek and I have been digging into this on IRC, and he's been very
>> helpful in trying to get further, but we're still stuck. We tried to
>> be more introspective, inserting strace both as "strace script wine"
>> and as "script strace wine". We tried running just "wine a.exe"
>> without any extra glue, and logging the raw SSH packets from putty.
>> After many iterations on these and other tests, Jacek finally had the
>> idea to try removing Windows entirely from the equation, and we ran
>> with a purely unix program / compiler combination:
>>
>> #include <unistd.h>
>>
>> int main()
>> {
>> write(1, "test\r\n", 6);
>> return 0;
>> }
>>
>> (and also as "test\n", 5)
>>
>> In both versions, the following was observed:
>>
>> case 1) ./a.out | xxd
>> case 2) script -c ./a.out out; xxd out
>> case 3) enable putting logging, ./a.out
>>
>> In case 1, xxd showed no extra \r's. In cases 2 and 3, there was an
>> extra \r (either 0d 0d 0a for test\r\n, or 0d 0a for test\n).
>>
>> So, is it possible after all of this back and forth regarding mingw,
>> wine, and others, that it's down to the write() system call that's
>> inserting extra \r's? Is this expected?
>
> "This is why DejaGnu testsuites must be prepared to discard excess
> carriage returns."
>
> The write(2) system call inserts nothing and simply hands off the buffer
> to the relevant part of the kernel I/O subsystem. (The kernel in POSIX
> is *not* a monolithic black box.) When stdout for your test program is
> a pty slave, that relevant part is the kernel terminal driver. The
> kernel terminal driver is converting "\n" to "\r\n" upon output to the
> associated port, since hardware terminals typically *do* require CRLF.
> The associated port in this case is virtual and part of the kernel pty
> subsystem, which presents octets written to that port to its associated
> pty master device. The user-visible pty slave device acts just like a
> serial terminal, including all translations normally done for handling
> serial terminals.
>
> A pty is conceptually a null-modem cable connected between two
> infinitely-fast serial ports on the same machine, although the slave
> will still report an actual baud rate if queried. (Run "stty" with no
> arguments under script(1), an ssh session, or an X11 terminal emulator
> to see what a pty slave looks like on your machine.)
>
> In your case 1, the pty subsystem is not used and output is collected
> over a pipe. Using "./a.out > out; xxd out" would produce the same
> results. In cases 2 and 3, there is a pty involved, either set up by
> script(1) or by sshd (assuming you meant "enable putty logging" in case
> 3) that performs the standard terminal translations. In all cases,
> strace(1) will show the exact string written to the pty slave device,
> which will not include any extra CRs because *those* *are* *inserted*
> *by* *the* *kernel* *terminal* *driver* as the data is transferred to
> the pty master device's read queue.
>
> This insertion of carriage returns is expected and standardized behavior
> in POSIX and is the reason Unix could use bare LF as end-of-line even
> though hardware terminals always needed CRLF. CP/M (and therefore
> MS-DOS which began its existence as a cheap CP/M knockoff) did not have
> this translation layer and instead dumped the complexity of a two-octet
> end-of-line sequence on user programs, leading to much confusion even
> today. This is not a Wine issue, although the terminal escape sequences
> in your original issue *were* from Wine. Note that the number of excess
> carriage returns that a DejaGnu testsuite must be prepared to discard is
> unspecified because running tests on remote targets may result in *any*
> *number* of CRs preceding each LF by the time the results reach the test
> driver machine in more complex testing lab environments.
>
>
> -- Jacob
Agree
next prev parent reply other threads:[~2023-01-06 3:44 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CAF1jjLtJW0juQR6L-VybJ8SSaqkfi=qN9FnxJVaY=oQBtkSLxA@mail.gmail.com>
[not found] ` <3f62bac2-ac1b-5c55-2488-ede2389d35d2@netcologne.de>
[not found] ` <CAF1jjLvJU2fnU0u0p9SwPre5mnhFdmv9pm_OvZGOvjQApCROqw@mail.gmail.com>
2022-12-17 10:52 ` Thomas Koenig
2022-12-17 23:24 ` NightStrike
2022-12-18 3:44 ` Jacob Bachmeyer
2022-12-18 21:13 ` NightStrike
2022-12-19 4:29 ` Jacob Bachmeyer
2022-12-19 10:43 ` Torbjorn SVENSSON
2022-12-19 11:00 ` NightStrike
2022-12-19 11:13 ` NightStrike
2022-12-20 3:51 ` Jacob Bachmeyer
[not found] ` <7cb45ab2-cc6e-c502-5592-51ffabcbc6f8@codeweavers.com>
2022-12-22 1:01 ` NightStrike
2022-12-22 4:37 ` Jacob Bachmeyer
2022-12-23 10:36 ` NightStrike
2022-12-23 12:43 ` Eric Pouech
2022-12-24 4:00 ` Jacob Bachmeyer
2022-12-24 11:05 ` Mark Wielaard
2023-01-05 2:50 ` NightStrike
2023-01-06 3:33 ` Jacob Bachmeyer
2023-01-06 3:44 ` Jerry D [this message]
2023-01-08 7:12 ` NightStrike
2023-01-11 2:30 ` Jacob Bachmeyer
2023-01-11 9:33 ` NightStrike
2023-01-12 4:11 ` Jacob Bachmeyer
2023-01-06 3:41 ` Jerry D
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0f560ae5-f829-b91c-b16c-0cad672a0b31@gmail.com \
--to=jvdelisle2@gmail.com \
--cc=dejagnu@gnu.org \
--cc=eric.pouech@orange.fr \
--cc=fortran@gcc.gnu.org \
--cc=gcc@gcc.gnu.org \
--cc=jacek@codeweavers.com \
--cc=jcb62281@gmail.com \
--cc=nightstrike@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).