From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oa1-x2d.google.com (mail-oa1-x2d.google.com [IPv6:2001:4860:4864:20::2d]) by sourceware.org (Postfix) with ESMTPS id 40E6E3858D28; Fri, 6 Jan 2023 03:33:19 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 40E6E3858D28 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-oa1-x2d.google.com with SMTP id 586e51a60fabf-14455716674so548394fac.7; Thu, 05 Jan 2023 19:33:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:references:subject:cc:to :mime-version:user-agent:reply-to:from:date:message-id:from:to:cc :subject:date:message-id:reply-to; bh=EIamy7EZgZLB61+90rsaqg/OwO3w4rDf8Op6azTh7f0=; b=kVRorIqqbwlKUySyIpJDWOFfpvGClnACnGWOArYkjPTkKlTIYuRFCZKbKZu1TUq7e6 oKf5HGtO5jtW6tPraCXC/sfBnAj7qTbSZZNO58TSlUJRgSAJJHlYgF8J7B7U5eoleegN TgokC71LqSkytmo81lwECb9WafM4YION+biszY2pCCJ6/adKjidc54qyvGZZMH0t+744 62uGit80OgtmU1/bY+cV58OEd2dx7KCJvDwvW8JtL/hWhIU/zxX1OisZpeaKmJdw0jzl /hQNWJuxOE1A8BZi+Jo0XZJwpWH+0mNEvKinCGJQSPApuyJ0MVwtn+ds4F54VlZY1vwv EvfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:references:subject:cc:to :mime-version:user-agent:reply-to:from:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=EIamy7EZgZLB61+90rsaqg/OwO3w4rDf8Op6azTh7f0=; b=F87wlA1L5c+xGQLIcRMxIN1l16tIXqstQWQRmaw6Y3bzOVrXbt7+bd/mksxe0WgXR7 JpROjy/oJfTNxPndmfOjG7WBAKkHQSfyJw+KEFGfUeg583+tmoSfL3wb6YYm9Flphfoq lJUAMlbHIAhttswzafjBEXU2KPvnYcypN7IiVW7mY0LVumWuuauU0RxwwzrQvm4k8xar q+m/BfQFbw/j5FfVUN/oGAS4ZSXW4snA3MdVICxzPnYKsMBiu+DSE3Zq/srador16NyH ZrRBJvyNGP7sbT1GNlOIHWKG4hMUfAHFsr6/NJ4Vk9dm6w58yx5WY9ldugAXC523thM6 RDuQ== X-Gm-Message-State: AFqh2kppvZ2X+xsd3ufmE0+z/2pp4CL5VoDqkUlFMpyTjv+O25sYA3o7 CwFB2EH+6v5vxruXnVG03b0= X-Google-Smtp-Source: AMrXdXt4RhYi+V94rLU96SIJPJeS+PB1vVNofC8hyCPh+JbGkb+qON3xy5x1cYNxJkpLeUaVbxpjEw== X-Received: by 2002:a05:6870:f815:b0:14f:a68c:7c76 with SMTP id fr21-20020a056870f81500b0014fa68c7c76mr26522722oab.42.1672975998197; Thu, 05 Jan 2023 19:33:18 -0800 (PST) Received: from [127.0.0.1] ([70.133.144.146]) by smtp.gmail.com with ESMTPSA id x4-20020a056870a78400b00143824af059sm92736oao.7.2023.01.05.19.33.16 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 05 Jan 2023 19:33:17 -0800 (PST) Message-ID: <63B7967B.60502@gmail.com> Date: Thu, 05 Jan 2023 21:33:15 -0600 From: Jacob Bachmeyer Reply-To: jcb62281@gmail.com User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.22) Gecko/20090807 SeaMonkey/1.1.17 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: NightStrike CC: Jacek Caban , fortran@gcc.gnu.org, Eric Pouech , "gcc@gcc.gnu.org" , dejagnu@gnu.org Subject: Re: testsuite under wine References: <639FE88D.7090408@gmail.com> <7cb45ab2-cc6e-c502-5592-51ffabcbc6f8@codeweavers.com> <63A3DF0E.1050902@gmail.com> <63A67964.6080902@gmail.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,FREEMAIL_REPLYTO_END_DIGIT,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: NightStrike wrote: > On Fri, Dec 23, 2022 at 11:00 PM Jacob Bachmeyer wrote: > >> NightStrike wrote: >> >>> On Wed, Dec 21, 2022 at 11:37 PM Jacob Bachmeyer wrote: >>> >>>> [...] >>> So at least we know for sure that this particular instance of extra >>> characters is coming from Wine. Maybe Wine can be smart enough to >>> only translate \n into \r\n instead of translating \r\n into \r\r\n. >>> Jacek / Eric, comments here? I'm happy to try another patch, the >>> first one was great. >>> >>> >> I doubt that Wine is doing that translation. MinGW libc produces output >> conformant to Windows conventions, so printf("\n") on a text handle >> emits "\r\n", which Wine passes along. POSIX convention is that "\n" is >> translated to "\r\n" in the kernel terminal driver upon output, so the >> kernel translates the "\n" in the "\r\n" into /another/ "\r\n", yielding >> "\r\r\n" at the pty master end. This is why DejaGnu testsuites must be >> prepared to discard excess carriage returns. The first CR came from >> MinGW libc; the second CR came from the kernel terminal driver; the LF >> was ultimately passed through. >> > > Jacek and I have been digging into this on IRC, and he's been very > helpful in trying to get further, but we're still stuck. We tried to > be more introspective, inserting strace both as "strace script wine" > and as "script strace wine". We tried running just "wine a.exe" > without any extra glue, and logging the raw SSH packets from putty. > After many iterations on these and other tests, Jacek finally had the > idea to try removing Windows entirely from the equation, and we ran > with a purely unix program / compiler combination: > > #include > > int main() > { > write(1, "test\r\n", 6); > return 0; > } > > (and also as "test\n", 5) > > In both versions, the following was observed: > > case 1) ./a.out | xxd > case 2) script -c ./a.out out; xxd out > case 3) enable putting logging, ./a.out > > In case 1, xxd showed no extra \r's. In cases 2 and 3, there was an > extra \r (either 0d 0d 0a for test\r\n, or 0d 0a for test\n). > > So, is it possible after all of this back and forth regarding mingw, > wine, and others, that it's down to the write() system call that's > inserting extra \r's? Is this expected? > "This is why DejaGnu testsuites must be prepared to discard excess carriage returns." The write(2) system call inserts nothing and simply hands off the buffer to the relevant part of the kernel I/O subsystem. (The kernel in POSIX is *not* a monolithic black box.) When stdout for your test program is a pty slave, that relevant part is the kernel terminal driver. The kernel terminal driver is converting "\n" to "\r\n" upon output to the associated port, since hardware terminals typically *do* require CRLF. The associated port in this case is virtual and part of the kernel pty subsystem, which presents octets written to that port to its associated pty master device. The user-visible pty slave device acts just like a serial terminal, including all translations normally done for handling serial terminals. A pty is conceptually a null-modem cable connected between two infinitely-fast serial ports on the same machine, although the slave will still report an actual baud rate if queried. (Run "stty" with no arguments under script(1), an ssh session, or an X11 terminal emulator to see what a pty slave looks like on your machine.) In your case 1, the pty subsystem is not used and output is collected over a pipe. Using "./a.out > out; xxd out" would produce the same results. In cases 2 and 3, there is a pty involved, either set up by script(1) or by sshd (assuming you meant "enable putty logging" in case 3) that performs the standard terminal translations. In all cases, strace(1) will show the exact string written to the pty slave device, which will not include any extra CRs because *those* *are* *inserted* *by* *the* *kernel* *terminal* *driver* as the data is transferred to the pty master device's read queue. This insertion of carriage returns is expected and standardized behavior in POSIX and is the reason Unix could use bare LF as end-of-line even though hardware terminals always needed CRLF. CP/M (and therefore MS-DOS which began its existence as a cheap CP/M knockoff) did not have this translation layer and instead dumped the complexity of a two-octet end-of-line sequence on user programs, leading to much confusion even today. This is not a Wine issue, although the terminal escape sequences in your original issue *were* from Wine. Note that the number of excess carriage returns that a DejaGnu testsuite must be prepared to discard is unspecified because running tests on remote targets may result in *any* *number* of CRs preceding each LF by the time the results reach the test driver machine in more complex testing lab environments. -- Jacob