From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by sourceware.org (Postfix) with ESMTPS id 5968E3858D28; Fri, 6 Jan 2023 03:44:43 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5968E3858D28 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pj1-x1030.google.com with SMTP id fz16-20020a17090b025000b002269d6c2d83so4983014pjb.0; Thu, 05 Jan 2023 19:44:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=G2zYv9/bqtAAsm9kxIEYIPCwwMZyAbu5NAfsNRrJX3Y=; b=aMk9bWfVENI/Pz77PKhMam9NaGUmSjDKbr2DgCZTC6jsQDNUaQEZw4K1ISUeC06u5J MXPlcFzdMBPKA0pVAJANqdermltMqIpLV8r7iBs6605rzIQAXV8Cii3EHDGLtJ/0iDdI FEyjBfciGMMYP39QydR51RYzNaheNjlGOd+34cuje2AOLOgtMD/7+Yx9Y9/Vw7ia3kso ujMlgoEhWBxy/ZRJvb/8aZX3azaxDu55o7bMOUxFp9LGjyDhQGy1HELc4Sh9zLNPbLCt vwSUGAN56emitLMWv72KPzWw4ot+kIbL3xVn3uhnAQU8MEX1an/cyQWQMBdtvOh/ZiYr nkxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=G2zYv9/bqtAAsm9kxIEYIPCwwMZyAbu5NAfsNRrJX3Y=; b=bMcB1FJ3vNOUyMqZIySeCI0yWiQpmeTHF9bJL0naIUoNndQ2P20m/IO2vcFQnkH7MS eITBNcPOvSgiuIs6tcMvLgVda+dUX0pOuGWcN0gyiIzg3AyQ5uNW2Hby0N48fcwY+iCA F9IFyVPcIaXxYfOHg9IFSkj0seVjU69Qkmpb0hQAXLLAVnIEQfDzoRIPZvPs9Be/yW9l 6N8bR2fyfqzOkgbUIJIYmAS76q3BMFgdakvdsR0qdi6f97gzm0cwwif1kVTWP7VaUDJZ IMjbJsNLUAv/8Y0TuPwcMWjyjRfUP4D/++AhbWQ9gCvuwek1wkvDmrnTxU7hmxCDa62i EVhg== X-Gm-Message-State: AFqh2kreVZBYnTIwdn4uUQyFYe1KdMKeQAA2PY88ZGYK0hX5ZF9QHIvF XPSacg8PZSXxyQyvQ0MRjW8= X-Google-Smtp-Source: AMrXdXtoG4xhjtGzJVh9WxjkOEFpUQpiiEdoKb9LtgCjR1pgB6AYfsRE4HFUGr+D7tmogFX4vqGbCg== X-Received: by 2002:a05:6a20:158b:b0:b1:d045:2818 with SMTP id h11-20020a056a20158b00b000b1d0452818mr20433650pzj.2.1672976682349; Thu, 05 Jan 2023 19:44:42 -0800 (PST) Received: from [192.168.1.20] ([50.37.188.226]) by smtp.gmail.com with ESMTPSA id o197-20020a62cdce000000b005817fa83bcesm44514pfg.76.2023.01.05.19.44.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 05 Jan 2023 19:44:42 -0800 (PST) Message-ID: <0f560ae5-f829-b91c-b16c-0cad672a0b31@gmail.com> Date: Thu, 5 Jan 2023 19:44:41 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.6.0 Subject: Re: testsuite under wine Content-Language: en-US To: jcb62281@gmail.com, NightStrike Cc: Jacek Caban , fortran@gcc.gnu.org, Eric Pouech , "gcc@gcc.gnu.org" , dejagnu@gnu.org References: <639FE88D.7090408@gmail.com> <7cb45ab2-cc6e-c502-5592-51ffabcbc6f8@codeweavers.com> <63A3DF0E.1050902@gmail.com> <63A67964.6080902@gmail.com> <63B7967B.60502@gmail.com> From: Jerry D In-Reply-To: <63B7967B.60502@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,BODY_8BITS,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 1/5/23 7:33 PM, Jacob Bachmeyer via Fortran wrote: > NightStrike wrote: >> On Fri, Dec 23, 2022 at 11:00 PM Jacob Bachmeyer >> wrote: >>> NightStrike wrote: >>>> On Wed, Dec 21, 2022 at 11:37 PM Jacob Bachmeyer >>>> wrote: >>>>> [...] >>>> So at least we know for sure that this particular instance of extra >>>> characters is coming from Wine.  Maybe Wine can be smart enough to >>>> only translate \n into \r\n instead of translating \r\n into \r\r\n. >>>> Jacek / Eric, comments here?  I'm happy to try another patch, the >>>> first one was great. >>>> >>> I doubt that Wine is doing that translation.  MinGW libc produces output >>> conformant to Windows conventions, so printf("\n") on a text handle >>> emits "\r\n", which Wine passes along.  POSIX convention is that "\n" is >>> translated to "\r\n" in the kernel terminal driver upon output, so the >>> kernel translates the "\n" in the "\r\n" into /another/ "\r\n", yielding >>> "\r\r\n" at the pty master end.  This is why DejaGnu testsuites must be >>> prepared to discard excess carriage returns.  The first CR came from >>> MinGW libc; the second CR came from the kernel terminal driver; the LF >>> was ultimately passed through. >> >> Jacek and I have been digging into this on IRC, and he's been very >> helpful in trying to get further, but we're still stuck.  We tried to >> be more introspective, inserting strace both as "strace script wine" >> and as "script strace wine".  We tried running just "wine a.exe" >> without any extra glue, and logging the raw SSH packets from putty. >> After many iterations on these and other tests, Jacek finally had the >> idea to try removing Windows entirely from the equation, and we ran >> with a purely unix program / compiler combination: >> >> #include >> >> int main() >> { >>         write(1, "test\r\n", 6); >>         return 0; >> } >> >> (and also as "test\n", 5) >> >> In both versions, the following was observed: >> >> case 1) ./a.out | xxd >> case 2) script -c ./a.out out; xxd out >> case 3) enable putting logging, ./a.out >> >> In case 1, xxd showed no extra \r's.  In cases 2 and 3, there was an >> extra \r (either 0d 0d 0a for test\r\n, or 0d 0a for test\n). >> >> So, is it possible after all of this back and forth regarding mingw, >> wine, and others, that it's down to the write() system call that's >> inserting extra \r's?  Is this expected? > > "This is why DejaGnu testsuites must be prepared to discard excess > carriage returns." > > The write(2) system call inserts nothing and simply hands off the buffer > to the relevant part of the kernel I/O subsystem.  (The kernel in POSIX > is *not* a monolithic black box.)  When stdout for your test program is > a pty slave, that relevant part is the kernel terminal driver.  The > kernel terminal driver is converting "\n" to "\r\n" upon output to the > associated port, since hardware terminals typically *do* require CRLF. > The associated port in this case is virtual and part of the kernel pty > subsystem, which presents octets written to that port to its associated > pty master device.  The user-visible pty slave device acts just like a > serial terminal, including all translations normally done for handling > serial terminals. > > A pty is conceptually a null-modem cable connected between two > infinitely-fast serial ports on the same machine, although the slave > will still report an actual baud rate if queried.  (Run "stty" with no > arguments under script(1), an ssh session, or an X11 terminal emulator > to see what a pty slave looks like on your machine.) > > In your case 1, the pty subsystem is not used and output is collected > over a pipe.  Using "./a.out > out; xxd out" would produce the same > results.  In cases 2 and 3, there is a pty involved, either set up by > script(1) or by sshd (assuming you meant "enable putty logging" in case > 3) that performs the standard terminal translations.  In all cases, > strace(1) will show the exact string written to the pty slave device, > which will not include any extra CRs because *those* *are* *inserted* > *by* *the* *kernel* *terminal* *driver* as the data is transferred to > the pty master device's read queue. > > This insertion of carriage returns is expected and standardized behavior > in POSIX and is the reason Unix could use bare LF as end-of-line even > though hardware terminals always needed CRLF.  CP/M (and therefore > MS-DOS which began its existence as a cheap CP/M knockoff) did not have > this translation layer and instead dumped the complexity of a two-octet > end-of-line sequence on user programs, leading to much confusion even > today.  This is not a Wine issue, although the terminal escape sequences > in your original issue *were* from Wine.  Note that the number of excess > carriage returns that a DejaGnu testsuite must be prepared to discard is > unspecified because running tests on remote targets may result in *any* > *number* of CRs preceding each LF by the time the results reach the test > driver machine in more complex testing lab environments. > > > -- Jacob Agree