From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by sourceware.org (Postfix) with ESMTPS id 645143858D28; Fri, 6 Jan 2023 03:42:00 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 645143858D28 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pj1-x1035.google.com with SMTP id o21so358958pjw.0; Thu, 05 Jan 2023 19:42:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:subject:from:references:cc:to :content-language:user-agent:mime-version:date:message-id:from:to:cc :subject:date:message-id:reply-to; bh=m1T9KFIgdINgVKFz29gmUgFel5drj1m4bCap49WXC74=; b=Ofs9GX6+Nl14XXX74nUpKqLHFMRaIGah23FzKted5QvQEhYiEnfvRTDNBYiX1DWkEr Rf1/boznLTqLuQk50nB8MJmPgjePzdQCr+KCb9dF/MhcZvvlyxTO0J+6pyaoCmMnrupG YEGUJ+KsF6iCqz7njzE+S1CcM9dPCaddpiSWqz5uXdTUO1GTtCY0j7ZffQOkvgA47F5D 2PPi5OGFbcguEOHmd3ekwg6f0Bfoyd4DxnqIyvcI0mWrZ1+csD9ejyFWOsx/HsjxDdA7 4M+haqZjFio1/dWO1kZ66NVGCsXT0tl6KHzHydj/CgJ7KC2q1VoJrbHuSSFHugZVG3a9 XxDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:subject:from:references:cc:to :content-language:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=m1T9KFIgdINgVKFz29gmUgFel5drj1m4bCap49WXC74=; b=D/0AhYi/OIfxvfsV3whh1Wvq1xMvGDmepH9fYRAgipMVuVqYDWZdZy+yxtbDA6lbS/ 1eXIqk92HJe8tSGsq2IoJZxN2UiOr+CBOqqP7udIhNFOzPt4il9ymeknAciv6GMryCSI OMBoQ2RTyZkGoxAYDP0Sk/X/iGSuQgt2kOrhXjSIDVG43C6wS0daB3zuO+AojKoh/zjK G4QZewiOp77b6uRX2hWH6i2shwaGQAGaNbtLsKGEW1cP+Ac5sXTDlLS4QMCm3Juebxwm CLhlH6gfXRR0hEJUmrAZSNZyzVQZ05a3M6Hn1mqh2DMyxUDbW0YSvR4Q2jLcnFO5i5UF kQ2Q== X-Gm-Message-State: AFqh2kpXDRB+jMYFvNddG4vV6ee85KR82A5jjwkJ/moz1EyKNBKVueT+ TVghJ8+EdL6eNsbt5YL4xr4= X-Google-Smtp-Source: AMrXdXvcZ9cVYjLDovQgYwa1bIH/CchRmenVVPyWsiXqhnIZnerEHTcz/pfd4lvOE3QtX4GS2kJygw== X-Received: by 2002:a05:6a20:1586:b0:ab:e177:111e with SMTP id h6-20020a056a20158600b000abe177111emr19574977pzj.5.1672976519312; Thu, 05 Jan 2023 19:41:59 -0800 (PST) Received: from [192.168.1.20] ([50.37.188.226]) by smtp.gmail.com with ESMTPSA id h19-20020a170902eed300b00186c5e8a8d7sm17468917plb.171.2023.01.05.19.41.58 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 05 Jan 2023 19:41:59 -0800 (PST) Message-ID: <1d87ebc9-fc2d-86cd-5873-a745accb6860@gmail.com> Date: Thu, 5 Jan 2023 19:41:58 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.6.0 Content-Language: en-US To: NightStrike , jcb62281@gmail.com Cc: Jacek Caban , fortran@gcc.gnu.org, Eric Pouech , "gcc@gcc.gnu.org" , dejagnu@gnu.org References: <639FE88D.7090408@gmail.com> <7cb45ab2-cc6e-c502-5592-51ffabcbc6f8@codeweavers.com> <63A3DF0E.1050902@gmail.com> <63A67964.6080902@gmail.com> From: Jerry D Subject: Re: testsuite under wine In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-3.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 1/4/23 6:50 PM, NightStrike via Fortran wrote: > On Fri, Dec 23, 2022 at 11:00 PM Jacob Bachmeyer wrote: >> NightStrike wrote: >>> On Wed, Dec 21, 2022 at 11:37 PM Jacob Bachmeyer wrote: >>>> NightStrike wrote: >>>> >>>>> [...] >>>>> Second, the problems with extra \r's still remain, but I think we've >>>>> generally come to think that that part isn't Wine and is instead >>>>> either the testsuite or deja. So I'll keep those replies to Jacob's >>>>> previous message. >>>>> >>>>> >>>> Most likely, it is a combination of the MinGW libc (which emits "\r\n" >>>> for end-of-line in accordance with Windows convention) and the kernel >>>> terminal driver (which passes "\r" and translates "\n" to "\r\n" in >>>> accordance with POSIX convention). Wine, short of trying to translate >>>> "\r\n" back to "\n" in accordance with POSIX conventions (and likely >>>> making an even bigger mess---does Wine know if a handle is supposed to >>>> be text or binary?) cannot really fix this, so the testsuite needs to >>>> handle non-POSIX-standard line endings. (The Rust tests probably have >>>> an outright bug if the newlines are being duplicated.) >>>> >>> >>> You may be onto something here. I ran wine under script as `script -c >>> "wine64 ./a.exe" out` (thanks, Arsen!), and it had the same extra \r >>> prepended to the \r\n. I was making the mistake previously of running >>> wine manually and capturing it to a file as `wine64 ./a.exe > out`, >>> which as several have pointed out in this thread, that would disable >>> the quirk, so of course it didn't reveal any problems. I'm behind, >>> but I'll catch up to you guys eventually :) >>> >> >> So close, and yet so far... script(1) /also/ uses a pty, so it is >> getting the same translations as Expect and therefore DejaGnu. >> >>> So at least we know for sure that this particular instance of extra >>> characters is coming from Wine. Maybe Wine can be smart enough to >>> only translate \n into \r\n instead of translating \r\n into \r\r\n. >>> Jacek / Eric, comments here? I'm happy to try another patch, the >>> first one was great. >>> >> >> I doubt that Wine is doing that translation. MinGW libc produces output >> conformant to Windows conventions, so printf("\n") on a text handle >> emits "\r\n", which Wine passes along. POSIX convention is that "\n" is >> translated to "\r\n" in the kernel terminal driver upon output, so the >> kernel translates the "\n" in the "\r\n" into /another/ "\r\n", yielding >> "\r\r\n" at the pty master end. This is why DejaGnu testsuites must be >> prepared to discard excess carriage returns. The first CR came from >> MinGW libc; the second CR came from the kernel terminal driver; the LF >> was ultimately passed through. > > Jacek and I have been digging into this on IRC, and he's been very > helpful in trying to get further, but we're still stuck. We tried to > be more introspective, inserting strace both as "strace script wine" > and as "script strace wine". We tried running just "wine a.exe" > without any extra glue, and logging the raw SSH packets from putty. > After many iterations on these and other tests, Jacek finally had the > idea to try removing Windows entirely from the equation, and we ran > with a purely unix program / compiler combination: > > #include > > int main() > { > write(1, "test\r\n", 6); > return 0; > } > > (and also as "test\n", 5) > > In both versions, the following was observed: > > case 1) ./a.out | xxd > case 2) script -c ./a.out out; xxd out > case 3) enable putting logging, ./a.out > > In case 1, xxd showed no extra \r's. In cases 2 and 3, there was an > extra \r (either 0d 0d 0a for test\r\n, or 0d 0a for test\n). > > So, is it possible after all of this back and forth regarding mingw, > wine, and others, that it's down to the write() system call that's > inserting extra \r's? Is this expected? I have reproduced this test with C. I suspect that the 'write' function was written to accommodate non standard behavior of windows which expects a CR-LF. This means that POSIX compliant code is adjusted by the C or gfortran libraries to emit a extra CR so that on Windows it will just work ok. So the test is doing exactly what you are telling it to do. An LF causes the run times to add a CR in front. With libgfortran I remember implementing some of this code myself and this is the reason, to keep applications happy on Windows. So the gfortran tests we wrote to accept either a CR or a CR-LF, and in the test code to only issue a normal line ending which on UNIX will be an LF and Windows an CR-LF. I lose track of details in between looking ta the test cases. let me know one of them to study that is gfortran side and will see what it is doing. Jerry