From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 22652 invoked by alias); 27 Feb 2018 15:03:16 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 22628 invoked by uid 89); 27 Feb 2018 15:03:15 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.9 required=5.0 tests=AWL,BAYES_00,KAM_LAZY_DOMAIN_SECURITY,RCVD_IN_DNSWL_LOW autolearn=no version=3.3.2 spammy=calgary, H*r:ip*192.168.1.100, Calgary, Alberta X-HELO: smtp-out-so.shaw.ca Received: from smtp-out-so.shaw.ca (HELO smtp-out-so.shaw.ca) (64.59.136.139) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 27 Feb 2018 15:03:13 +0000 Received: from [192.168.1.100] ([24.64.240.204]) by shaw.ca with ESMTP id qgmce6fnVYxCTqgmdepxtc; Tue, 27 Feb 2018 08:03:12 -0700 X-Authority-Analysis: v=2.3 cv=cav8UELM c=1 sm=1 tr=0 a=MVEHjbUiAHxQW0jfcDq5EA==:117 a=MVEHjbUiAHxQW0jfcDq5EA==:17 a=N659UExz7-8A:10 a=CCpqsmhAAAAA:8 a=b_J7Jx8z1COwrFdO0U8A:9 a=pILNOxqGKmIA:10 a=FFR_xKssrUEA:10 a=Z3i3OTMkIkoA:10 a=rhrKo6LjUkkA:10 a=CHDpIXivPXAA:10 a=ul9cdbp4aOFLsgKbc677:22 From: Brian Inglis Subject: Re: gawk Regression: CR characters are not stripped on Windows Reply-To: Brian.Inglis@SystematicSw.ab.ca To: cygwin@cygwin.com References: Message-ID: Date: Tue, 27 Feb 2018 15:03:00 -0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-CMAE-Envelope: MS4wfCQeEp2AU4wXdxHayKTOPOwaCMNX515M54BfRvB/tF6w7MyghcvyikPovuJD+FpVV4G8pD2BLVYVECSa5hW+msmotiU/ZuCCMIET/KLHfe9QQiAc2cY9 Me83h1G9MBlpgdZrSTC7qLAulOeafRWB3LvQWSdsFfTdyyaLAijKH7aKq/ose/vKedyOmRXFKXoSwA== X-IsSubscribed: yes X-SW-Source: 2018-02/txt/msg00295.txt.bz2 On 2018-02-27 00:22, Orgad Shaneh wrote: > Cross-posting per Eli Zaretskii's request. > CR characters used to be automatically stripped on Windows (MSYS2 and > Cygwin environments). This is broken in 4.2.0. Cygwin binary mounts treat files as on Unix. You missed all the discussions in early 2017 about gawk, grep, sed EOL handling: https://sourceware.org/ml/cygwin/2017-02/msg00152.html https://sourceware.org/ml/cygwin/2017-02/msg00188.html https://sourceware.org/ml/cygwin/2017-02/msg00189.html following on from discussions about bash after ShellShock: https://sourceware.org/ml/cygwin/2016-08/msg00097.html > Minimal example: > echo -en "foo\r\n\r\nbar\r\n" > foo.txt > awk '/^$/ { print "found" }' foo.txt # This worked with 4.1.4 and > doesn't work with 4.2.0 > awk '/^\r$/ { print "found" }' foo.txt # This works with 4.2.0 and > doesn't work with 4.1.4 >> Under MS-Windows, 'gawk' (and many other text programs) silently >> translates end-of-line '\r\n' to '\n' on input and '\n' to '\r\n' on >> output. Cygwin does not try to be an MS Windows environment. Cygwin tries its best to be a POSIX/Unix/Linux environment. > and on Feb 8 the following section was added: >> Recent versions of Cygwin open all files in binary mode. This means >> that you should use 'RS = "\r?\n"' in order to be able to handle >> standard MS-Windows text files with carriage-return plus line-feed line >> endings. Use DOS files from a Cygwin text mount which does the conversion. > This breaks compatibility between different gawk versions. What were > the reasons for this change in cygwin, and why was it pushed upstream? Compatibility with POSIX/Unix/Linux systems, except on a text mount, to allow scripts which deal with binary data or embedded \r to work correctly, and require scripts which work correctly, on Windows or Unix text as the application provides, prefers, or ignores, and under Unix/Cygwin/Msys/Mingw. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple