From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 38641 invoked by alias); 8 Jun 2017 19:57:21 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 38278 invoked by uid 89); 8 Jun 2017 19:57:20 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-6.9 required=5.0 tests=BAYES_00,GIT_PATCH_2,SPF_HELO_PASS,T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 spammy=GOOD, HAD, price, wishes X-HELO: mx1.redhat.com Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Thu, 08 Jun 2017 19:57:19 +0000 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 635746525 for ; Thu, 8 Jun 2017 19:57:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 635746525 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 635746525 Received: from [10.10.121.94] (ovpn-121-94.rdu2.redhat.com [10.10.121.94]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0371818221 for ; Thu, 8 Jun 2017 19:57:21 +0000 (UTC) Subject: Re: CR-LF handling behavior of SED changed recently - this breaks a lot of MinGW cross build scripts To: cygwin@cygwin.com References: <0F7D3B1B3C4B894D824F5B822E3E5A175B2636E4@IRSMSX103.ger.corp.intel.com> <0F7D3B1B3C4B894D824F5B822E3E5A175B26CE47@IRSMSX102.ger.corp.intel.com> <59399CC5.60900@tlinx.org> From: Eric Blake Openpgp: url=http://people.redhat.com/eblake/eblake.gpg Message-ID: <417f84ac-5d9f-dc50-e912-973e90b8a128@redhat.com> Date: Thu, 08 Jun 2017 19:57:00 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.1.0 MIME-Version: 1.0 In-Reply-To: <59399CC5.60900@tlinx.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="aOpHDQtNx64G95usMMPfiwiGFD68eWlSB" X-IsSubscribed: yes X-SW-Source: 2017-06/txt/msg00071.txt.bz2 --aOpHDQtNx64G95usMMPfiwiGFD68eWlSB Content-Type: multipart/mixed; boundary="bUAqd226t2nEAMgOXBdKvWftv5Ujg5kdO"; protected-headers="v1" From: Eric Blake To: cygwin@cygwin.com Message-ID: <417f84ac-5d9f-dc50-e912-973e90b8a128@redhat.com> Subject: Re: CR-LF handling behavior of SED changed recently - this breaks a lot of MinGW cross build scripts References: <0F7D3B1B3C4B894D824F5B822E3E5A175B2636E4@IRSMSX103.ger.corp.intel.com> <0F7D3B1B3C4B894D824F5B822E3E5A175B26CE47@IRSMSX102.ger.corp.intel.com> <59399CC5.60900@tlinx.org> In-Reply-To: <59399CC5.60900@tlinx.org> --bUAqd226t2nEAMgOXBdKvWftv5Ujg5kdO Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Content-length: 2925 On 06/08/2017 01:51 PM, L A Walsh wrote: >> 1.) I build many (~ 50) unix libs and tools MinGW cross on cygwin from >> sources and this breaks many of the configure and other scripts.=20 > --- > But didn't one have to use 'sed -b' before, in order to > strip out CR's? No, the exact opposite. It used to be that you HAD to use 'sed -b' to preserve CRs on a binary mount; now binary mounts preserve CRs automatically, making 'sed -b' a no-op on binary mounts. (This is closer to Linux behavior, where 'sed' preserves CRs automatically because everything is binary mount, and 'sed -b' is a no-op). On text mounts, 'sed -b' allows you to preserve CRs where they would otherwise be stripped automatically. > I.e. wouldn't all the individual lib/tool maintainers have > been required to add '-b' to their sed scripts? Sort of. The problem was that it used to be difficult to write portable scripts that worked on Cygwin and non-cygwin and still dealt with CRs. That's because you could not rely on 'sed -b' existing (not all the world uses GNU sed, and POSIX doesn't require -b to exist). But if you omitted the -b on Cygwin, your data was silently corrupted. With the change back in February, now Cygwin sed defaults to POSIX behavior on binary mounts, and the ONLY people that still have to use 'sed -b' are those who use text mounts; while remembering that text mounts are not the default. > Seems either way, > you have the undesirability of forcing external products to change to > support cygwin. External products were being lazy by relying on cygwin to strip CR when they should have stripped it themselves. But 'sed -b' does NOT strip CR (it is the exact opposite, of keeping CR unstripped). >=20 > Whereas, what I'd wonder is, how you are supplying input to sed > in the first place? I.e. how did CR's get into the stream to begin with. > If you used cygwin and some tool on cygwin generated CR's into the output > stream, I'd think that'd be a problem (or bug). But if you are mixing > DOS/Win-progs w/cygwin, then you need to adapt the DOS/Win progs' > outputs to > not have CR in them. Exactly - it used to be you could be lazy and feed the DOS/Win prog output (with CRs) to cygwin, and cygwin would ignore the CR - but that laziness came at a price that it would silently corrupt data for someone that was not aware that they needed the non-portable 'sed -b' to preserve CR when operating on known-binary data. Yes, the change is forcing clients of external data to be more explicit about the CR in their data, but in my mind, that's a GOOD thing - it's always better to be explicit about intentions, and the new behavior is something YOU control by whether you pre-filter the data, and not something that sed FORCED on you by using text mode against your wishes. --=20 Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org --bUAqd226t2nEAMgOXBdKvWftv5Ujg5kdO-- --aOpHDQtNx64G95usMMPfiwiGFD68eWlSB Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" Content-length: 604 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJZOawgAAoJEKeha0olJ0Nq7JsH/1YmOy9SbE/Pj+CzgDIo3skV IPsdoTlUTbGbz6o/R3iLj17SXJe7ThK5EmqVSn3JxFTRtvlHIeWeqv5EyVbtyzqG ogK+WtUHksGekuUaf0J/RPaz7lh7M0h64iGaZjpWWDzFXm7p0Oynx5P3FMX8LZqx Lr/6leRgnofAgWD30dK/BAHJmqEnka+uHboE87nU1oSmdnSZ9UR9f066/C415ftb Uuy0KM71/r8oy+T98GNxuirGXxnA1EFJ4hcKg8KC6/xv95x5y6MkrqEwp9/hkUNJ pDPozXerBSKQt+CDbm+KGXP1C/5otHwUzgl5VvZKwZj7W0CKRl1TMA7SLSnrBWc= =zVfx -----END PGP SIGNATURE----- --aOpHDQtNx64G95usMMPfiwiGFD68eWlSB--