From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 113649 invoked by alias); 27 Feb 2018 07:22:23 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 113636 invoked by uid 89); 27 Feb 2018 07:22:22 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS autolearn=ham version=3.3.2 spammy=Recent, Hx-languages-length:1921, states X-HELO: mail-vk0-f67.google.com Received: from mail-vk0-f67.google.com (HELO mail-vk0-f67.google.com) (209.85.213.67) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 27 Feb 2018 07:22:21 +0000 Received: by mail-vk0-f67.google.com with SMTP id s1so3852419vke.5 for ; Mon, 26 Feb 2018 23:22:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=5pvlUv03KSr6UztkNjHq2NQ8FM1qFElkiiTa0wP6Sns=; b=BEaX1YNiU2nqQXv6bf591y4P8mGIgxwR8sa2/VPU6q60MFS0ansa1nyUCaNggS7puI Biyqf2IjiXmc0Cj1EAUFU51ymzX0U56bD1k6jtz/5cyK357LSt3UFpWUfjvEgXlOPcez MjY1cW2uXjef8EHJXCNerLiHZh+zyzSpAg48dFLwKMPtaDJQZPXuEvWFScQhhrbaQJ/a o4mPQfFUaHVklKY++jbm8jAY3IFAl5X1eBmf4erUUP59yMegjNpAh+SqItWnmdPBROmK vxwCfaNQFRn6Br/kOLZqcX6+8HIcyzlgzg0+0b8sgTlqhst1+QptOzxnRA6AtFbSgScJ yGDA== X-Gm-Message-State: APf1xPAiSaMK2hL071vjUG+8sSsbAD/g3DtCvDbsCHdHUW4W7gEC/mi2 Om7QSLD5l+5RimE0ue1NWXW1T7TWNuuLn7lXPS3G1w== X-Google-Smtp-Source: AG47ELsh+eyi+zAdwVhW4GS7ej2GZQhGjzTK+xQfEe7OkQbfq5lKRbx6T+RD5/DAjZ175Tx/VDTLq+SXWXwWQe+U1G8= X-Received: by 10.31.233.135 with SMTP id g129mr10234700vkh.177.1519716138969; Mon, 26 Feb 2018 23:22:18 -0800 (PST) MIME-Version: 1.0 Received: by 10.176.3.115 with HTTP; Mon, 26 Feb 2018 23:22:18 -0800 (PST) From: Orgad Shaneh Date: Tue, 27 Feb 2018 07:22:00 -0000 Message-ID: Subject: gawk Regression: CR characters are not stripped on Windows To: cygwin@cygwin.com, bug-gawk@gnu.org, Eli Zaretskii Content-Type: multipart/mixed; boundary="94eb2c0948209f929f05662c7cba" X-SW-Source: 2018-02/txt/msg00280.txt.bz2 --94eb2c0948209f929f05662c7cba Content-Type: text/plain; charset="UTF-8" Content-length: 1862 Hi, Cross-posting per Eli Zaretskii's request. CR characters used to be automatically stripped on Windows (MSYS2 and Cygwin environments). This is broken in 4.2.0. Minimal example: echo -en "foo\r\n\r\nbar\r\n" > foo.txt awk '/^$/ { print "found" }' foo.txt # This worked with 4.1.4 and doesn't work with 4.2.0 awk '/^\r$/ { print "found" }' foo.txt # This works with 4.2.0 and doesn't work with 4.1.4 Bisected to commit 5db38f775d9ba239e125d81dff2010a2ddacb48e: (* gawkmisc.c (cygwin_premain0, cygwin_premain2): Remove. No longer needed). Apparently it's still needed... This issue was reported in https://github.com/git-for-windows/git/issues/1524 Proposed patch is attached. As Eli said, this change was deliberate. But this has several drawbacks. 1. The gawk info page states that: > Under MS-Windows, 'gawk' (and many other text programs) silently > translates end-of-line '\r\n' to '\n' on input and '\n' to '\r\n' on > output. and on Feb 8 the following section was added: > Recent versions of Cygwin open all files in binary mode. This means > that you should use 'RS = "\r?\n"' in order to be able to handle > standard MS-Windows text files with carriage-return plus line-feed line > endings. This breaks compatibility between different gawk versions. What were the reasons for this change in cygwin, and why was it pushed upstream? 2. Git and other tools automatically convert text files to CRLF on Windows. This means that any awk script that runs on both platforms must use RS = "\r?\n". One example that was broken by this behavior change is gerrit's commit-msg hook[1], which scans for empty lines by /^$/ regexp. Please consider reverting this change. Patch attached. [1] https://gerrit.googlesource.com/gerrit/+/376a7bbb64f1b3f13c261f4efa0af0e8538cfe9b/resources/com/google/gerrit/server/tools/root/hooks/commit-msg#101 - Orgad --94eb2c0948209f929f05662c7cba Content-Type: application/octet-stream; name="0001-Revert-default-mode-on-Cygwin-from-binary-back-to-te.patch" Content-Disposition: attachment; filename="0001-Revert-default-mode-on-Cygwin-from-binary-back-to-te.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_je5bqs0a0 Content-length: 6304 RnJvbSA0YmY5MGUwMmIzMTk0MDViODA3NzU1ZDZkMTZlMjE4ZTYyOTVkMjA3 IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBPcmdhZCBTaGFuZWgg PG9yZ2Fkc0BnbWFpbC5jb20+CkRhdGU6IE1vbiwgMjYgRmViIDIwMTggMDg6 MTA6NDIgKzAyMDAKU3ViamVjdDogW1BBVENIXSBSZXZlcnQgZGVmYXVsdCBt b2RlIG9uIEN5Z3dpbiBmcm9tIGJpbmFyeSBiYWNrIHRvIHRleHQuCgoxLiBU aGUgZ2F3ayBpbmZvIHBhZ2Ugc3RhdGVzIHRoYXQ6Cgo+IFVuZGVyIE1TLVdp bmRvd3MsICdnYXdrJyAoYW5kIG1hbnkgb3RoZXIgdGV4dCBwcm9ncmFtcykg c2lsZW50bHkKPiB0cmFuc2xhdGVzIGVuZC1vZi1saW5lICdcclxuJyB0byAn XG4nIG9uIGlucHV0IGFuZCAnXG4nIHRvICdcclxuJyBvbgo+IG91dHB1dC4K ClRoaXMgYnJlYWtzIGNvbXBhdGliaWxpdHkgYmV0d2VlbiBkaWZmZXJlbnQg Z2F3ayB2ZXJzaW9ucy4KCjIuIEdpdCBhbmQgb3RoZXIgdG9vbHMgYXV0b21h dGljYWxseSBjb252ZXJ0IHRleHQgZmlsZXMgdG8gQ1JMRiBvbiBXaW5kb3dz LgpUaGlzIG1lYW5zIHRoYXQgYW55IGF3ayBzY3JpcHQgdGhhdCBydW5zIG9u IGJvdGggcGxhdGZvcm1zIG11c3QgdXNlIFJTID0KIlxyP1xuIi4gT25lIGV4 YW1wbGUgdGhhdCB3YXMgYnJva2VuIGJ5IHRoaXMgYmVoYXZpb3IgY2hhbmdl IGlzIGdlcnJpdCdzCmNvbW1pdC1tc2cgaG9va1sxXSwgd2hpY2ggc2NhbnMg Zm9yIGVtcHR5IGxpbmVzIGJ5IC9eJC8gcmVnZXhwLgoKWzFdIGh0dHBzOi8v Z2Vycml0Lmdvb2dsZXNvdXJjZS5jb20vZ2Vycml0LysvMzc2YTdiYmI2NGYx YjNmMTNjMjYxZjRlZmEwYWYwZTg1MzhjZmU5Yi9yZXNvdXJjZXMvY29tL2dv b2dsZS9nZXJyaXQvc2VydmVyL3Rvb2xzL3Jvb3QvaG9va3MvY29tbWl0LW1z ZyMxMDEKClRoaXMgcmV2ZXJ0cyBjb21taXRzIDVkYjM4Zjc3NWQ5YmEyMzll MTI1ZDgxZGZmMjAxMGEyZGRhY2I0OGUKYW5kIHBhcnRpYWxseSA0MTk3MDYz MzM1ZDc1OGQwZWFjYjhiNGNiNzRiMjMyYWQ1OTZjNzg0LgotLS0KIGRvYy9D aGFuZ2VMb2cgICAgfCAgNCArKysrCiBkb2MvZ2F3ay5pbmZvICAgIHwgIDgg LS0tLS0tLS0KIGRvYy9nYXdrLnRleGkgICAgfCAgNSAtLS0tLQogZG9jL2dh d2t0ZXhpLmluICB8ICA1IC0tLS0tCiBwb3NpeC9DaGFuZ2VMb2cgIHwgIDUg KysrKysKIHBvc2l4L2dhd2ttaXNjLmMgfCAxOSArKysrKysrKysrKysrKysr KysrCiA2IGZpbGVzIGNoYW5nZWQsIDI4IGluc2VydGlvbnMoKyksIDE4IGRl bGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL2RvYy9DaGFuZ2VMb2cgYi9kb2Mv Q2hhbmdlTG9nCmluZGV4IGZlNDhlMDg4Li5mOWJmNTg3MyAxMDA2NDQKLS0t IGEvZG9jL0NoYW5nZUxvZworKysgYi9kb2MvQ2hhbmdlTG9nCkBAIC0xLDMg KzEsNyBAQAorMjAxOC0wMi0yNyAgICAgICAgIE9yZ2FkIFNoYW5laCAgICAg PG9yZ2Fkc0BnbWFpbC5jb20+CisKKwkqIFJldmVydCBkZWZhdWx0IG1vZGUg ZnJvbSBiaW5hcnkgYmFjayB0byB0ZXh0IG9uIEN5Z3dpbi4KKwogMjAxOC0w Mi0xNyAgICAgICAgIEFybm9sZCBELiBSb2JiaW5zICAgICA8YXJub2xkQHNr ZWV2ZS5jb20+CiAKIAkqIGdhd2t0ZXhpLmluOiBGdXJ0aGVyIGZpeCB0byBO T05GQVRBTCBzdHVmZi4gVGhhbmtzIHRvCmRpZmYgLS1naXQgYS9kb2MvZ2F3 ay5pbmZvIGIvZG9jL2dhd2suaW5mbwppbmRleCA2YWQzNGFlOS4uYTk5Njg0 MDEgMTAwNjQ0Ci0tLSBhL2RvYy9nYXdrLmluZm8KKysrIGIvZG9jL2dhd2su aW5mbwpAQCAtMjkyMzgsMTQgKzI5MjM4LDYgQEAgQ3lnd2luIGlzIHRoZSBz YW1lIGFzIGZvciBhIFVuaXggc3lzdGVtOgogb24gQ3lnd2luIHRha2VzIGNv bnNpZGVyYWJseSBsb25nZXIuICBIb3dldmVyLCBpdCBkb2VzIGZpbmlzaCwg YW5kIHRoZW4KIHRoZSAnbWFrZScgcHJvY2VlZHMgYXMgdXN1YWwuCiAKLSAg IFJlY2VudCB2ZXJzaW9ucyBvZiBDeWd3aW4gb3BlbiBhbGwgZmlsZXMgaW4g YmluYXJ5IG1vZGUuICBUaGlzIG1lYW5zCi10aGF0IHlvdSBzaG91bGQgdXNl ICdSUyA9ICJccj9cbiInIGluIG9yZGVyIHRvIGJlIGFibGUgdG8gaGFuZGxl Ci1zdGFuZGFyZCBNUy1XaW5kb3dzIHRleHQgZmlsZXMgd2l0aCBjYXJyaWFn ZS1yZXR1cm4gcGx1cyBsaW5lLWZlZWQgbGluZQotZW5kaW5ncy4KLQotICAg VGhlIEN5Z3dpbiBlbnZpcm9ubWVudCBzdXBwb3J0cyBib3RoIHRoZSAnfCYn IG9wZXJhdG9yIGFuZCBUQ1AvSVAKLW5ldHdvcmtpbmcgKCpub3RlIFRDUC9J UCBOZXR3b3JraW5nOjopLgotCiAfCiBGaWxlOiBnYXdrLmluZm8sICBOb2Rl OiBNU1lTLCAgUHJldjogQ3lnd2luLCAgVXA6IFBDIEluc3RhbGxhdGlvbgog CmRpZmYgLS1naXQgYS9kb2MvZ2F3ay50ZXhpIGIvZG9jL2dhd2sudGV4aQpp bmRleCA1YzgxMmM3ZC4uMzY3ZGY0NDYgMTAwNjQ0Ci0tLSBhL2RvYy9nYXdr LnRleGkKKysrIGIvZG9jL2dhd2sudGV4aQpAQCAtMzk4NzMsMTEgKzM5ODcz LDYgQEAgV2hlbiBjb21wYXJlZCB0byBHTlUvTGludXggb24gdGhlIHNhbWUg c3lzdGVtLCB0aGUgQHNhbXB7Y29uZmlndXJlfQogc3RlcCBvbiBDeWd3aW4g dGFrZXMgY29uc2lkZXJhYmx5IGxvbmdlci4gIEhvd2V2ZXIsIGl0IGRvZXMg ZmluaXNoLAogYW5kIHRoZW4gdGhlIEBzYW1we21ha2V9IHByb2NlZWRzIGFz IHVzdWFsLgogCi1SZWNlbnQgdmVyc2lvbnMgb2YgQ3lnd2luIG9wZW4gYWxs IGZpbGVzIGluIGJpbmFyeSBtb2RlLiBUaGlzIG1lYW5zCi10aGF0IHlvdSBz aG91bGQgdXNlIEBzYW1we1JTID0gIlxyP1xuIn0gaW4gb3JkZXIgdG8gYmUg YWJsZSB0bwotaGFuZGxlIHN0YW5kYXJkIE1TLVdpbmRvd3MgdGV4dCBmaWxl cyB3aXRoIGNhcnJpYWdlLXJldHVybiBwbHVzCi1saW5lLWZlZWQgbGluZSBl bmRpbmdzLgotCiBUaGUgQ3lnd2luIGVudmlyb25tZW50IHN1cHBvcnRzCiBi b3RoIHRoZSBAc2FtcHt8Jn0gb3BlcmF0b3IgYW5kIFRDUC9JUCBuZXR3b3Jr aW5nCiAoQHB4cmVme1RDUC9JUCBOZXR3b3JraW5nfSkuCmRpZmYgLS1naXQg YS9kb2MvZ2F3a3RleGkuaW4gYi9kb2MvZ2F3a3RleGkuaW4KaW5kZXggMGNk M2JkNDcuLmRiNTU2MDhlIDEwMDY0NAotLS0gYS9kb2MvZ2F3a3RleGkuaW4K KysrIGIvZG9jL2dhd2t0ZXhpLmluCkBAIC0zODg0NywxMSArMzg4NDcsNiBA QCBXaGVuIGNvbXBhcmVkIHRvIEdOVS9MaW51eCBvbiB0aGUgc2FtZSBzeXN0 ZW0sIHRoZSBAc2FtcHtjb25maWd1cmV9CiBzdGVwIG9uIEN5Z3dpbiB0YWtl cyBjb25zaWRlcmFibHkgbG9uZ2VyLiAgSG93ZXZlciwgaXQgZG9lcyBmaW5p c2gsCiBhbmQgdGhlbiB0aGUgQHNhbXB7bWFrZX0gcHJvY2VlZHMgYXMgdXN1 YWwuCiAKLVJlY2VudCB2ZXJzaW9ucyBvZiBDeWd3aW4gb3BlbiBhbGwgZmls ZXMgaW4gYmluYXJ5IG1vZGUuIFRoaXMgbWVhbnMKLXRoYXQgeW91IHNob3Vs ZCB1c2UgQHNhbXB7UlMgPSAiXHI/XG4ifSBpbiBvcmRlciB0byBiZSBhYmxl IHRvCi1oYW5kbGUgc3RhbmRhcmQgTVMtV2luZG93cyB0ZXh0IGZpbGVzIHdp dGggY2FycmlhZ2UtcmV0dXJuIHBsdXMKLWxpbmUtZmVlZCBsaW5lIGVuZGlu Z3MuCi0KIFRoZSBDeWd3aW4gZW52aXJvbm1lbnQgc3VwcG9ydHMKIGJvdGgg dGhlIEBzYW1we3wmfSBvcGVyYXRvciBhbmQgVENQL0lQIG5ldHdvcmtpbmcK IChAcHhyZWZ7VENQL0lQIE5ldHdvcmtpbmd9KS4KZGlmZiAtLWdpdCBhL3Bv c2l4L0NoYW5nZUxvZyBiL3Bvc2l4L0NoYW5nZUxvZwppbmRleCAxYjc0ODA0 Yi4uYWYxZDUxYTIgMTAwNjQ0Ci0tLSBhL3Bvc2l4L0NoYW5nZUxvZworKysg Yi9wb3NpeC9DaGFuZ2VMb2cKQEAgLTEsMyArMSw4IEBACisyMDE4LTAyLTI2 ICAgICAgICAgT3JnYWQgU2hhbmVoICAgICAgPG9yZ2Fkc0BnbWFpbC5jb20+ CisKKwkqIGdhd2ttaXNjLmMgKGN5Z3dpbl9wcmVtYWluMCwgY3lnd2luX3By ZW1haW4yKTogUmVjb3Zlci4KKwlCcm9rZSBzdHJpcHBpbmcgb2YgXHIgb24g V2luZG93cy4KKwogMjAxNy0xMC0xOSAgICAgICAgIEFybm9sZCBELiBSb2Ji aW5zICAgICA8YXJub2xkQHNrZWV2ZS5jb20+CiAKIAkqIDQuMi4wOiBSZWxl YXNlIHRhciBiYWxsIG1hZGUuCmRpZmYgLS1naXQgYS9wb3NpeC9nYXdrbWlz Yy5jIGIvcG9zaXgvZ2F3a21pc2MuYwppbmRleCBlY2Y1YWZmOS4uNDA5NTA4 MjYgMTAwNjQ0Ci0tLSBhL3Bvc2l4L2dhd2ttaXNjLmMKKysrIGIvcG9zaXgv Z2F3a21pc2MuYwpAQCAtMjg5LDMgKzI4OSwyMiBAQCB2b2lkCiBpbml0X3Nv Y2tldHModm9pZCkKIHsKIH0KKworI2lmZGVmIF9fQ1lHV0lOX18KK3ZvaWQK K2N5Z3dpbl9wcmVtYWluMChpbnQgYXJnYywgY2hhciAqKmFyZ3YsIHN0cnVj dCBwZXJfcHJvY2VzcyAqbXlzZWxmKQoreworCXN0YXRpYyBzdHJ1Y3QgX19j eWd3aW5fcGVyZmlsZSBwZltdID0geworCQl7ICIiLCBPX1JET05MWSB8IE9f VEVYVCB9LAorCQkvKnsgIiIsIE9fV1JPTkxZIHwgT19CSU5BUlkgfSwqLwor CQl7IE5VTEwsIDAgfQorCX07CisJY3lnd2luX2ludGVybmFsKENXX1BFUkZJ TEUsIHBmKTsKK30KKwordm9pZAorY3lnd2luX3ByZW1haW4yKGludCBhcmdj LCBjaGFyICoqYXJndiwgc3RydWN0IHBlcl9wcm9jZXNzICpteXNlbGYpCit7 CisJc2V0bW9kZShmaWxlbm8gKHN0ZGluKSwgT19URVhUKTsKK30KKyNlbmRp ZgotLSAKMi4xNi4yCgo= --94eb2c0948209f929f05662c7cba Content-Type: text/plain; charset=us-ascii Content-length: 219 -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple --94eb2c0948209f929f05662c7cba--