From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 31385 invoked by alias); 5 Sep 2016 23:37:21 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 31373 invoked by uid 89); 5 Sep 2016 23:37:20 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-0.0 required=5.0 tests=AWL,BAYES_50,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2 spammy=kitchen, viable, portability, eating X-HELO: mail-it0-f52.google.com Received: from mail-it0-f52.google.com (HELO mail-it0-f52.google.com) (209.85.214.52) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Mon, 05 Sep 2016 23:37:10 +0000 Received: by mail-it0-f52.google.com with SMTP id e124so166422376ith.0 for ; Mon, 05 Sep 2016 16:37:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Z0AooWgfAoyGHZFzuWiUNxs1xZ8XxZuJ2uqAoQOTWXM=; b=cnxobFSkKXno4ccy2QQiqreEXOp6oWJIkbTvC5RaqywZvArZzx4ivZCZrqJwxuTqIQ Oqu1m3hoYNASoAGzKcjUDabSzDVK73WLuAwOwMRpzuSYZX0vx/glP9lHeU1I3VYeqdPs buBpAFG+0YxYG9HRs+BAA+YY7kwBZK5Xf7Op80rhjUcb/XZJfxqoI1ePEzfUa5pupZ0k cOcXKjMzqngblBALynf4il4kEmkXCctcdC91dBD/UeKQj6RcFs2xPfuaonQ9vuUUT9ir M6qszl1NiZzBvuMdKSUKSEJimdlBVp1DdjJF/StO16WF3rpWb+zuMxLmih5fhLVd3FXP OkVg== X-Gm-Message-State: AE9vXwPN6F/+7yw+U6yLpaiv56m32rpv5rZsJ4GBRa2C0tTHsQEw1j5X6SKIjuNzwC4sNF1ANHQ626HGfi5fRA== X-Received: by 10.107.53.213 with SMTP id k82mr1865091ioo.133.1473118628643; Mon, 05 Sep 2016 16:37:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.116.26 with HTTP; Mon, 5 Sep 2016 16:37:08 -0700 (PDT) In-Reply-To: References: <93be816b-952c-20cd-575e-940cdf4fbbd1@redhat.com> <1de2efdc-b26c-4914-580c-1a640d0a46fd@redhat.com> <7314ffba-a927-f565-e38e-7454d6c2ef0f@gmail.com> From: Gene Pavlovsky Date: Mon, 05 Sep 2016 23:37:00 -0000 Message-ID: Subject: Re: Script broken after updating bash to 4.3.46-7? To: cygwin@cygwin.com Content-Type: text/plain; charset=UTF-8 X-IsSubscribed: yes X-SW-Source: 2016-09/txt/msg00088.txt.bz2 Lee, you have a point. Adding mysql.sh that runs `mysql.exe | d2u`, or using a cygwin version of mysql looks like the most viable solution at the moment. Or staying with the older version of bash, in which `read` still handled `\r\n` line endings... My point is OK I'll fix that script, who knows what other scripts might be broken now? And each one might need an investigation before I find out which program's windows version needs a wrapper script etc. --Gene On 4 September 2016 at 20:37, Lee wrote: > On 9/4/16, Gene Pavlovsky wrote: >> This issue never affected me personally, but it sounds like a serious >> bug and I'm as glad as anybody that it's finally fixed. >> However, having existing scripts suddenly breaking is not great. I'd >> like to remind that I'm not the author of the script in question, >> [AutoMySQLBackup](http://sourceforge.net/projects/automysqlbackup/). >> If I put "| d2u" there I'll have to remember it and do it every time >> automysqlbackup is updated ... > > Right - updating something like automysqlbackup to add "| d2u" after > every dos program call is a non-starter. But would putting the dos > program inside a script that converted dos -> unix line endings work? > eg - have a mysql script that comes before the windows version of > mysql in your path that does something like > /path/to/windows/version/of/mysql $@ | dos2unix > >> - or create and maintain a Cygwin package for this script. > > It seems to me that it isn't the script that's broken - it's the whole > idea of having a dos program transparently integrate into an > environment that expects unix line endings that's broken. So until > there's a bash option that automatically translates '\r\n' line > endings into '\n' line endings you're stuck doing some kind of > work-around or using a cygwin version of mysql. > > Lee > > >> And who knows how many other scripts might be broken, >> I just didn't find it yet? >> >> Having a `read`-specific shell option telling read to treat `\r\n` >> (and only `\r\n`, not \r followed by something else) same as `\n` >> would be bad things to have? For me, this kind of option would be more >> useful than the `igncr` crutch >> Let me say it another way - in OOP programming, one of good practices >> is Single Responsibility Principle - a class should be responsible for >> only one feature/function, and that feature/function should be totally >> encapsulated in that class. Similar to that, an option should be >> responsible for one behavior. With this change to `read`, the `igncr` >> shell option is starting to look like a kitchen sink... split it into >> separate options, please! >> I think making UNIX scripts work on Cygwin with no or minimum >> modifications (or bug-hunting) should be one of high priorities, no? >> If some scripts erroneously have CRLF line endings, it's easy to find >> and `d2u` them, rather than using the `igncr` crutch, but with the >> recent change to `read`, countless scripts might be broken in a >> non-obvious way. Fixing them would require finding out they're broken, >> in the first place. Imagine if I didn't set up my cron to e-mail me >> the cron jobs output? My backup script would just stop working >> silently, and some time later when I needed a recent backup, I would >> find out there aren't any. There might be something else lurking that >> I haven't found yet. Once a script, broken by this change to `read`, >> is found, it must be checked thoroughly to find out where exactly is >> the problem, where to put '| d2u', or maybe 'set -o igncr'. These >> fixes must also be applied anytime a 3rd party script is updated. >> Quite a lot of work! >> Hope you will consider my point. >> Regards, >> Gene. >> >> On 30 August 2016 at 23:57, cyg Simple wrote: >>> On 8/30/2016 1:38 PM, Eric Blake wrote: >>>> On 08/30/2016 12:04 PM, cyg Simple wrote: >>>>> On 8/29/2016 2:30 PM, Eric Blake wrote: >>>>>> >>>>>> Simplest fix: >>>>>> >>>>>> read ... < <(mysql ... | dos2unix) >>>>>> >>>>> >>>>> This will break when the data returned by mysql is supposed to contain >>>>> \r. >>>>> >>>>>> There. Now you aren't feeding \r to read in the first place. >>>>>> >>>>> >>>>> But you might want to feed \r to read. It isn't a fix, it is a >>>>> potential work around dependent on the data set results. If a read >>>>> that >>>>> is supposed to be reading binary data doesn't pass all of the data to >>>>> the routine then it is broken. >>>> >>>> Now we're talking past each other. >>>> >>>> That's what the recent bash fixed. 'read' in bash 3.2.42-4 was broken - >>>> it corrupted binary data, with no recourse, by eating \r (and worse, by >>>> sometimes eating the byte after \r). 'read' in bash 3.2.46-7 is fixed - >>>> by default it is strictly binary (all bytes are read as-is, including >>>> \r), but can also be switched to text mode (using 'igncr', all \r are >>>> ignored). If you want to preserve mid-line \r but treat line endings of >>>> \r\n as a single byte, then leave binary mode on and strip the line >>>> endings via a separate tool like d2u (note, however, that it is very >>>> rare to have data where mid-line \r is important but line-ending \r\n >>>> should be treated as plain \n). >>>> >>>> I strongly think that using igncr is a crutch, and you normally >>>> shouldn't use it; particularly not if you want to be portable to other >>>> platforms. Instead, massaging your data through d2u is a great way to >>>> be portable. But sometimes the ease of ignoring ALL \r is easier than >>>> worrying about portability, so I keep the 'igncr' code in Cygwin. >>>> >>>> And it is only because the OP tried using 'igncr' in the first place >>>> (whether or not it was actually needed) that we have now flushed out the >>>> existence of a latent bug in the 'igncr' implementation that interacts >>>> weirdly with $()\n in PS1. On that front, I'm still hoping to find time >>>> to debug and/or for someone to post a patch. But whether PS1 behaves >>>> weirdly under 'igncr' is orthogonal to my suggestion above - using >>>> 'mysql|d2u' is a great way to avoid the need to worry about 'igncr'. >>>> >>> >>> Thank you for the retort Eric. Happy to know that it is fixed which in >>> the back of my mind I knew already. I can imagine data such as full >>> message email or a small document data containing \r\n as valid data in >>> the database field and if you use a line ending conversion utility you >>> might loose that data. >>> >>> -- >>> cyg Simple >>> >>> -- >>> Problem reports: http://cygwin.com/problems.html >>> FAQ: http://cygwin.com/faq/ >>> Documentation: http://cygwin.com/docs.html >>> Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple >>> >> >> -- >> Problem reports: http://cygwin.com/problems.html >> FAQ: http://cygwin.com/faq/ >> Documentation: http://cygwin.com/docs.html >> Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple >> >> > > -- > Problem reports: http://cygwin.com/problems.html > FAQ: http://cygwin.com/faq/ > Documentation: http://cygwin.com/docs.html > Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple > -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple