From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 4648 invoked by alias); 18 Jul 2018 01:52:45 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 2536 invoked by uid 89); 18 Jul 2018 01:52:44 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-1.1 required=5.0 tests=BAYES_00,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL autolearn=no version=3.3.2 spammy=H*r:sk:sonic30, HX-HELO:sk:sonic30, Hx-spam-relays-external:sk:sonic30, H*RU:sk:sonic30 X-HELO: sonic303-4.consmr.mail.bf2.yahoo.com Received: from sonic303-4.consmr.mail.bf2.yahoo.com (HELO sonic303-4.consmr.mail.bf2.yahoo.com) (74.6.131.43) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 18 Jul 2018 01:52:42 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1531878760; bh=ah/xITCoo1u8bw0l1JXaJk9z4IxBaWe3utBLC5xddlo=; h=Subject:From:To:References:Date:In-Reply-To:From:Subject; b=RoTEGaDWlLK/a06zbBuIeD9xi6/LcSoBgJBRCSidaDi4KPVKp1Pksfebp1UB1pcPI5p3pL86uqjCZAkZibTdF4fGI35N8Wwu/x3OiGoDRXA5K3VMREdzyZxkvUQhPfZrIUtoi7XeijV7Aq9KLcHvXBP+xHTaxP7NjTOokb9APxWAHUrKiAhMK6NGCvsIuDdLHIupIVcGwGm7DTWoWLIFE/jgNj7Vmtbg74mwxXY+f7n20egaGzgdWEeEodTpPQuNfMiL1lEBlX+0OU9Vmy25/J1bVe5QAEdA4PX9QXnvJXBRfhDb078itI2pkU3U7tkbw6OhW/KDptLQAfZjWusEKA== Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.bf2.yahoo.com with HTTP; Wed, 18 Jul 2018 01:52:40 +0000 Received: from pool-173-66-240-59.washdc.fios.verizon.net (EHLO [192.168.1.8]) ([173.66.240.59]) by smtp409.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 1dd97bb43161d343d9a29e31d3082bca for ; Wed, 18 Jul 2018 01:52:40 +0000 (UTC) Subject: Re: BASH 4.4 mapfile/readarray/read builtins mis-behaving with pipe [edit] documentation bug From: BloomingAzaleas To: cygwin@cygwin.com References: <69b0bc3c-7ead-920e-f04b-ec631c3453b7@verizon.net> Message-ID: <2beb175e-2291-6b12-6efd-c84704f6762f@verizon.net> Date: Wed, 18 Jul 2018 11:17:00 -0000 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 In-Reply-To: <69b0bc3c-7ead-920e-f04b-ec631c3453b7@verizon.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-SW-Source: 2018-07/txt/msg00168.txt.bz2 Reply to Steven Penny : no mis-behaving: this is intended behavior - you yourself have given workarounds: either redirect output to a file that can be later read, o= r pipe to command grouping ala {} or () and read stdin from inside the subshell im not sure what you are asking here - it seems you have a grasp alread= y of the "problem" and "solution" - so nothing more needs to be said really, as = nothing needs to be fixed - another option is "lastpipe" If the reported behavior is the intended behavior, then this=20 complaint is now a documentation complaint.=C2=A0 Neither the Cygwin=20 man pages (quoted in the original report) nor the upstream GNU=20 documentation indicate any exception to stdin redirect with pipe=20 in the doc paragraphs for the subject builtins.=C2=A0 What confuses=20 the situation is that some stdin re-directions work for these=20 builtins in a parent shell context (left-chevs)=C2=A0 but not others=20 (pipes) UNLESS performed strictly at mid-night under a full moon=20 using obscure incantation hidden in pieces throughout a 178-page=20 pdf.=C2=A0 In the GNU doc, the sub-shell for builtins in a pipeline=20 rule involved is obscured by burial in paragraph six of section=20 3.7.3 Command Execution Environment. I suggest the following adjustment to the man pages inserting a=20 parenthetical cue regards behavior in pipes: =C2=A0 mapfile [-d delim] [-n count] [-O origin] [-s count] [-t] [-u=20 fd]=C2=A0 [-C=C2=A0 callback]=C2=A0 [-c quantum] [array] =C2=A0 readarray=C2=A0 [-d delim] [-n count] [-O origin] [-s count] [-t]=20 [-u fd] [-C callback] [-c =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 quantum] [ar= ray] =C2=A0=C2=A0=C2=A0 Read lines from the standard input (see pipeline=20 restriction) into the indexed array=C2=A0 variable=C2=A0 array,=C2=A0 or ... =C2=A0read [-ers] [-a aname] [-d delim] [-i text] [-n nchars] [-N=20 nchars]=C2=A0 [-p=C2=A0 prompt]=C2=A0 [-t timeout] [-u fd] [name ...] =C2=A0=C2=A0=C2=A0=C2=A0 One=C2=A0 line=C2=A0 is read from the standard in= put (see pipeline=20 restriction), or ... At least the reader gets a cue to modify expectations and=20 research the "pipeline restriction" compared to all other=20 commands that work just peachy with stdin pipe re-direction.=C2=A0=20 Regards the shopt lastpipe doc single-sentence paragraph in GNU=20 section 4.3.2 The Shopt Builtin, it could be improved by adding:=20 "This option will change the parent shell context behavior of=20 shell builtins mapfile, readarray and read when they are used as=20 the last command of a pipeline." As to the left-chev work-arounds, yes they work, but that would=20 require some script re-structuring.=C2=A0 In any case, "what you are=20 asking here" is that there appears to be a difference between a=20 plain reading of the doc paragraphs for the builtins and the=20 builtins behavior and so a bug against one or the other.=C2=A0 If this=20 were a unix/linux kernel environment, the triage emphasis would=20 be documentation wording rather than implementation bug.=C2=A0 Given=20 the Cygwin context of necessary hidden emulation chicanery on=20 Windows, triage emphasis is not that obvious.=C2=A0 I exerted the=20 effort to construct a reasonably complete report with multiple=20 illustration cases contrasted to the doc to improve the FOSS, be=20 it implementation or documentation.=C2=A0 I didn't need too. Regards. On 7/15/2018 7:01 PM, BloomingAzaleas wrote: > Windows 10 Pro 10.0.17134 N/A Build 17134 patched through 15=20 > July 2018 > Cygwin 2.10.0(0.325/5/3) > Bash 4.4.12(3) > > Cygwin man pages show: > > =C2=A0 mapfile [-d delim] [-n count] [-O origin] [-s count] [-t] [-u=20 > fd]=C2=A0 [-C=C2=A0 callback]=C2=A0 [-c quantum] [array] > =C2=A0 readarray=C2=A0 [-d delim] [-n count] [-O origin] [-s count] [-t]= =20 > [-u fd] [-C callback] [-c =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 quantum] [= array] > =C2=A0=C2=A0=C2=A0 Read lines from the standard input into the indexed ar= ray=20 > variable=C2=A0 array,=C2=A0 or ... > > =C2=A0read [-ers] [-a aname] [-d delim] [-i text] [-n nchars] [-N=20 > nchars]=C2=A0 [-p=C2=A0 prompt]=C2=A0 [-t timeout] [-u fd] [name ...] > =C2=A0=C2=A0=C2=A0=C2=A0 One=C2=A0 line=C2=A0 is read from the standard i= nput, or ... > > So the expectation is that all bash stdin re-directions such as=20 > pipes and left chevs should work.=C2=A0 For mapfile/readarry, the=20 > output array variable MAPFILE is described as being created if=20 > not array_var argument is given to mapfile. > > A) Searched cygwin email list with terms 'mapfile', 'readarray'=20 > and 'read builtin'.=C2=A0 No obvious hits of recent vintage in=20 > summary result list. > > B) Confirm, not using mapfile/readarry/read, that bash stdin=20 > redirs work as expected. > > echo multi-line_arg | /bin/cat > > /bin/cat < some_unix_fmt_file > > /bin/cat < some > lines > END > > all work as expected. > > C) Test mapfile/readarray/read in various stdin redir=20 > situations.=C2=A0 Basic harnesses are : > > mapfile/readarry HARNESS: > stdin or fd 0 feed to mapfile > then > : "${MAPFILE[@]:?MAPFILE null or unset}" > if get past the ${:?} test then dump MAPFILE with a simple /for=20 > /loop as: for ent in "${MAPFILE[@]}" ; do echo =3D"${ent}"=3D ; done > > read HARNESS: > stdin feed to 'read foo' > then > : "${foo:?foo null or unset}" > if get past the ${:?} test then dump variable foo with echo=20 > =3D"${foo}"=3D > > CASES: > echo multi-line_arg | mapfile > cat multi-line_unix_fmt_file | mapfile > RESULT: > Fail with MAPFILE null or unset > If 'for' loop forced, "${MAPFILE[@]}" expands to zero words. > > CASE: > echo multi-line_arg | read foo > cat multi-line_unix_fmt_file | read foo > =C2=A0RESULT: > Fail with variable foo null or unset > > CASES: left chev re-directions > mapfile < some_unix_fmt_file > mapfile < read < some_unix_fmt_file > read < RESULT: > Work as expected > > CASES: mapfile or read in a sub-shell > echo multi-line_arg | ( mapfile ; stuff ) > echo multi-line_arg | ( read foo; stuff ) > RESULT: > Work as expected.=C2=A0 However, since mapfile/read result vars are=20 > not accessible in parent shell context, can force a notable=20 > re-structuring of a script to use sub-shells. > > CONCLUSIONS: > For bash builtins mapfile/readarray and read, stdin pipe=20 > re-direction directly to the builtin fails as if the builtin=20 > read()/fread() sees a silent pre-mature EOF on the pipe.=C2=A0 In=20 > the sub-shell tests, the pipe re-direction happens for the=20 > sub-shell as a whole before commands internal to the sub-shell=20 > get processed for individual re-dirs, so no re-dir processing=20 > directly for the builtin. > -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple