From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from conssluserg-03.nifty.com (conssluserg-03.nifty.com [210.131.2.82]) by sourceware.org (Postfix) with ESMTPS id DA8763858D39 for ; Mon, 15 Nov 2021 14:50:32 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org DA8763858D39 Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=nifty.ne.jp Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=nifty.ne.jp Received: from Express5800-S70 (z221123.dynamic.ppp.asahi-net.or.jp [110.4.221.123]) (authenticated) by conssluserg-03.nifty.com with ESMTP id 1AFEoIhj018837 for ; Mon, 15 Nov 2021 23:50:18 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conssluserg-03.nifty.com 1AFEoIhj018837 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.ne.jp; s=dec2015msa; t=1636987818; bh=8ANjCNmVDc1WIPaO/+yD0Jnu3Iq1g+zYoRpx4tjWz4s=; h=Date:From:To:Subject:In-Reply-To:References:From; b=KllixaXYEwIYNkYsjK9nvrjKevR1A/hzESB9OUF3ZYXsOboFOFMMWvUVilE5MKNNV r2zNCk11LmG7JRlQeyY92i1aFToRxehfIxV5KDomJR4HF3XwlT999iJ/osQUoni/cs Z7Sj8gDQJ091dC2HIVerD3cARtVw5RagmRyrPgeAwfbXcSVMZe4RE3tH+YxKVVD+ir 8I1iwFNN5TFqzgXg9hz11INdhjWLf1y6AY1syAXVEdhOBy3M3xp26x20x6BllWz0WJ J39u98iTSFn2Nntd9rdI2167q+J3Fyajvft+RUdCbjFuY1Sa+9WiFIW2wcy/Z+YkS4 Oua/zIqlI22ww== X-Nifty-SrcIP: [110.4.221.123] Date: Mon, 15 Nov 2021 23:50:21 +0900 From: Takashi Yano To: cygwin-developers@cygwin.com Subject: Re: cygwin 3.3.x: another problem that may be related to pipes Message-Id: <20211115235021.0f0f64b1b0e2a7bd6d16be80@nifty.ne.jp> In-Reply-To: References: <115203324.649908.1636923059546.ref@mail.yahoo.com> <115203324.649908.1636923059546@mail.yahoo.com> <20211115171811.844dce9cce2b4d13262d64f2@nifty.ne.jp> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.30; i686-pc-mingw32) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-5.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2021 14:50:34 -0000 On Mon, 15 Nov 2021 14:01:12 +0100 Corinna Vinschen wrote: > [Redirected to cygwin-developers] > > On Nov 15 17:18, Takashi Yano via Cygwin wrote: > > On Sun, 14 Nov 2021 20:50:59 +0000 (UTC) > > bf wrote: > > > I've a shell script that processes several hundred xz-compressed text files with a pipeline of the form: > > > > > > find . -depth -type f  ... -print0 | \ > > > xargs -0tI @ xzcat -- @ | \ > > > iconv --byte-subst='�' --unicode-subst='�' --widechar-subst='�' -f MS-ANSI -t UTF-8 -- | \ > > > grep ... \ > > > sed ... | \ > > > sort ... | \ > > > sort ... > > > > > > . Since updating to cygwin 3.3.x (but not on cygwin 3.2.x) the script consistently fails when decompressing a portion of the files, resulting in lost data, and error messages of the form: > > > > > > xzcat: (stdout): Write error: Bad address > > > > Thanks for the report. > > I could reproduce your problem and found the cause. > > > > raw_read()/raw_write() in fhandler_pipe.cc seems to need handling > > of STATUS_PENDING in nonblocking mode. > > > > I also confirmed the following patch fixes the issue. However, I > > am not very sure that this is the right thing. > > > > Corinna, Ken, what do you think? > > I'm puzzled. non-blocking pipes return STATUS_PENDING? What on earth > is that supposed to mean on non-blocking (as opposed to asynchronous) > pipes? The problem is that STATUS_PENDING theoretically requires > to wait for... something, the pipe handle, perhaps, and then to check > io.Status. > > Is it possible that nonblocking pipes need to be synchronous, i. e., > setting the FILE_SYNCHRONOUS_IO_NONALERT would fix that? Yes. Setting FILE_SYNCHRONOUS_IO_NONALERT for the write pipe seems to fix the issue. Proberbly, this needs the modification for raw_write() like the modification I have made for raw_read() in topic/pipe branch. -- Takashi Yano