From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from conssluserg-05.nifty.com (conssluserg-05.nifty.com [210.131.2.90]) by sourceware.org (Postfix) with ESMTPS id 1FD973858403 for ; Mon, 15 Nov 2021 08:18:32 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 1FD973858403 Received: from Express5800-S70 (z221123.dynamic.ppp.asahi-net.or.jp [110.4.221.123]) (authenticated) by conssluserg-05.nifty.com with ESMTP id 1AF8IBlC020841; Mon, 15 Nov 2021 17:18:11 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conssluserg-05.nifty.com 1AF8IBlC020841 X-Nifty-SrcIP: [110.4.221.123] Date: Mon, 15 Nov 2021 17:18:11 +0900 From: Takashi Yano To: cygwin@cygwin.com Cc: bf Subject: Re: cygwin 3.3.x: another problem that may be related to pipes Message-Id: <20211115171811.844dce9cce2b4d13262d64f2@nifty.ne.jp> In-Reply-To: <115203324.649908.1636923059546@mail.yahoo.com> References: <115203324.649908.1636923059546.ref@mail.yahoo.com> <115203324.649908.1636923059546@mail.yahoo.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.30; i686-pc-mingw32) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-11.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2021 08:18:36 -0000 On Sun, 14 Nov 2021 20:50:59 +0000 (UTC) bf wrote: > I've a shell script that processes several hundred xz-compressed text files with a pipeline of the form: > > find . -depth -type f  ... -print0 | \ > xargs -0tI @ xzcat -- @ | \ > iconv --byte-subst='�' --unicode-subst='�' --widechar-subst='�' -f MS-ANSI -t UTF-8 -- | \ > grep ... \ > sed ... | \ > sort ... | \ > sort ... > > . Since updating to cygwin 3.3.x (but not on cygwin 3.2.x) the script consistently fails when decompressing a portion of the files, resulting in lost data, and error messages of the form: > > xzcat: (stdout): Write error: Bad address Thanks for the report. I could reproduce your problem and found the cause. raw_read()/raw_write() in fhandler_pipe.cc seems to need handling of STATUS_PENDING in nonblocking mode. I also confirmed the following patch fixes the issue. However, I am not very sure that this is the right thing. Corinna, Ken, what do you think? >From 2261c7d7df0173d2fb6755f6b9ccccf02b27015e Mon Sep 17 00:00:00 2001 From: Takashi Yano Date: Mon, 15 Nov 2021 12:30:53 +0900 Subject: [PATCH] Cygwin: pipe: Call CancelIo() if I/O is pending in nonblocking mode. - Currently, handling for STATUS_PENDING in raw_read()/raw_write() in nonblocking mode is missing. This patch fixes the issue. Addresses: https://cygwin.com/pipermail/cygwin/2021-November/249910.html --- winsup/cygwin/fhandler_pipe.cc | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/winsup/cygwin/fhandler_pipe.cc b/winsup/cygwin/fhandler_pipe.cc index 1ebf4de10..0de01befb 100644 --- a/winsup/cygwin/fhandler_pipe.cc +++ b/winsup/cygwin/fhandler_pipe.cc @@ -355,6 +355,9 @@ fhandler_pipe::raw_read (void *ptr, size_t& len) else status = io.Status; } + else if (status == STATUS_PENDING) + /* CancelIo() if STATUS_PENDING in nonblocking mode. */ + CancelIo (get_handle ()); if (isclosed ()) /* A signal handler might have closed the fd. */ { if (waitret == WAIT_OBJECT_0) @@ -538,6 +541,9 @@ fhandler_pipe_fifo::raw_write (const void *ptr, size_t len) else status = io.Status; } + else if (status == STATUS_PENDING) + /* CancelIo() if STATUS_PENDING in nonblocking mode. */ + CancelIo (get_handle ()); if (isclosed ()) /* A signal handler might have closed the fd. */ { if (waitret == WAIT_OBJECT_0) -- 2.33.0 -- Takashi Yano