From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.kundenserver.de (mout.kundenserver.de [217.72.192.75]) by sourceware.org (Postfix) with ESMTPS id 57B0D385AC2E for ; Mon, 15 Nov 2021 13:01:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 57B0D385AC2E Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=cygwin.com Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=cygwin.com Received: from calimero.vinschen.de ([24.134.7.25]) by mrelayeu.kundenserver.de (mreue109 [212.227.15.183]) with ESMTPSA (Nemesis) id 1MAORp-1mxM3P3dyV-00BvVV for ; Mon, 15 Nov 2021 14:01:12 +0100 Received: by calimero.vinschen.de (Postfix, from userid 500) id 74424A80D4C; Mon, 15 Nov 2021 14:01:12 +0100 (CET) Date: Mon, 15 Nov 2021 14:01:12 +0100 From: Corinna Vinschen To: cygwin-developers@cygwin.com Subject: Re: cygwin 3.3.x: another problem that may be related to pipes Message-ID: Reply-To: cygwin-developers@cygwin.com Mail-Followup-To: cygwin-developers@cygwin.com References: <115203324.649908.1636923059546.ref@mail.yahoo.com> <115203324.649908.1636923059546@mail.yahoo.com> <20211115171811.844dce9cce2b4d13262d64f2@nifty.ne.jp> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20211115171811.844dce9cce2b4d13262d64f2@nifty.ne.jp> X-Provags-ID: V03:K1:HjeJ7D7y4ZfZJb5FFYY0Z9V9Rs0CNRRgwS1aIVckfKT8yMgaoEa sU1UeBwpsoXtWc45wS/D6pZSG9avlqklFNM+2VtnXD5Pgj6mJ5I2IqwFZ0xSCtrrD7GQxn0 SC4v+dMDGtRuNd1VfbQr0oGjQM4p3UWk9JWP4xl1K3BDpnnPR24vre9iLrYDbLTMXRS4hxC rCKhtSm2YbpAfsl9YRFWQ== X-UI-Out-Filterresults: notjunk:1;V03:K0:uYR5Zaae8hA=:ItqxtXIWC8p0jTlUso7QmG Cpb153WIj9z8Qi8mgxd0HPb6QSH8h+8VCQTfPJtr2548xT6cXkcyJeuUKU5oIOvUjGbqNRxbf jhyfGZd1/oHAZJEGvLZalGrb6cbE1q1mzhVpx2BIgEvnst80lqeVnolnze7m+xXF3gqFTwlsh jFOIIaVdxKP2SefbfhEEw713ig6FZUawqG2d1u/o5e9ehcfnbHHedLjbY4TVKeZUTY18GgUBu LyGmv1TfrwW83ozEiz0ncPmKcpiDpFrkhSBgH0ftYYek9RPV+U+0AmrtjrXnGhuRuQgJr1YrD AoQTZNDgC+BBTsILwLKwTNU3ApH6c3aheL6kawxs/Lcurw9iU6s5aUQiy4hKD+iJZ+QMfB10C gJ9JBGgZWQHqCSUJNQp8KwA9cJkfGHLPnH7fJRrKantP/9hNrP+PkWgsTR/ylhMnHzC9ecRy7 yl5D3cNobbYPq/ZZGGhvHd1KbZsr/xSjsU5wGb2NImksE1CqN0s45TXM+j1DZm4u1Rz1QsTR1 7QHmHXjy1rtG+7zLaAMDpdffVd+t9h88Zu/XiIXj4KUUj2MKyi4GRlGMMnZwvkCvoiK2tKqe3 rgouTgOe+BQL6CONaDh+Q73RwfoKu266KDM2zbkoKwTm41k5m3xu4k79r9Gb+WURoRcW4sQwH L6xV3ylxNFQc05lH1gjJ349bT8LpvHQIPBPDAU5R4E8P7vzgf0mN3tC+s5SbJxVWQ29UOToyt 7hazGZlkaYtYBq8/ X-Spam-Status: No, score=-99.3 required=5.0 tests=BAYES_00, GOOD_FROM_CORINNA_CYGWIN, KAM_DMARC_NONE, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NEUTRAL, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2021 13:01:19 -0000 [Redirected to cygwin-developers] On Nov 15 17:18, Takashi Yano via Cygwin wrote: > On Sun, 14 Nov 2021 20:50:59 +0000 (UTC) > bf wrote: > > I've a shell script that processes several hundred xz-compressed text files with a pipeline of the form: > > > > find . -depth -type f  ... -print0 | \ > > xargs -0tI @ xzcat -- @ | \ > > iconv --byte-subst='�' --unicode-subst='�' --widechar-subst='�' -f MS-ANSI -t UTF-8 -- | \ > > grep ... \ > > sed ... | \ > > sort ... | \ > > sort ... > > > > . Since updating to cygwin 3.3.x (but not on cygwin 3.2.x) the script consistently fails when decompressing a portion of the files, resulting in lost data, and error messages of the form: > > > > xzcat: (stdout): Write error: Bad address > > Thanks for the report. > I could reproduce your problem and found the cause. > > raw_read()/raw_write() in fhandler_pipe.cc seems to need handling > of STATUS_PENDING in nonblocking mode. > > I also confirmed the following patch fixes the issue. However, I > am not very sure that this is the right thing. > > Corinna, Ken, what do you think? I'm puzzled. non-blocking pipes return STATUS_PENDING? What on earth is that supposed to mean on non-blocking (as opposed to asynchronous) pipes? The problem is that STATUS_PENDING theoretically requires to wait for... something, the pipe handle, perhaps, and then to check io.Status. Is it possible that nonblocking pipes need to be synchronous, i. e., setting the FILE_SYNCHRONOUS_IO_NONALERT would fix that? Thanks, Corinna