From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from omta002.cacentral1.a.cloudfilter.net (omta002.cacentral1.a.cloudfilter.net [3.97.99.33]) by sourceware.org (Postfix) with ESMTPS id F305A3858D28 for ; Mon, 28 Aug 2023 18:20:44 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org F305A3858D28 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=Shaw.ca Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=shaw.ca Received: from shw-obgw-4003a.ext.cloudfilter.net ([10.228.9.183]) by cmsmtp with ESMTP id aevrqBmci6Nwhagqyq1iru; Mon, 28 Aug 2023 18:20:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=shaw.ca; s=s20180605; t=1693246844; bh=a/TWds3Z3cDEFMg92vhGQ2LW+eU1wnYGDTKuiCoOjCA=; h=Date:Reply-To:Subject:To:References:From:Cc:In-Reply-To; b=b2HL3WKQIPsChsTDtuc9CsfmPDL5RxBFMJSVKR995epGao7/EPZL0qdRxMKPlo6Xz xksYVWNEsPDh+lzJ3kMSBBJm5OlLH6BPk19mAyiNffbmnUCHMTJJxZ9Y48+kxkU/cs Gu5f0U0LMbImFdwuzquMkeB1Wt3eWnEnvcvaMXzH0J8pWuqFfAjTgaiochJdlQhj47 XEX6M7QpuIC/bk/pDguOAY141np3mX7OyTRFq+mSndK/FXwqyAuXErS83mRAfctyds LFy26Jh4mMSAAWN/mQ1cbGKqpWWOl7vkJUb0kC5kswIY3NBIu2HbNdeQSrgHhUM0/6 kuAIq9SVxJ23w== Received: from [10.0.0.5] ([184.64.102.149]) by cmsmtp with ESMTP id agqxqG4jOcyvuagqyqphzB; Mon, 28 Aug 2023 18:20:44 +0000 X-Authority-Analysis: v=2.4 cv=VbHkgXl9 c=1 sm=1 tr=0 ts=64ece57c a=DxHlV3/gbUaP7LOF0QAmaA==:117 a=DxHlV3/gbUaP7LOF0QAmaA==:17 a=IkcTkHD0fZMA:10 a=w_pzkKWiAAAA:8 a=30KeihfUasEDSKDPGhkA:9 a=QEXdDO2ut3YA:10 a=sRI3_1zDfAgwuvI8zelB:22 Message-ID: <50471900-708d-2e92-bd90-aaacfb64873a@Shaw.ca> Date: Mon, 28 Aug 2023 12:20:43 -0600 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.14.0 Reply-To: cygwin@cygwin.com Subject: Re: gawk core dumped on too many input values Content-Language: en-CA To: cygwin@cygwin.com References: <0b55205e-fc7c-98cf-c186-783e5c0655b0@comcast.net> From: Brian Inglis Organization: Inglis Cc: Joshuah Hurst In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-CMAE-Envelope: MS4xfCl9dmoJh1JDz42VeQWHhEweVGUfybE4WcWy8pw4Zr477JcvtfHVGU4yd3tQOuaK2c4a92Gkw/JzBqT4Va86/AJyKowWpMrdx9G8zXmYVtzVijFu+Ky3 ZbWEti2mdAgwvzj07F4MIxZb6rjRb5ql3Hv7d+db9TtDVz7GbyE0NdDQwHnPXZy1LDf6C4/9rlgIlTrz5yenwhtdEeQzC79IzkY= X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 2023-08-28 05:47, Joshuah Hurst via Cygwin wrote: > On Mon, Aug 28, 2023 at 1:08 AM Jeremy Hetzler via Cygwin > wrote: >> >> On Sun, Aug 27, 2023 at 2:25 PM Ed Morton via Cygwin >> wrote: >>> >>> This (original email below) turned out to be a general cygwin issue, not >>> a gawk issue: >>> >>> $ LC_ALL=C sed 's/x/y/' $(seq 1000000) >>> Segmentation fault (core dumped) >>> >>> $ LC_ALL=C grep 'foo' $(seq 1000000) >>> Segmentation fault (core dumped) >>> >> >> Seems that all commands linked with cygwin1.dll will fault if you pass them >> a long enough arglist. >> >> For me, /bin/true faults on {1..258231} but not {1..258230}. >> >>> $ /bin/true {1..258230} >>> >> >> >>> $ /bin/true {1..258231} >>> Segmentation fault (core dumped) >> >> >> strace, which is not linked with cygwin1.dll, exits cleanly. >> >>> $ /bin/strace {1..300000} >>> -bash: /bin/strace: Argument list too long >> >> >> See this page [1] on maximum argument lengths. >> >> It would be nice to document this limit, whatever it is. > > Is this limit? > > $ getconf -a | grep -E 'ARG_MAX' > _POSIX_ARG_MAX 4096 > ARG_MAX 32000 On my system, /bin/true and /bin/echo get to between 250K and 256K before seg faulting, requiring between digits + separator + ptr (10+8)*(250-256)KB == 3750-3840KB; current Cygwin and `bash` `:` and `echo` internal commands get to between 128M and 256M before `bash` fork fails, and Windows gets unhappy with memory full, paging high, and page thrashing on both paging devices: calmed eventually with a few interrupts then `exec bash`; requiring between digits + separator + ptr (10+8)*(128-256)MB == 2.25-4.5GB; This seems to be extremely conservative: $ xargs -r --show-limits <<< ' ' Your environment variables take up 9246 bytes POSIX upper limit on argument length (this system): 20706 POSIX smallest allowable upper limit on argument length (all systems): 4096 Maximum length of command we could actually use: 11460 Size of command buffer we are actually using: 20706 Maximum parallelism (--max-procs must be no greater): 2147483647 $ seq $((256*1024)) | xargs | wc -lwcL 102 262144 1723903 18094 Once a patch is in a test version of Cygwin, I can rebuild and release a test version of findutils, which includes xargs, with updated results in the test release announcement. -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut -- Antoine de Saint-Exupéry