From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by sourceware.org (Postfix) with ESMTPS id BF9FE386F02B for ; Tue, 21 Apr 2020 13:00:23 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org BF9FE386F02B Received: by mail-qk1-x742.google.com with SMTP id o19so14330239qkk.5 for ; Tue, 21 Apr 2020 06:00:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:cc:references:from:autocrypt:subject :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=y7++qLnqTvYI5z+3bcStFs+nsmK3xWuousgjdD6SFAA=; b=AfYKO4jHq9Ost8UDKHMTPoqyGg4TKOJXGua+Mrx/JlrjyxNplfrDflWQjSwlxeEf0F VXQy/M123VjTeXXYIXDS6X5XUjWCsUlMq85m4iPAn1CCTIhyMBkDLph/KZB7CRUHZ68K 7taQ5P3+qmwHk7s7CSQ90x3jVP/OpYRbxG2HULxxPRR9RsBPsPaB2x2+YH5cH4ULeuZ+ WAn2d1fwx9FYezMZlC/O8mifaeYKIGT8tP5eZ+Zkeo5A10RCZ6h4vzYku7C0uhBzhtmo zwCqCTC6GZJWhnwzb21kdAht9KR0tyJ9XHb3nprmso2hHj10W3AEJhRhFWsyo+iI1PSc eKFA== X-Gm-Message-State: AGi0Pua2wlr8MXO3cSzPvg9dmpMrCvZTRJTjZNzMQh3wJsbrUtS4FSGE SeBFOFK3i2KsGDtBzVXiEts278XKp4FFWQ== X-Google-Smtp-Source: APiQypKudoMaac3dS1zt8+siY5gNC18GPUK2CNa4hgrCOasb78OvwewB8f4YOAICT7/zASgmBQRm8w== X-Received: by 2002:a37:9b4f:: with SMTP id d76mr21807020qke.305.1587474019951; Tue, 21 Apr 2020 06:00:19 -0700 (PDT) Received: from [192.168.1.4] ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id e4sm1695069qkn.11.2020.04.21.06.00.18 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 21 Apr 2020 06:00:19 -0700 (PDT) To: Florian Weimer Cc: Adhemerval Zanella via Libc-alpha References: <20200417132209.22065-1-adhemerval.zanella@linaro.org> <20200417132209.22065-3-adhemerval.zanella@linaro.org> <87wo69t936.fsf@mid.deneb.enyo.de> <9ffd3bb5-ab53-b4ac-162b-c2e2d2a9a21f@linaro.org> <87blnlrq3r.fsf@mid.deneb.enyo.de> From: Adhemerval Zanella Autocrypt: addr=adhemerval.zanella@linaro.org; prefer-encrypt=mutual; keydata= xsFNBFcVGkoBEADiQU2x/cBBmAVf5C2d1xgz6zCnlCefbqaflUBw4hB/bEME40QsrVzWZ5Nq 8kxkEczZzAOKkkvv4pRVLlLn/zDtFXhlcvQRJ3yFMGqzBjofucOrmdYkOGo0uCaoJKPT186L NWp53SACXguFJpnw4ODI64ziInzXQs/rUJqrFoVIlrPDmNv/LUv1OVPKz20ETjgfpg8MNwG6 iMizMefCl+RbtXbIEZ3TE/IaDT/jcOirjv96lBKrc/pAL0h/O71Kwbbp43fimW80GhjiaN2y WGByepnkAVP7FyNarhdDpJhoDmUk9yfwNuIuESaCQtfd3vgKKuo6grcKZ8bHy7IXX1XJj2X/ BgRVhVgMHAnDPFIkXtP+SiarkUaLjGzCz7XkUn4XAGDskBNfbizFqYUQCaL2FdbW3DeZqNIa nSzKAZK7Dm9+0VVSRZXP89w71Y7JUV56xL/PlOE+YKKFdEw+gQjQi0e+DZILAtFjJLoCrkEX w4LluMhYX/X8XP6/C3xW0yOZhvHYyn72sV4yJ1uyc/qz3OY32CRy+bwPzAMAkhdwcORA3JPb kPTlimhQqVgvca8m+MQ/JFZ6D+K7QPyvEv7bQ7M+IzFmTkOCwCJ3xqOD6GjX3aphk8Sr0dq3 4Awlf5xFDAG8dn8Uuutb7naGBd/fEv6t8dfkNyzj6yvc4jpVxwARAQABzUlBZGhlbWVydmFs IFphbmVsbGEgTmV0dG8gKExpbmFybyBWUE4gS2V5KSA8YWRoZW1lcnZhbC56YW5lbGxhQGxp bmFyby5vcmc+wsF3BBMBCAAhBQJXFRpKAhsDBQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAAAoJ EKqx7BSnlIjv0e8P/1YOYoNkvJ+AJcNUaM5a2SA9oAKjSJ/M/EN4Id5Ow41ZJS4lUA0apSXW NjQg3VeVc2RiHab2LIB4MxdJhaWTuzfLkYnBeoy4u6njYcaoSwf3g9dSsvsl3mhtuzm6aXFH /Qsauav77enJh99tI4T+58rp0EuLhDsQbnBic/ukYNv7sQV8dy9KxA54yLnYUFqH6pfH8Lly sTVAMyi5Fg5O5/hVV+Z0Kpr+ZocC1YFJkTsNLAW5EIYSP9ftniqaVsim7MNmodv/zqK0IyDB GLLH1kjhvb5+6ySGlWbMTomt/or/uvMgulz0bRS+LUyOmlfXDdT+t38VPKBBVwFMarNuREU2 69M3a3jdTfScboDd2ck1u7l+QbaGoHZQ8ZNUrzgObltjohiIsazqkgYDQzXIMrD9H19E+8fw kCNUlXxjEgH/Kg8DlpoYJXSJCX0fjMWfXywL6ZXc2xyG/hbl5hvsLNmqDpLpc1CfKcA0BkK+ k8R57fr91mTCppSwwKJYO9T+8J+o4ho/CJnK/jBy1pWKMYJPvvrpdBCWq3MfzVpXYdahRKHI ypk8m4QlRlbOXWJ3TDd/SKNfSSrWgwRSg7XCjSlR7PNzNFXTULLB34sZhjrN6Q8NQZsZnMNs TX8nlGOVrKolnQPjKCLwCyu8PhllU8OwbSMKskcD1PSkG6h3r0AqzsFNBFcVGkoBEACgAdbR Ck+fsfOVwT8zowMiL3l9a2DP3Eeak23ifdZG+8Avb/SImpv0UMSbRfnw/N81IWwlbjkjbGTu oT37iZHLRwYUFmA8fZX0wNDNKQUUTjN6XalJmvhdz9l71H3WnE0wneEM5ahu5V1L1utUWTyh VUwzX1lwJeV3vyrNgI1kYOaeuNVvq7npNR6t6XxEpqPsNc6O77I12XELic2+36YibyqlTJIQ V1SZEbIy26AbC2zH9WqaKyGyQnr/IPbTJ2Lv0dM3RaXoVf+CeK7gB2B+w1hZummD21c1Laua +VIMPCUQ+EM8W9EtX+0iJXxI+wsztLT6vltQcm+5Q7tY+HFUucizJkAOAz98YFucwKefbkTp eKvCfCwiM1bGatZEFFKIlvJ2QNMQNiUrqJBlW9nZp/k7pbG3oStOjvawD9ZbP9e0fnlWJIsj 6c7pX354Yi7kxIk/6gREidHLLqEb/otuwt1aoMPg97iUgDV5mlNef77lWE8vxmlY0FBWIXuZ yv0XYxf1WF6dRizwFFbxvUZzIJp3spAao7jLsQj1DbD2s5+S1BW09A0mI/1DjB6EhNN+4bDB SJCOv/ReK3tFJXuj/HbyDrOdoMt8aIFbe7YFLEExHpSk+HgN05Lg5TyTro8oW7TSMTk+8a5M kzaH4UGXTTBDP/g5cfL3RFPl79ubXwARAQABwsFfBBgBCAAJBQJXFRpKAhsMAAoJEKqx7BSn lIjvI/8P/jg0jl4Tbvg3B5kT6PxJOXHYu9OoyaHLcay6Cd+ZrOd1VQQCbOcgLFbf4Yr+rE9l mYsY67AUgq2QKmVVbn9pjvGsEaz8UmfDnz5epUhDxC6yRRvY4hreMXZhPZ1pbMa6A0a/WOSt AgFj5V6Z4dXGTM/lNManr0HjXxbUYv2WfbNt3/07Db9T+GZkpUotC6iknsTA4rJi6u2ls0W9 1UIvW4o01vb4nZRCj4rni0g6eWoQCGoVDk/xFfy7ZliR5B+3Z3EWRJcQskip/QAHjbLa3pml xAZ484fVxgeESOoaeC9TiBIp0NfH8akWOI0HpBCiBD5xaCTvR7ujUWMvhsX2n881r/hNlR9g fcE6q00qHSPAEgGr1bnFv74/1vbKtjeXLCcRKk3Ulw0bY1OoDxWQr86T2fZGJ/HIZuVVBf3+ gaYJF92GXFynHnea14nFFuFgOni0Mi1zDxYH/8yGGBXvo14KWd8JOW0NJPaCDFJkdS5hu0VY 7vJwKcyHJGxsCLU+Et0mryX8qZwqibJIzu7kUJQdQDljbRPDFd/xmGUFCQiQAncSilYOcxNU EMVCXPAQTteqkvA+gNqSaK1NM9tY0eQ4iJpo+aoX8HAcn4sZzt2pfUB9vQMTBJ2d4+m/qO6+ cFTAceXmIoFsN8+gFN3i8Is3u12u8xGudcBPvpoy4OoG Subject: Re: [PATCH 03/10] linux: Add __readdir_unlocked Message-ID: <4a96fd5e-7a06-90c8-c2f2-8e39246d15ce@linaro.org> Date: Tue, 21 Apr 2020 10:00:16 -0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <87blnlrq3r.fsf@mid.deneb.enyo.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-13.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Apr 2020 13:00:25 -0000 On 21/04/2020 09:16, Florian Weimer wrote: > * Adhemerval Zanella: > >> On 21/04/2020 07:41, Florian Weimer wrote: >>> * Adhemerval Zanella via Libc-alpha: >>> >>>> And use it on readdir_r implementation. >>> >>> I think __readdir_unlocked will not really be async-signal-safe if we >>> have to call malloc during readdir, to allocate the long-to-off64_t >>> translation table for the real fix for bug 23960 (when the kernel does >>> not provide 32-bit off64_t values). >>> >>> That's why I think this change does not go in the right direction, sorry. >>> >> >> I am not following, the whole set explicit avoid malloc by using a reserved >> space in the allocated but on opendir. Even on mips64, which requires >> a special getdents64 implementation, avoid memory allocation. > > The fix for bug 23960 needs to introduce some kind of memory > allocation to store the mapping. We can avoid it in most cases, but I > think the readdir interface has too much baggage to make it > async-signal-safe. My proposed solution is to first set readdir use internally off64_t (even for non-LFS) and whence he memory allocation is done on telldir which will try to first allocate an entry on the dynamic array embedded on allocated opendir __dirstream. > >> And it is for both non-LFS and LFS interfaces. The only issue for >> async-signal-safeness is on telldir for non-LFS mode which might require >> to extend the dynamic array internal buffer if the entry could not be >> added in the internal pre-allocated buffer (and this could be mitigated >> if we added a mmap variant of dynamic array). > > telldir cannot allocate because there is no way to return error. The > allocation has to happen in readdir. > > We should be able to optimize out allocations if telldir is never > called (basically, pre-allocate one slot to be used for telldir). But > all this is really tricky. I do wonder if it makes more sense to call > getdents64 directly if one needs async-signal-safety. This is essentially what the patchset does: it uses gendents64 on both LFS and non-LFS interface, the position is already place on the __dirstream and a default dynamic array of off64_t is used to map internal directory offset to long int. The problem is not really telldir, but rather seekdir that does not allow to signal an invalid position. The standard states that any value obtained from telldir is valid. My proposed fix uses a dynamic array with the default pre-defined size to allocated the off64_t mapping and returns -1 if the dynamic array could not be extended. Although it is not what POSIX state, man-pages documents as a possible return error. But even not allowing the pre-allocated buffer to extend over its initial size, we will still need to handle a telldir call with the buffer full. So I see we might have two options: 1. Just abort on telldir once it requires an additional entry in the pre-allocated mapping buffer. 2. Return -1 on failure and handle it as special sentinel value that does not update the directory position. So user might still try to check if seekdir does succeeded by: long int pre = telldir (dirp); seek (dirp, value); if (pre == telldir (dirp) // position not updated, invalid value In any case, this fix is only for *no-LFS* which should be considered a legacy interface a long time ago. The default LFS interface does not suffer with this limitation.