From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt1-x844.google.com (mail-qt1-x844.google.com [IPv6:2607:f8b0:4864:20::844]) by sourceware.org (Postfix) with ESMTPS id B508E383E823 for ; Fri, 22 May 2020 16:36:26 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org B508E383E823 Received: by mail-qt1-x844.google.com with SMTP id p12so8718903qtn.13 for ; Fri, 22 May 2020 09:36:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:autocrypt:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=5tk5LTset/Q++U7sG9sdoLR1SbIUVuAZVy+hRrRGIO0=; b=qsrD4A3D8x3LPvipg+tMSS6TOxOJ1DQCHRyaZccB+WRyw68fWiEMn4+elF1vRlbena iEEqQBc093zV7HrMea71sttg95fGoHOZsvv8sCm0fJkJMcQixDY/dQBF7OoNV63ZU9Qm pRj8isC34hb7WqmXC+ydXqqXoAN0PPJm4qWA9n5cdxcBJJkJL3mLIPhA7UYK6n4PzORU WjPe3MwwcKTmh4eme+fXavT3mESUe421uZ7TOcG6r29dRrBLOk9G1u1ChSxuuFR9X3Nk xL+IVyoD1Bwj3aRiznjIlDxck9/fslZXJ7YV2vqDwkQg4cYY+RvavrbmkWYicpUo25H6 +LQg== X-Gm-Message-State: AOAM531L5oPc+auyha4ILXkIV6umMQq+j8Tkd2fz9sG3APZWJ+q1ZnHX F1a0Q9ekLQ+RbPoxr2z5MbHZ934wGZs= X-Google-Smtp-Source: ABdhPJzvy4aGnA0t9UYaoFN5ewVrLMb+VX3tzjPKR6BrYXWblvxvvPVwu8ogpKOSLkHglW3v0y1MGg== X-Received: by 2002:ac8:302f:: with SMTP id f44mr16323623qte.121.1590165386002; Fri, 22 May 2020 09:36:26 -0700 (PDT) Received: from [192.168.1.4] ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id n20sm7981137qtk.89.2020.05.22.09.36.24 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 22 May 2020 09:36:25 -0700 (PDT) Subject: Re: [PATCH 2/2] manual: Document __libc_single_threaded To: libc-alpha@sourceware.org References: <2c218c9ed9586ed5491f6fa08045d1e883b126c3.1589998207.git.fweimer@redhat.com> <724ecd59-d6e4-9f52-f425-8a4ff795114f@linaro.org> <20200521130927.GF14130@arm.com> <39687760-f8b1-4209-8798-ed8a16b87b57@linaro.org> <20200521133046.GH14130@arm.com> <87k115768f.fsf@oldenburg2.str.redhat.com> <20200522100146.GA29518@arm.com> <877dx45low.fsf@oldenburg2.str.redhat.com> <20200522105458.GB29518@arm.com> <20200522150720.GR1079@brightrain.aerifal.cx> <20200522161413.GU1079@brightrain.aerifal.cx> From: Adhemerval Zanella Autocrypt: addr=adhemerval.zanella@linaro.org; prefer-encrypt=mutual; keydata= mQINBFcVGkoBEADiQU2x/cBBmAVf5C2d1xgz6zCnlCefbqaflUBw4hB/bEME40QsrVzWZ5Nq 8kxkEczZzAOKkkvv4pRVLlLn/zDtFXhlcvQRJ3yFMGqzBjofucOrmdYkOGo0uCaoJKPT186L NWp53SACXguFJpnw4ODI64ziInzXQs/rUJqrFoVIlrPDmNv/LUv1OVPKz20ETjgfpg8MNwG6 iMizMefCl+RbtXbIEZ3TE/IaDT/jcOirjv96lBKrc/pAL0h/O71Kwbbp43fimW80GhjiaN2y WGByepnkAVP7FyNarhdDpJhoDmUk9yfwNuIuESaCQtfd3vgKKuo6grcKZ8bHy7IXX1XJj2X/ BgRVhVgMHAnDPFIkXtP+SiarkUaLjGzCz7XkUn4XAGDskBNfbizFqYUQCaL2FdbW3DeZqNIa nSzKAZK7Dm9+0VVSRZXP89w71Y7JUV56xL/PlOE+YKKFdEw+gQjQi0e+DZILAtFjJLoCrkEX w4LluMhYX/X8XP6/C3xW0yOZhvHYyn72sV4yJ1uyc/qz3OY32CRy+bwPzAMAkhdwcORA3JPb kPTlimhQqVgvca8m+MQ/JFZ6D+K7QPyvEv7bQ7M+IzFmTkOCwCJ3xqOD6GjX3aphk8Sr0dq3 4Awlf5xFDAG8dn8Uuutb7naGBd/fEv6t8dfkNyzj6yvc4jpVxwARAQABtElBZGhlbWVydmFs IFphbmVsbGEgTmV0dG8gKExpbmFybyBWUE4gS2V5KSA8YWRoZW1lcnZhbC56YW5lbGxhQGxp bmFyby5vcmc+iQI3BBMBCAAhBQJXFRpKAhsDBQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAAAoJ EKqx7BSnlIjv0e8P/1YOYoNkvJ+AJcNUaM5a2SA9oAKjSJ/M/EN4Id5Ow41ZJS4lUA0apSXW NjQg3VeVc2RiHab2LIB4MxdJhaWTuzfLkYnBeoy4u6njYcaoSwf3g9dSsvsl3mhtuzm6aXFH /Qsauav77enJh99tI4T+58rp0EuLhDsQbnBic/ukYNv7sQV8dy9KxA54yLnYUFqH6pfH8Lly sTVAMyi5Fg5O5/hVV+Z0Kpr+ZocC1YFJkTsNLAW5EIYSP9ftniqaVsim7MNmodv/zqK0IyDB GLLH1kjhvb5+6ySGlWbMTomt/or/uvMgulz0bRS+LUyOmlfXDdT+t38VPKBBVwFMarNuREU2 69M3a3jdTfScboDd2ck1u7l+QbaGoHZQ8ZNUrzgObltjohiIsazqkgYDQzXIMrD9H19E+8fw kCNUlXxjEgH/Kg8DlpoYJXSJCX0fjMWfXywL6ZXc2xyG/hbl5hvsLNmqDpLpc1CfKcA0BkK+ k8R57fr91mTCppSwwKJYO9T+8J+o4ho/CJnK/jBy1pWKMYJPvvrpdBCWq3MfzVpXYdahRKHI ypk8m4QlRlbOXWJ3TDd/SKNfSSrWgwRSg7XCjSlR7PNzNFXTULLB34sZhjrN6Q8NQZsZnMNs TX8nlGOVrKolnQPjKCLwCyu8PhllU8OwbSMKskcD1PSkG6h3r0AquQINBFcVGkoBEACgAdbR Ck+fsfOVwT8zowMiL3l9a2DP3Eeak23ifdZG+8Avb/SImpv0UMSbRfnw/N81IWwlbjkjbGTu oT37iZHLRwYUFmA8fZX0wNDNKQUUTjN6XalJmvhdz9l71H3WnE0wneEM5ahu5V1L1utUWTyh VUwzX1lwJeV3vyrNgI1kYOaeuNVvq7npNR6t6XxEpqPsNc6O77I12XELic2+36YibyqlTJIQ V1SZEbIy26AbC2zH9WqaKyGyQnr/IPbTJ2Lv0dM3RaXoVf+CeK7gB2B+w1hZummD21c1Laua +VIMPCUQ+EM8W9EtX+0iJXxI+wsztLT6vltQcm+5Q7tY+HFUucizJkAOAz98YFucwKefbkTp eKvCfCwiM1bGatZEFFKIlvJ2QNMQNiUrqJBlW9nZp/k7pbG3oStOjvawD9ZbP9e0fnlWJIsj 6c7pX354Yi7kxIk/6gREidHLLqEb/otuwt1aoMPg97iUgDV5mlNef77lWE8vxmlY0FBWIXuZ yv0XYxf1WF6dRizwFFbxvUZzIJp3spAao7jLsQj1DbD2s5+S1BW09A0mI/1DjB6EhNN+4bDB SJCOv/ReK3tFJXuj/HbyDrOdoMt8aIFbe7YFLEExHpSk+HgN05Lg5TyTro8oW7TSMTk+8a5M kzaH4UGXTTBDP/g5cfL3RFPl79ubXwARAQABiQIfBBgBCAAJBQJXFRpKAhsMAAoJEKqx7BSn lIjvI/8P/jg0jl4Tbvg3B5kT6PxJOXHYu9OoyaHLcay6Cd+ZrOd1VQQCbOcgLFbf4Yr+rE9l mYsY67AUgq2QKmVVbn9pjvGsEaz8UmfDnz5epUhDxC6yRRvY4hreMXZhPZ1pbMa6A0a/WOSt AgFj5V6Z4dXGTM/lNManr0HjXxbUYv2WfbNt3/07Db9T+GZkpUotC6iknsTA4rJi6u2ls0W9 1UIvW4o01vb4nZRCj4rni0g6eWoQCGoVDk/xFfy7ZliR5B+3Z3EWRJcQskip/QAHjbLa3pml xAZ484fVxgeESOoaeC9TiBIp0NfH8akWOI0HpBCiBD5xaCTvR7ujUWMvhsX2n881r/hNlR9g fcE6q00qHSPAEgGr1bnFv74/1vbKtjeXLCcRKk3Ulw0bY1OoDxWQr86T2fZGJ/HIZuVVBf3+ gaYJF92GXFynHnea14nFFuFgOni0Mi1zDxYH/8yGGBXvo14KWd8JOW0NJPaCDFJkdS5hu0VY 7vJwKcyHJGxsCLU+Et0mryX8qZwqibJIzu7kUJQdQDljbRPDFd/xmGUFCQiQAncSilYOcxNU EMVCXPAQTteqkvA+gNqSaK1NM9tY0eQ4iJpo+aoX8HAcn4sZzt2pfUB9vQMTBJ2d4+m/qO6+ cFTAceXmIoFsN8+gFN3i8Is3u12u8xGudcBPvpoy4OoG Message-ID: <7e092a19-0b9f-172c-f019-18424512337b@linaro.org> Date: Fri, 22 May 2020 13:36:22 -0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <20200522161413.GU1079@brightrain.aerifal.cx> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-11.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 May 2020 16:36:28 -0000 On 22/05/2020 13:14, Rich Felker wrote: > On Fri, May 22, 2020 at 11:07:20AM -0400, Rich Felker wrote: >> On Fri, May 22, 2020 at 11:54:58AM +0100, Szabolcs Nagy wrote: >>> The 05/22/2020 12:05, Florian Weimer wrote: >>>> * Szabolcs Nagy: >>>> >>>>> The 05/21/2020 15:44, Florian Weimer wrote: >>>>>> * Szabolcs Nagy: >>>>>>> what's wrong with pthread_join updating it? >>>>>> >>>>>> It's tricky do it correctly if there are two remaining threads, one of >>>>>> them the one being joined, the other one a detached thread. A >>>>>> straightforward implementation merely looking at __nptl_nthreads before >>>>>> returning from pthread_join would not perform the required >>>>>> synchronization on the detached thread exit. >>>>> >>>>> i'm trying to understand this, but don't see >>>>> what's wrong if the last thread is detached. >>>> >>>> Sorry, I meant three reamining threads in total, i.e., two more threads >>>> in addition to the one thread that keeps going after the other two >>>> exited, and may use __libc_single_threaded in the future. >>>> >>>> Clearer now? >>> >>> hm so a detached thread is concurrently exiting with >>> a pthread_join which sees a decremented __nptl_nthreads >>> but the detached thread has not actually exited yet. >> >> In principle this is no big deal as long as the exiting thread cannot >> make any further actions where its existence causes an observable >> effect on users of __libc_single_threaded. (For this purpose, I think >> you actually need to define what uses are valid, though; see setxid >> remarks below.) If it makes a problem for pthread_join that's an >> implementation detail that should be fixable. The bigger issue is >> memory synchronization. >> >>> i think glibc can issue a memory barrier syscall before >>> decrementing __nptl_nthreads in a detached thread, this >>> means if pthread_join observes __nptl_nthreads==1 >>> then user memory accesses in the detached thread are >>> synchronized with non-atomic memory accesses after >>> pthread_join returns. (i.e. __nptl_nthreads==1 should >>> mean at all times that as far as user code is concerned >>> the process is single threaded even if some detached >>> thread is still hanging around) >> >> This still has consequences for setxid safety which is why musl now >> fully synchronizes the existing threads list. But if you're not using >> the thread count for that, it's not an issue. Indeed I think >> SYS_membarrier is a solution here, but if it's not supported or >> blocked by seccomp then __libc_single_threaded must not be made true >> again at this time. > > Uhg, SYS_membarrier is *not* a solution here. The problem is far > worse, because the user of __libc_single_threaded potentially lacks > *compiler barriers* too. > > Consider something like: > > if (!__libc_single_threaded) { lock(); need_unlock=1; } > x = *p; > if (need_unlock) unlock(); > /* ... */ > if (!__libc_single_threaded) { lock(); need_unlock=1; } > x = *p; > if (need_unlock) unlock(); > > Here, in the case where __libc_single_threaded is true the second time > around, there is no (memory or compiler) acquire barrier between the > first access to *p and the second. Thus the compiler can (and actually > does! I don't have a minimal PoC but musl actually just hit a bug very > close to this) omit the second load from memory, and uses the cached > value, which may be incorrect because the exiting thread modified it. Does it help to enforce a relaxed atomic MO on __libc_single_threaded access in this example? > > This could potentially be avoided with complex contracts about > barriers needed to use __libc_single_threaded, but it seems highly > error-prone. > > Note that the issue becomes much easier to hit with a sort of "pretest > not under lock, re-check with lock" idiom of the form: > > x = *p; > if (predicate(x)) { > if (!__libc_single_threaded) { lock(); need_unlock=1; } > x = *p; > /* ... */ > if (need_unlock) unlock(); > } > > Unlike the above, this one does not depend on the release barrier in > unlock() not also being a compiler acquire barrier. > > Rich >