From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id B795B385840F for ; Tue, 3 May 2022 07:22:54 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org B795B385840F Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-183-SUJSE_PkMvq3VdBiJH03gw-1; Tue, 03 May 2022 03:22:50 -0400 X-MC-Unique: SUJSE_PkMvq3VdBiJH03gw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E87A185A5BC; Tue, 3 May 2022 07:22:49 +0000 (UTC) Received: from oldenburg.str.redhat.com (unknown [10.39.192.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2F3BA2166B4C; Tue, 3 May 2022 07:22:41 +0000 (UTC) From: Florian Weimer To: Jonathon Anderson Cc: Carlos O'Donell , Ben Woodard , Adhemerval Zanella , "Legendre, Matthew P." , libc-alpha@sourceware.org, John Mellor-Crummey Subject: Re: LD_AUDIT: Not enough space in static TLS block References: <87fsmiiw3x.fsf@oldenburg.str.redhat.com> Date: Tue, 03 May 2022 09:22:40 +0200 In-Reply-To: <87fsmiiw3x.fsf@oldenburg.str.redhat.com> (Florian Weimer's message of "Tue, 12 Apr 2022 09:44:50 +0200") Message-ID: <875ymnxepr.fsf@oldenburg.str.redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-5.1 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_LOW, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 May 2022 07:22:56 -0000 * Florian Weimer: > * Jonathon Anderson: > >> Hello all, >> >> We (the HPCToolkit team) have encountered another critical LD_AUDIT >> bug. When LD_AUDIT is specified, the allocation of the static TLS >> block does not account for the TLS requirements of executable >> dependencies or of the auditors themselves. If: >> =C2=A0- an executable accesses a thread-local variable in a linked libra= ry >> with sufficiently large TLS requirements, or >> =C2=A0- an auditor itself uses sufficiently large TLS and optimizes acce= ss >> with `-ftls-model=3Dinitial-exec`, >> >> then the process or auditor will fail with the error "cannot allocate >> memory in static TLS block." > > We have a tunable that can be used as a workaround. Your reproducer > passes for me with our 2.28 backport (glibc-2.28-164.el8) if I run it > like this: > > GLIBC_TUNABLES=3Dglibc.rtld.optional_static_tls=3D4000 make > > The best we can do in the short term would be an increase of the default > limit. On 64-bit platforms, defaulting to a dozen or so kilobytes per > thread should not be a problem as far as virtual address space > consumption is concerned. We can also add an additional reservation of > similar size for every auditor that is loaded, to compensate for the > lack of auto-tuning of the TLS allocation size in auditing mode. > > The fundamental issue is that there is always going to be a hard limit > for initial-exec TLS. Initial-exec TLS requires a fixed offset from the > thread pointer, and we cannot relocate TLS variables because they are > ordinary C objects with an observable address. There are some other > things we can try to improve auto-tuning, but in the end, there is > always going to be a fixed-size reserved area dedicated to initial-exec > TLS set up at process startup, and with dlopen, that might not be enough > even without any auditor use. Jonathon, does setting the environment variable work for you? Thanks, Florian