From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-74.mimecast.com (us-smtp-delivery-74.mimecast.com [170.10.133.74]) by sourceware.org (Postfix) with ESMTPS id D34CE3858D3C for ; Thu, 5 May 2022 17:30:55 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org D34CE3858D3C Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-260-5e2znY7SPruud4IuqHDY4w-1; Thu, 05 May 2022 13:30:52 -0400 X-MC-Unique: 5e2znY7SPruud4IuqHDY4w-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E3494296A60A; Thu, 5 May 2022 17:30:51 +0000 (UTC) Received: from oldenburg.str.redhat.com (unknown [10.39.192.40]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1987D2166B17; Thu, 5 May 2022 17:30:38 +0000 (UTC) From: Florian Weimer To: Jonathon Anderson Cc: Carlos O'Donell , Ben Woodard , Adhemerval Zanella , "Legendre, Matthew P." , libc-alpha@sourceware.org, John Mellor-Crummey Subject: Re: LD_AUDIT: Not enough space in static TLS block References: <87fsmiiw3x.fsf@oldenburg.str.redhat.com> <875ymnxepr.fsf@oldenburg.str.redhat.com> Date: Thu, 05 May 2022 19:30:37 +0200 In-Reply-To: <875ymnxepr.fsf@oldenburg.str.redhat.com> (Florian Weimer's message of "Tue, 03 May 2022 09:22:40 +0200") Message-ID: <87k0azlwea.fsf@oldenburg.str.redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-5.3 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 May 2022 17:30:57 -0000 * Florian Weimer: > * Florian Weimer: > >> * Jonathon Anderson: >> >>> Hello all, >>> >>> We (the HPCToolkit team) have encountered another critical LD_AUDIT >>> bug. When LD_AUDIT is specified, the allocation of the static TLS >>> block does not account for the TLS requirements of executable >>> dependencies or of the auditors themselves. If: >>> =C2=A0- an executable accesses a thread-local variable in a linked libr= ary >>> with sufficiently large TLS requirements, or >>> =C2=A0- an auditor itself uses sufficiently large TLS and optimizes acc= ess >>> with `-ftls-model=3Dinitial-exec`, >>> >>> then the process or auditor will fail with the error "cannot allocate >>> memory in static TLS block." >> >> We have a tunable that can be used as a workaround. Your reproducer >> passes for me with our 2.28 backport (glibc-2.28-164.el8) if I run it >> like this: >> >> GLIBC_TUNABLES=3Dglibc.rtld.optional_static_tls=3D4000 make >> >> The best we can do in the short term would be an increase of the default >> limit. On 64-bit platforms, defaulting to a dozen or so kilobytes per >> thread should not be a problem as far as virtual address space >> consumption is concerned. We can also add an additional reservation of >> similar size for every auditor that is loaded, to compensate for the >> lack of auto-tuning of the TLS allocation size in auditing mode. >> >> The fundamental issue is that there is always going to be a hard limit >> for initial-exec TLS. Initial-exec TLS requires a fixed offset from the >> thread pointer, and we cannot relocate TLS variables because they are >> ordinary C objects with an observable address. There are some other >> things we can try to improve auto-tuning, but in the end, there is >> always going to be a fixed-size reserved area dedicated to initial-exec >> TLS set up at process startup, and with dlopen, that might not be enough >> even without any auditor use. > > Jonathon, > > does setting the environment variable work for you? Do you have any additional feedback here? In the meantime, we have updated Fedora rawhide with the bug fix to enable early usage from auditors, and the new RTLD_DI_PHDR dlinfo is included as well. If you could test glibc-2.35.9000-16 or later, that would be great. Thanks, Florian