From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 4E7F23858D20 for ; Sat, 13 Jan 2024 12:49:15 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4E7F23858D20 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 4E7F23858D20 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1705150158; cv=none; b=MrJq4gxLsrJLNEN2iyNds4LeX78ZlWKnLXCSsGD5CAdbwjCJeAGSXk8RWtRs2210stEJCmYj9N/5JGf6m82z9n5FIOdclr9thLc3tF4bxsaFvGjjbgGLvpHzEXZQsTSmjXEw+UgBYxHwjFNs8xi4g21cLcAU9dix6hNN0VFVnhI= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1705150158; c=relaxed/simple; bh=MJ6ixvqW+K4+KTV8AVl3OnMEhp3bh0oCU2gNg7yu2SM=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=sKY/uWe2N9BCAXf3Kk2h91zgtag1x6tEUtpV6MUfcOsDS3BVIdG/TayBe4wzm+RbYphUZzs/1rKfzkFFgZtEWWb1mL0z1yWCkCj3obIfxpi0VO3xvIOo+l8K9NA/j4kylkjURSAg3esL0mv9gOa/TOSBkS4AMoIb7UyZBZf41RM= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705150155; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=1jsLJq8sPP3YODX3p/t+p2J4qvOUMGMMmpOa1Pe/qfQ=; b=AIlDmZoEGunPb2MDhoKjUsEAXsVpMH2iHQ1EIxTdho9vsiN/vEhWom1tg0d0hbjf1FW5dX uM9TW1oXYOPpBwYtSSQilZ3dcxL7KkaU3VpWdlngDAjtVnnOKGzE8MQngqM+QFjWBa1tuw pzPasvoYX7RqSEtsmYClvYw+3/CGQKM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-308-W-dsHOidPMeHYoXYVbNQ_g-1; Sat, 13 Jan 2024 07:49:10 -0500 X-MC-Unique: W-dsHOidPMeHYoXYVbNQ_g-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 274F485A588; Sat, 13 Jan 2024 12:49:10 +0000 (UTC) Received: from oldenburg.str.redhat.com (unknown [10.39.192.161]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E91991C0652C; Sat, 13 Jan 2024 12:49:07 +0000 (UTC) From: Florian Weimer To: gcc@gcc.gnu.org, libc-alpha@sourceware.org Cc: Iain Sandoe , aburgess@redhat.com, lttng-dev@lists.lttng.org, Szabolcs Nagy Subject: New TLS usage in libgcc_s.so.1, compatibility impact Date: Sat, 13 Jan 2024 13:49:05 +0100 Message-ID: <8734v1ieke.fsf@oldenburg.str.redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.3 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: This commit commit 8abddb187b33480d8827f44ec655f45734a1749d Author: Andrew Burgess Date: Sat Aug 5 14:31:06 2023 +0200 libgcc: support heap-based trampolines Add support for heap-based trampolines on x86_64-linux, aarch64-linux, and x86_64-darwin. Implement the __builtin_nested_func_ptr_created and __builtin_nested_func_ptr_deleted functions for these targets. Co-Authored-By: Maxim Blinov Co-Authored-By: Iain Sandoe Co-Authored-By: Francois-Xavier Coudert added TLS usage to libgcc_s.so.1. The way that libgcc_s is currently built, it ends up using a dynamic TLS variant on the Linux targets. This means that there is no up-front TLS allocation with glibc (but there would be one with musl). There is still a compatibility impact because glibc assigns a TLS module ID upfront. This seems to be what causes the ust/libc-wrapper/test_libc-wrapper test in lttng-tools to fail. We end up with an infinite regress during process termination because libgcc_s.so.1 has been loaded, resulting in a DTV update. When this happens, the bottom of the stack looks like this: #4447 0x00007ffff7f288f0 in free () from /lib64/liblttng-ust-libc-wrapper.so.1 #4448 0x00007ffff7fdb142 in free (ptr=) at ../include/rtld-malloc.h:50 #4449 _dl_update_slotinfo (req_modid=3, new_gen=2) at ../elf/dl-tls.c:822 #4450 0x00007ffff7fdb214 in update_get_addr (ti=0x7ffff7f2bfc0, gen=) at ../elf/dl-tls.c:916 #4451 0x00007ffff7fddccc in __tls_get_addr () at ../sysdeps/x86_64/tls_get_addr.S:55 #4452 0x00007ffff7f288f0 in free () from /lib64/liblttng-ust-libc-wrapper.so.1 #4453 0x00007ffff7fdb142 in free (ptr=) at ../include/rtld-malloc.h:50 #4454 _dl_update_slotinfo (req_modid=2, new_gen=2) at ../elf/dl-tls.c:822 #4455 0x00007ffff7fdb214 in update_get_addr (ti=0x7ffff7f39fa0, gen=) at ../elf/dl-tls.c:916 #4456 0x00007ffff7fddccc in __tls_get_addr () at ../sysdeps/x86_64/tls_get_addr.S:55 #4457 0x00007ffff7f36113 in lttng_ust_cancelstate_disable_push () from /lib64/liblttng-ust-common.so.1 #4458 0x00007ffff7f4c2e8 in ust_lock_nocheck () from /lib64/liblttng-ust.so.1 #4459 0x00007ffff7f5175a in lttng_ust_cleanup () from /lib64/liblttng-ust.so.1 #4460 0x00007ffff7fca0f2 in _dl_call_fini ( closure_map=closure_map@entry=0x7ffff7fbe000) at dl-call_fini.c:43 #4461 0x00007ffff7fce06e in _dl_fini () at dl-fini.c:114 #4462 0x00007ffff7d82fe6 in __run_exit_handlers () from /lib64/libc.so.6 Cc:ing for awareness. The issue also requires a recent glibc with changes to DTV management: commit d2123d68275acc0f061e73d5f86ca504e0d5a344 ("elf: Fix slow tls access after dlopen [BZ #19924]"). If I understand things correctly, before this glibc change, we didn't deallocate the old DTV, so there was no call to the free function. On the glibc side, we should recommend that intercepting mallocs and its dependencies use initial-exec TLS because that kind of TLS does not use malloc. If intercepting mallocs using dynamic TLS work at all, that's totally by accident, and was in the past helped by glibc bug 19924. (I don't think there is anything special about libgcc_s.so.1 that triggers the test failure above, it is just an object with dynamic TLS that is implicitly loaded via dlopen at the right stage of the test.) In this particular case, we can also paper over the test failure in glibc by not call free at all because the argument is a null pointer: diff --git a/elf/dl-tls.c b/elf/dl-tls.c index 7b3dd9ab60..14c71cbd06 100644 --- a/elf/dl-tls.c +++ b/elf/dl-tls.c @@ -819,7 +819,8 @@ _dl_update_slotinfo (unsigned long int req_modid, size_t new_gen) dtv entry free it. Note: this is not AS-safe. */ /* XXX Ideally we will at some point create a memory pool. */ - free (dtv[modid].pointer.to_free); + if (dtv[modid].pointer.to_free != NULL) + free (dtv[modid].pointer.to_free); dtv[modid].pointer.val = TLS_DTV_UNALLOCATED; dtv[modid].pointer.to_free = NULL; As the comment hints, we shouldn't be using malloc for TLS memory at all because it is not AS-safe, but that's a long-term change. This change seems rather specific to this particular test case failure because it relies on libgcc_s.so.1 never using TLS before it gets unloaded. Regarding the libgcc_s side, I'm not sure if the TLS usage there should be considered a real problem, although I'm a bit nervous about it. However, the current implementation caches one page of trampolines past the outermost nested function pointer deallocation (otherwise creating one function pointer per thread in a loop would be really expensive). It looks to me that is never freed, so if the thread exits even with proper unwinding (e.g., on glibc with code compiled with -fexceptions), there is a memory leak. Integration with glibc could avoid this issue, and also help with the longjmp problem, and fix setcontext/swapcontext, too. Thanks, Florian