From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by sourceware.org (Postfix) with ESMTPS id 703393858C30 for ; Mon, 15 Jan 2024 13:55:12 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 703393858C30 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linaro.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 703393858C30 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::62b ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1705326919; cv=none; b=mfSoXHtf9/1dbc5bLPSfES1fGb+p9viCsFpTKIcAFNBxHiPrgTfWEBNVPe+K5m+xYZq5NZaNHrt/W/Sn6efHfO7ilgE97lM7No17QL5xmMN7gABZFNM4oq4PIa13vMZCG7ilMnnD6dl/LT5BpA3qAUbCgYgYTyijyMvYQdeg+Aw= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1705326919; c=relaxed/simple; bh=xAE9+sUCzx3nOFl/b9ZCyOa6Ibgib459hvth8qGUZmY=; h=DKIM-Signature:Message-ID:Date:MIME-Version:Subject:To:From; b=v9v82On0vczX9wXijnp7NHCNj0JLQjZFXRmzzVqr9H19sXTI5Rw+gl7jReJmwHBK3L9WvI4/RImO44ytIQGIDWzrXngtY441VIrOIDubVnWpMjvxRhSyNp7HQA7fKffSBOVqDsuGEz676s0e32guWgdSAirm4DDCWFJD/wnHHxA= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1d409bcb0e7so45387485ad.1 for ; Mon, 15 Jan 2024 05:55:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1705326911; x=1705931711; darn=sourceware.org; h=content-transfer-encoding:in-reply-to:organization:from:references :cc:to:content-language:subject:user-agent:mime-version:date :message-id:from:to:cc:subject:date:message-id:reply-to; bh=2KFcPV7PoxoUdfV29AVpBoNXVzKLhiNl2DX8t0nocwY=; b=PNxvhhE6UAZVLq4KNJBsAiJtrFFMXzF31JBiLcZuNP5e6ygjS3xdDGLhzyKiblTeMn XikhnOFBfunYZ0nC+eXKFKdfmtQaKuSZr74BYGtRptV2pJBAV2yuvHaXJMpZZUF7owgp 0dYDypZ4w0zP4XJxmyUviExZXBb8WwYSNIThGZbkFvG9c5k01pbGsuqvIgKfWcj7394W DAMfd+a0NY2b8nrWgpypEq5Lr0hecR/Prs5wxWV1VSRLhECiWJMJhT5fvG/wY3q5NaYI 3jEQEblhkq9EEuEqlS7VkzeKujtjflry7vXI5a9tLCuIF2AJ9XwhFM5oYArDljl0O64k RtJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705326911; x=1705931711; h=content-transfer-encoding:in-reply-to:organization:from:references :cc:to:content-language:subject:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=2KFcPV7PoxoUdfV29AVpBoNXVzKLhiNl2DX8t0nocwY=; b=muSI+9sgNjLRa1Y3XzUVJkm7kGRpaiEEuc3n/hDQwF2eQ5QpezwqPKuxeQTxItuZUU iO/e4U3jPdchq71YuIDfU3UKtnNTqdHuV51uQNEA48dvWQbqrT43OA+xsZqyxDzxloc3 cC31TxXdZ+cLFg8hU3RTUW6a/HorCMgYdYSOcJ/MOj/yfbFYOn7dXFStiWYjGnxaF1GA nK1uLdDp4FtaVfRMwUGVYfU6cFLd7PMkuqkF132tvag9wt21q0iLtJR5BfzYxkMohLvu 0hsqcFgyUMEA5nBte5o2qeVsr8daI1tWCP/ehc2UCjsJutvKtzrTRBiGIt4cw7eeraax pqRg== X-Gm-Message-State: AOJu0YzggoHf2wu+j/Fr1YS3YCfib1oxtlZIKg5cW4iipGDJiET9eFx6 WeHiVKQ9nYBbpPw4g0TF/VlliHsFsfVT3g== X-Google-Smtp-Source: AGHT+IFGOjQQW68H+aWN+rzlOPDzc6NWHdtb1xQzifAtwvtaJSKaH5BpY5PDq6j3uKiBQC/uLo7Puw== X-Received: by 2002:a17:902:6509:b0:1d4:94f6:56cb with SMTP id b9-20020a170902650900b001d494f656cbmr2547229plk.117.1705326911339; Mon, 15 Jan 2024 05:55:11 -0800 (PST) Received: from ?IPV6:2804:1b3:a7c2:2787:1d58:abc7:72ea:8c9c? ([2804:1b3:a7c2:2787:1d58:abc7:72ea:8c9c]) by smtp.gmail.com with ESMTPSA id m3-20020a1709026bc300b001d4a49b30f4sm7577079plt.61.2024.01.15.05.55.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 15 Jan 2024 05:55:10 -0800 (PST) Message-ID: Date: Mon, 15 Jan 2024 10:55:06 -0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: New TLS usage in libgcc_s.so.1, compatibility impact Content-Language: en-US To: Szabolcs Nagy , Florian Weimer , gcc@gcc.gnu.org, libc-alpha@sourceware.org Cc: Iain Sandoe , aburgess@redhat.com, lttng-dev@lists.lttng.org References: <8734v1ieke.fsf@oldenburg.str.redhat.com> From: Adhemerval Zanella Netto Organization: Linaro In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-13.5 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 15/01/24 09:46, Szabolcs Nagy wrote: > The 01/13/2024 13:49, Florian Weimer wrote: >> This commit >> >> commit 8abddb187b33480d8827f44ec655f45734a1749d >> Author: Andrew Burgess >> Date: Sat Aug 5 14:31:06 2023 +0200 >> >> libgcc: support heap-based trampolines >> >> Add support for heap-based trampolines on x86_64-linux, aarch64-linux, >> and x86_64-darwin. Implement the __builtin_nested_func_ptr_created and >> __builtin_nested_func_ptr_deleted functions for these targets. >> >> Co-Authored-By: Maxim Blinov >> Co-Authored-By: Iain Sandoe >> Co-Authored-By: Francois-Xavier Coudert >> >> added TLS usage to libgcc_s.so.1. The way that libgcc_s is currently >> built, it ends up using a dynamic TLS variant on the Linux targets. >> This means that there is no up-front TLS allocation with glibc (but >> there would be one with musl). >> >> There is still a compatibility impact because glibc assigns a TLS module >> ID upfront. This seems to be what causes the >> ust/libc-wrapper/test_libc-wrapper test in lttng-tools to fail. We end >> up with an infinite regress during process termination because >> libgcc_s.so.1 has been loaded, resulting in a DTV update. When this >> happens, the bottom of the stack looks like this: >> >> #4447 0x00007ffff7f288f0 in free () from /lib64/liblttng-ust-libc-wrapper.so.1 >> #4448 0x00007ffff7fdb142 in free (ptr=) >> at ../include/rtld-malloc.h:50 >> #4449 _dl_update_slotinfo (req_modid=3, new_gen=2) at ../elf/dl-tls.c:822 >> #4450 0x00007ffff7fdb214 in update_get_addr (ti=0x7ffff7f2bfc0, >> gen=) at ../elf/dl-tls.c:916 >> #4451 0x00007ffff7fddccc in __tls_get_addr () >> at ../sysdeps/x86_64/tls_get_addr.S:55 >> #4452 0x00007ffff7f288f0 in free () from /lib64/liblttng-ust-libc-wrapper.so.1 >> #4453 0x00007ffff7fdb142 in free (ptr=) >> at ../include/rtld-malloc.h:50 >> #4454 _dl_update_slotinfo (req_modid=2, new_gen=2) at ../elf/dl-tls.c:822 >> #4455 0x00007ffff7fdb214 in update_get_addr (ti=0x7ffff7f39fa0, >> gen=) at ../elf/dl-tls.c:916 >> #4456 0x00007ffff7fddccc in __tls_get_addr () >> at ../sysdeps/x86_64/tls_get_addr.S:55 >> #4457 0x00007ffff7f36113 in lttng_ust_cancelstate_disable_push () >> from /lib64/liblttng-ust-common.so.1 >> #4458 0x00007ffff7f4c2e8 in ust_lock_nocheck () from /lib64/liblttng-ust.so.1 >> #4459 0x00007ffff7f5175a in lttng_ust_cleanup () from /lib64/liblttng-ust.so.1 >> #4460 0x00007ffff7fca0f2 in _dl_call_fini ( >> closure_map=closure_map@entry=0x7ffff7fbe000) at dl-call_fini.c:43 >> #4461 0x00007ffff7fce06e in _dl_fini () at dl-fini.c:114 >> #4462 0x00007ffff7d82fe6 in __run_exit_handlers () from /lib64/libc.so.6 >> >> Cc:ing for awareness. >> >> The issue also requires a recent glibc with changes to DTV management: >> commit d2123d68275acc0f061e73d5f86ca504e0d5a344 ("elf: Fix slow tls >> access after dlopen [BZ #19924]"). If I understand things correctly, >> before this glibc change, we didn't deallocate the old DTV, so there was >> no call to the free function. > > with 19924 fixed, after a dlopen or dlclose every thread updates > its dtv on the next dynamic tls access. > > before that, dtv was only updated up to the generation of the > module being accessed for a particular tls access. > > so hitting the free in the dtv update path is now more likely > but the free is not new, it was there before. > > also note that this is unlikely to happen on aarch64 since > tlsdesc only does dynamic tls access after a 512byte static > tls reservation runs out. > >> >> On the glibc side, we should recommend that intercepting mallocs and its >> dependencies use initial-exec TLS because that kind of TLS does not use >> malloc. If intercepting mallocs using dynamic TLS work at all, that's >> totally by accident, and was in the past helped by glibc bug 19924. (I > > right. > >> don't think there is anything special about libgcc_s.so.1 that triggers >> the test failure above, it is just an object with dynamic TLS that is >> implicitly loaded via dlopen at the right stage of the test.) In this >> particular case, we can also paper over the test failure in glibc by not >> call free at all because the argument is a null pointer: >> >> diff --git a/elf/dl-tls.c b/elf/dl-tls.c >> index 7b3dd9ab60..14c71cbd06 100644 >> --- a/elf/dl-tls.c >> +++ b/elf/dl-tls.c >> @@ -819,7 +819,8 @@ _dl_update_slotinfo (unsigned long int req_modid, size_t new_gen) >> dtv entry free it. Note: this is not AS-safe. */ >> /* XXX Ideally we will at some point create a memory >> pool. */ >> - free (dtv[modid].pointer.to_free); >> + if (dtv[modid].pointer.to_free != NULL) >> + free (dtv[modid].pointer.to_free); >> dtv[modid].pointer.val = TLS_DTV_UNALLOCATED; >> dtv[modid].pointer.to_free = NULL; > > can be done, but !=NULL is more likely since we do modid reuse > after dlclose. > > there is also a realloc in dtv resizing which happens when more > than 16 modules with tls are loaded after thread creation > (DTV_SURPLUS). > > i'm not sure if it's worth supporting malloc interposers that > only work sometimes. > Maybe one option would to try reinstate the async-signal-safe TLS code to avoid malloc/free in dynamic TLS altogether. We revert it on 2.14 release cause it broke ASAN/LSAN [1], but I think we might try to reinstate on 2.40 and work with sanitizer project to get this sort out. [1] https://sourceware.org/pipermail/libc-alpha/2014-January/047931.html