From: Florian Weimer <fweimer@redhat.com>
To: Jonathon Anderson <janderson@rice.edu>
Cc: Carlos O'Donell <carlos@redhat.com>,
Ben Woodard <woodard@redhat.com>,
Adhemerval Zanella <adhemerval.zanella@linaro.org>,
"Legendre, Matthew P." <legendre1@llnl.gov>,
libc-alpha@sourceware.org,
John Mellor-Crummey <johnmc@rice.edu>
Subject: Re: LD_AUDIT: Not enough space in static TLS block
Date: Tue, 03 May 2022 09:22:40 +0200 [thread overview]
Message-ID: <875ymnxepr.fsf@oldenburg.str.redhat.com> (raw)
In-Reply-To: <87fsmiiw3x.fsf@oldenburg.str.redhat.com> (Florian Weimer's message of "Tue, 12 Apr 2022 09:44:50 +0200")
* Florian Weimer:
> * Jonathon Anderson:
>
>> Hello all,
>>
>> We (the HPCToolkit team) have encountered another critical LD_AUDIT
>> bug. When LD_AUDIT is specified, the allocation of the static TLS
>> block does not account for the TLS requirements of executable
>> dependencies or of the auditors themselves. If:
>> - an executable accesses a thread-local variable in a linked library
>> with sufficiently large TLS requirements, or
>> - an auditor itself uses sufficiently large TLS and optimizes access
>> with `-ftls-model=initial-exec`,
>>
>> then the process or auditor will fail with the error "cannot allocate
>> memory in static TLS block."
>
> We have a tunable that can be used as a workaround. Your reproducer
> passes for me with our 2.28 backport (glibc-2.28-164.el8) if I run it
> like this:
>
> GLIBC_TUNABLES=glibc.rtld.optional_static_tls=4000 make
>
> The best we can do in the short term would be an increase of the default
> limit. On 64-bit platforms, defaulting to a dozen or so kilobytes per
> thread should not be a problem as far as virtual address space
> consumption is concerned. We can also add an additional reservation of
> similar size for every auditor that is loaded, to compensate for the
> lack of auto-tuning of the TLS allocation size in auditing mode.
>
> The fundamental issue is that there is always going to be a hard limit
> for initial-exec TLS. Initial-exec TLS requires a fixed offset from the
> thread pointer, and we cannot relocate TLS variables because they are
> ordinary C objects with an observable address. There are some other
> things we can try to improve auto-tuning, but in the end, there is
> always going to be a fixed-size reserved area dedicated to initial-exec
> TLS set up at process startup, and with dlopen, that might not be enough
> even without any auditor use.
Jonathon,
does setting the environment variable work for you?
Thanks,
Florian
next prev parent reply other threads:[~2022-05-03 7:22 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-11 20:24 Jonathon Anderson
2022-04-12 7:44 ` Florian Weimer
2022-05-03 7:22 ` Florian Weimer [this message]
2022-05-05 17:30 ` Florian Weimer
2022-05-05 19:56 ` Jonathon Anderson
2022-05-11 13:59 ` Florian Weimer
2022-05-11 17:31 ` Jonathon Anderson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=875ymnxepr.fsf@oldenburg.str.redhat.com \
--to=fweimer@redhat.com \
--cc=adhemerval.zanella@linaro.org \
--cc=carlos@redhat.com \
--cc=janderson@rice.edu \
--cc=johnmc@rice.edu \
--cc=legendre1@llnl.gov \
--cc=libc-alpha@sourceware.org \
--cc=woodard@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).