Re: Problem with static const objects and LTO

[Jeff Law <law@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 4160 bytes --]


On 9/17/20 1:04 AM, Richard Biener wrote:
> On Wed, Sep 16, 2020 at 10:24 PM Jeff Law via Gcc-patches
> <gcc-patches@gcc.gnu.org> wrote:
>>
>> On 9/16/20 11:52 AM, Joseph Myers wrote:
>>> On Wed, 16 Sep 2020, Jeff Law via Gcc-patches wrote:
>>>
>>>> ISTM this is a lot like the problem we have where we inline functions
>>>> with static data.   To fix those we use STB_GNU_UNIQUE.  But I don't see
>>>> any code in the C front-end which would utilize STB_GNU_UNIQUE.  It's
>>>> support seems limited to C++.
>>>>
>>>>
>>>> How is this supposed to work for C?
>>> C inline functions don't try to address this.  The standard has a rule "An
>>> inline definition of a function with external linkage shall not contain a
>>> definition of a modifiable object with static or thread storage duration,
>>> and shall not contain a reference to an identifier with internal
>>> linkage.", which avoids some cases of this, but you can still get multiple
>>> copies of objects (with static storage duration, not modifiable, no
>>> linkage, i.e. static const defined inside an inline function) as noted in
>>> a footnote "Since an inline definition is distinct from the corresponding
>>> external definition and from any other corresponding inline definitions in
>>> other translation units, all corresponding objects with static storage
>>> duration are also distinct in each of the definitions.".
>> I was kindof guessing C would end up with something like the above --
>> I'm just happy its as well documented as it is.  Thanks!
>>
>>
>>
>> The more I ponder this problem the more worried I get.   Before LTO we
>> could consider the TU an indivisible unit and if the main program
>> referenced something in the TU, then a static link of that TU would
>> bring the entire TU into the main program and that would satisfy all
>> references to all symbols defined by the TU, including those in any DSOs
>> used by the main program.  So it's an indivisible unit at runtime too.
>> I could change internal data structures, rebuild the static library &
>> main program (leaving the DSOs alone) and it'd just work.
>>
>>
>> In an LTO world the TU isn't indivisible anymore.  LTO will happily
>> discard things which don't appear to be used.   So parts of the TU may
>> be in the main program, other parts may be in DSOs used by the main
>> program.   This can mean that objects are unexpectedly being passed
>> across DSO boundaries and the details of those objects has, in effect,
>> become part of the ABI.  If I was to change an internal data structure,
>> build the static library and main program we could get bad behavior
>> because an instance of that data structure could be passed to a DSO
>> because the main executable is "incomplete" and we end up calling copies
>> of routines from the (not rebuilt) DSOs.
> I think the situation is simpler - LTO can duplicate data objects for the
> purpose of optimization within some constraints and there might be a simple
> error in it thinking duplicating of the static const object into two LTRANS
> units is OK.  So - do you have a testcase?

man-db trips over this.  The executable links against a static version
of gnulib as well as linking dynamically to DSOs which themselves were
linked against a static gnulib.  We're getting bits of regcomp.c in the
main executable and other bits are in a DSO.  We have two copies of
utf8_sb_map (one in the main executable, another in a DSO) -- and the
code within regcomp assumes there is only one copy of utf8_sb_map.

Prior to LTO, when the main executable linked against the static gnulib
and had to pull in anything from regcomp.c, it ended up pulling in *all*
of regcomp.c into the main executable and those definitions overrode
anything in the DSO and it "just worked", though it is a nightmare from
a composability standpoint.


>
> That said, LTO doesn't produce both the main program and the DSO
> at the same time - you produce both separately and the latent "hidden ABI"
> issue would exist even w/o LTO.

Absolutely.  This is a composibility problem that's made worse by LTO.


jeff


[-- Attachment #2: pEpkey.asc --]
[-- Type: application/pgp-keys, Size: 1763 bytes --]