* Resolving LTO symbols for static library boundary
@ 2018-02-05 11:27 Allan Sandfeld Jensen
2018-02-05 17:45 ` Richard Biener
0 siblings, 1 reply; 4+ messages in thread
From: Allan Sandfeld Jensen @ 2018-02-05 11:27 UTC (permalink / raw)
To: gcc
Hello GCC
In trying to make it possible to use LTO for distro-builds of Qt, I have again
hit the problem of static libraries. In Qt in general we for LTO rely on a
library boundary, where LTO gets resolved when generating the library but no
LTO-symbols are exported in the shared library. This ensure the library has a
well defined binary compatible interface and gets LTO optimizations within
each library. For some private libraries we use static libraries however,
because we don't need binary compatibility, but though we don't need BC
between Qt versions, the libraries should still be linkable with different gcc
versions (and with different compilers). However when LTO is enabled the
static libraries will contain definitions that depend on a single gcc version
making it unsuitable for distribution.
One solution is to enable fat-lto object files for static libraries but that
is both a waste of space and compile time, and disables any LTO optimization
within the library. Ideally I would like to have the static library do LTO
optimizations internally just like a shared library, but then exported as
static library.
I suspect this is more of gcc task than ar/ld task, as it basically boils down
to gcc doing for a static library what it does for shared library, but maybe
export the result as a single combined .o file, that can then be ar'ed into a
compatible static library.
Is this possible?
Best regards
'Allan Jensen
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Resolving LTO symbols for static library boundary
2018-02-05 11:27 Resolving LTO symbols for static library boundary Allan Sandfeld Jensen
@ 2018-02-05 17:45 ` Richard Biener
2018-02-06 0:01 ` Jan Hubicka
0 siblings, 1 reply; 4+ messages in thread
From: Richard Biener @ 2018-02-05 17:45 UTC (permalink / raw)
To: gcc, Allan Sandfeld Jensen, jh
On February 5, 2018 12:26:58 PM GMT+01:00, Allan Sandfeld Jensen <linux@carewolf.com> wrote:
>Hello GCC
>
>In trying to make it possible to use LTO for distro-builds of Qt, I
>have again
>hit the problem of static libraries. In Qt in general we for LTO rely
>on a
>library boundary, where LTO gets resolved when generating the library
>but no
>LTO-symbols are exported in the shared library. This ensure the library
>has a
>well defined binary compatible interface and gets LTO optimizations
>within
>each library. For some private libraries we use static libraries
>however,
>because we don't need binary compatibility, but though we don't need BC
>
>between Qt versions, the libraries should still be linkable with
>different gcc
>versions (and with different compilers). However when LTO is enabled
>the
>static libraries will contain definitions that depend on a single gcc
>version
>making it unsuitable for distribution.
>
>One solution is to enable fat-lto object files for static libraries but
>that
>is both a waste of space and compile time, and disables any LTO
>optimization
>within the library. Ideally I would like to have the static library do
>LTO
>optimizations internally just like a shared library, but then exported
>as
>static library.
>
>I suspect this is more of gcc task than ar/ld task, as it basically
>boils down
>to gcc doing for a static library what it does for shared library, but
>maybe
>export the result as a single combined .o file, that can then be ar'ed
>into a
>compatible static library.
>
>Is this possible?
Hmm. I think you could partially link the static archive contents into a single relocatable object. Or we could add a mode where you do a 1to1 LTO link of the objects and stop at the ltrans object files. You could stuff those into an archive again.
I'm not sure how far Honza got partial LTO linking to work?
Richard.
>Best regards
>'Allan Jensen
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Resolving LTO symbols for static library boundary
2018-02-05 17:45 ` Richard Biener
@ 2018-02-06 0:01 ` Jan Hubicka
2018-02-07 8:40 ` Allan Sandfeld Jensen
0 siblings, 1 reply; 4+ messages in thread
From: Jan Hubicka @ 2018-02-06 0:01 UTC (permalink / raw)
To: Richard Biener; +Cc: gcc, Allan Sandfeld Jensen, jh
Dne 2018-02-05 18:44, Richard Biener napsal:
> On February 5, 2018 12:26:58 PM GMT+01:00, Allan Sandfeld Jensen
> <linux@carewolf.com> wrote:
>> Hello GCC
>>
>> In trying to make it possible to use LTO for distro-builds of Qt, I
>> have again
>> hit the problem of static libraries. In Qt in general we for LTO rely
>> on a
>> library boundary, where LTO gets resolved when generating the library
>> but no
>> LTO-symbols are exported in the shared library. This ensure the
>> library
>> has a
>> well defined binary compatible interface and gets LTO optimizations
>> within
>> each library. For some private libraries we use static libraries
>> however,
>> because we don't need binary compatibility, but though we don't need
>> BC
>>
>> between Qt versions, the libraries should still be linkable with
>> different gcc
>> versions (and with different compilers). However when LTO is enabled
>> the
>> static libraries will contain definitions that depend on a single gcc
>> version
>> making it unsuitable for distribution.
>>
>> One solution is to enable fat-lto object files for static libraries
>> but
>> that
>> is both a waste of space and compile time, and disables any LTO
>> optimization
>> within the library. Ideally I would like to have the static library do
>> LTO
>> optimizations internally just like a shared library, but then exported
>> as
>> static library.
>>
>> I suspect this is more of gcc task than ar/ld task, as it basically
>> boils down
>> to gcc doing for a static library what it does for shared library, but
>> maybe
>> export the result as a single combined .o file, that can then be ar'ed
>> into a
>> compatible static library.
>>
>> Is this possible?
>
> Hmm. I think you could partially link the static archive contents into
> a single relocatable object. Or we could add a mode where you do a
> 1to1 LTO link of the objects and stop at the ltrans object files. You
> could stuff those into an archive again.
>
> I'm not sure how far Honza got partial LTO linking to work?
Parital linking of lto .o files into single non-lto .o file should work
and it will get you cross-module optimization done. The problem is that
without resolution info compiler needs to assume that all symbols
exported by object files are possibly referneced by the later
incremental link and thus the code quality will definitly not be
comparable with what you get for LTO on final binary or DSO. Still
should be better than non-lto build.
I would be curious if it is useful for you in practice.
Honza
>
> Richard.
>
>> Best regards
>> 'Allan Jensen
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Resolving LTO symbols for static library boundary
2018-02-06 0:01 ` Jan Hubicka
@ 2018-02-07 8:40 ` Allan Sandfeld Jensen
0 siblings, 0 replies; 4+ messages in thread
From: Allan Sandfeld Jensen @ 2018-02-07 8:40 UTC (permalink / raw)
To: gcc; +Cc: Jan Hubicka, Richard Biener, jh
On Dienstag, 6. Februar 2018 01:01:29 CET Jan Hubicka wrote:
> Dne 2018-02-05 18:44, Richard Biener napsal:
> > On February 5, 2018 12:26:58 PM GMT+01:00, Allan Sandfeld Jensen
> >
> > <linux@carewolf.com> wrote:
> >> Hello GCC
> >>
> >> In trying to make it possible to use LTO for distro-builds of Qt, I
> >> have again
> >> hit the problem of static libraries. In Qt in general we for LTO rely
> >> on a
> >> library boundary, where LTO gets resolved when generating the library
> >> but no
> >> LTO-symbols are exported in the shared library. This ensure the
> >> library
> >> has a
> >> well defined binary compatible interface and gets LTO optimizations
> >> within
> >> each library. For some private libraries we use static libraries
> >> however,
> >> because we don't need binary compatibility, but though we don't need
> >> BC
> >>
> >> between Qt versions, the libraries should still be linkable with
> >> different gcc
> >> versions (and with different compilers). However when LTO is enabled
> >> the
> >> static libraries will contain definitions that depend on a single gcc
> >> version
> >> making it unsuitable for distribution.
> >>
> >> One solution is to enable fat-lto object files for static libraries
> >> but
> >> that
> >> is both a waste of space and compile time, and disables any LTO
> >> optimization
> >> within the library. Ideally I would like to have the static library do
> >> LTO
> >> optimizations internally just like a shared library, but then exported
> >> as
> >> static library.
> >>
> >> I suspect this is more of gcc task than ar/ld task, as it basically
> >> boils down
> >> to gcc doing for a static library what it does for shared library, but
> >> maybe
> >> export the result as a single combined .o file, that can then be ar'ed
> >> into a
> >> compatible static library.
> >>
> >> Is this possible?
> >
> > Hmm. I think you could partially link the static archive contents into
> > a single relocatable object. Or we could add a mode where you do a
> > 1to1 LTO link of the objects and stop at the ltrans object files. You
> > could stuff those into an archive again.
> >
> > I'm not sure how far Honza got partial LTO linking to work?
>
> Parital linking of lto .o files into single non-lto .o file should work
> and it will get you cross-module optimization done. The problem is that
> without resolution info compiler needs to assume that all symbols
> exported by object files are possibly referneced by the later
> incremental link and thus the code quality will definitly not be
> comparable with what you get for LTO on final binary or DSO. Still
> should be better than non-lto build.
> I would be curious if it is useful for you in practice.
>
How would I do that partial link, and what are the requirements?
Best regards
'Allan
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2018-02-07 8:40 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-02-05 11:27 Resolving LTO symbols for static library boundary Allan Sandfeld Jensen
2018-02-05 17:45 ` Richard Biener
2018-02-06 0:01 ` Jan Hubicka
2018-02-07 8:40 ` Allan Sandfeld Jensen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).