Incremental LTO linking part 7: documentation

Incremental LTO linking part 7: documentation

[Jan Hubicka]

Hi,
this patch adds documentation of -flinker-output.

	* doc/invoke.texi (-flinker-output): Document
Index: doc/invoke.texi
===================================================================
--- doc/invoke.texi	(revision 260042)
+++ doc/invoke.texi	(working copy)
@@ -12208,6 +12208,50 @@
 object file names should not be used as arguments.  @xref{Overall
 Options}.
 
+@item -flinker-output=@var{type}
+@opindex -flinker-output
+This option controls the code generation of the link time optimizer.  By
+default the linker output is determined by the linker plugin automatically. For
+debugging the compiler and in the case of incremental linking to non-lto object
+file is desired, it may be useful to control the type manually.
+
+If @var{type} is @samp{exec} the code generation is configured to produce static
+binary. In this case @option{-fpic} and @option{-fpie} are both disabled.
+
+If @var{type} is @samp{dyn} the code generation is configured to produce shared
+library. In this case @option{-fpic} or @option{-fPIC} is preserved, but not
+enabled automatically.  This makes it possible to build shared libraries without
+position independent code on architectures this is possible, i.e. on x86.
+
+If @var{type} is @samp{pie} the code generation is configured to produce
+@option{-fpie} executable. This result in similar optimizations as @samp{exec}
+except that @option{-fpie} is not disabled if specified at compilation time.
+
+If @var{type} is @samp{rel} the compiler assumes that incremental linking is
+done.  The sections containing intermediate code for link-time optimization are
+merged, pre-optimized, and output to the resulting object file. In addition, if
+@option{-ffat-lto-objects} is specified the binary code is produced for future
+non-lto linking. The object file produced by incremental linking will be smaller
+than a static library produced from the same object files.  At link-time the
+result of incremental linking will also load faster to compiler than a static
+library assuming that majority of objects in the library are used.
+
+Finally @samp{nolto-rel} configure compiler to for incremental linking where
+code generation is forced, final binary is produced and the intermediate code
+for later link-time optimization is stripped. When multiple object files are
+linked together the resulting code will be optimized better than with link time
+optimizations disabled (for example, the cross-module inlining will happen),
+most of benefits of whole program optimizations are however lost. 
+
+During the incremental link (by @option{-r}) the linker plugin will default to
+@option{rel}. With current interfaces to GNU Binutils it is however not
+possible to link incrementally LTO objects and non-LTO objects into a single
+mixed object file.  In the case any of object files in incremental link can not
+be used for link-time optimization the linker plugin will output warning and
+use @samp{nolto-rel}. To maintain the whole program optimization it is
+recommended to link such objects into static library instead. Alternatively it
+is possible to use H.J. Lu's binutils with support for mixed objects.
+
 @item -fuse-ld=bfd
 @opindex fuse-ld=bfd
 Use the @command{bfd} linker instead of the default linker.

Re: Incremental LTO linking part 7: documentation

[Martin Jambor]

Hi,

just small nits:

On Tue, May 08 2018, Jan Hubicka wrote:
> Hi,
> this patch adds documentation of -flinker-output.
>
> 	* doc/invoke.texi (-flinker-output): Document
> Index: doc/invoke.texi
> ===================================================================
> --- doc/invoke.texi	(revision 260042)
> +++ doc/invoke.texi	(working copy)
> @@ -12208,6 +12208,50 @@
>  object file names should not be used as arguments.  @xref{Overall
>  Options}.
>  
> +@item -flinker-output=@var{type}
> +@opindex -flinker-output
> +This option controls the code generation of the link time optimizer.  By
> +default the linker output is determined by the linker plugin automatically. For
> +debugging the compiler and in the case of incremental linking to non-lto object
> +file is desired, it may be useful to control the type manually.
> +
> +If @var{type} is @samp{exec} the code generation is configured to produce static
> +binary. In this case @option{-fpic} and @option{-fpie} are both disabled.
> +
> +If @var{type} is @samp{dyn} the code generation is configured to produce shared
> +library. In this case @option{-fpic} or @option{-fPIC} is preserved, but not
> +enabled automatically.  This makes it possible to build shared libraries without
> +position independent code on architectures this is possible, i.e. on x86.

on architectures *where* this is possible?

> +
> +If @var{type} is @samp{pie} the code generation is configured to produce
> +@option{-fpie} executable. This result in similar optimizations as @samp{exec}
> +except that @option{-fpie} is not disabled if specified at compilation time.
> +
> +If @var{type} is @samp{rel} the compiler assumes that incremental linking is
> +done.  The sections containing intermediate code for link-time optimization are
> +merged, pre-optimized, and output to the resulting object file. In addition, if
> +@option{-ffat-lto-objects} is specified the binary code is produced for future
> +non-lto linking. The object file produced by incremental linking will be smaller
> +than a static library produced from the same object files.  At link-time the
> +result of incremental linking will also load faster to compiler than a static
> +library assuming that majority of objects in the library are used.
> +
> +Finally @samp{nolto-rel} configure compiler to for incremental linking where
> +code generation is forced, final binary is produced and the intermediate code
> +for later link-time optimization is stripped. When multiple object files are
> +linked together the resulting code will be optimized better than with link time
> +optimizations disabled (for example, the cross-module inlining will happen),
> +most of benefits of whole program optimizations are however lost. 
> +
> +During the incremental link (by @option{-r}) the linker plugin will default to
> +@option{rel}. With current interfaces to GNU Binutils it is however not
> +possible to link incrementally LTO objects and non-LTO objects into a single
> +mixed object file.  In the case any of object files in incremental link can not
> +be used for link-time optimization the linker plugin will output warning and
> +use @samp{nolto-rel}. To maintain the whole program optimization it is
> +recommended to link such objects into static library instead. Alternatively it
> +is possible to use H.J. Lu's binutils with support for mixed objects.
> +

I wonder whether this will be still true and what will people make of
this two years from now.  Perhaps add a reference to the current
binutils version?

Martin

Re: Incremental LTO linking part 7: documentation

[Jan Hubicka]

> Hi,
> 
> just small nits:
> 
> On Tue, May 08 2018, Jan Hubicka wrote:
> > Hi,
> > this patch adds documentation of -flinker-output.
> >
> > 	* doc/invoke.texi (-flinker-output): Document
> > Index: doc/invoke.texi
> > ===================================================================
> > --- doc/invoke.texi	(revision 260042)
> > +++ doc/invoke.texi	(working copy)
> > @@ -12208,6 +12208,50 @@
> >  object file names should not be used as arguments.  @xref{Overall
> >  Options}.
> >  
> > +@item -flinker-output=@var{type}
> > +@opindex -flinker-output
> > +This option controls the code generation of the link time optimizer.  By
> > +default the linker output is determined by the linker plugin automatically. For
> > +debugging the compiler and in the case of incremental linking to non-lto object
> > +file is desired, it may be useful to control the type manually.
> > +
> > +If @var{type} is @samp{exec} the code generation is configured to produce static
> > +binary. In this case @option{-fpic} and @option{-fpie} are both disabled.
> > +
> > +If @var{type} is @samp{dyn} the code generation is configured to produce shared
> > +library. In this case @option{-fpic} or @option{-fPIC} is preserved, but not
> > +enabled automatically.  This makes it possible to build shared libraries without
> > +position independent code on architectures this is possible, i.e. on x86.
> 
> on architectures *where* this is possible?
> 
> > +
> > +If @var{type} is @samp{pie} the code generation is configured to produce
> > +@option{-fpie} executable. This result in similar optimizations as @samp{exec}
> > +except that @option{-fpie} is not disabled if specified at compilation time.
> > +
> > +If @var{type} is @samp{rel} the compiler assumes that incremental linking is
> > +done.  The sections containing intermediate code for link-time optimization are
> > +merged, pre-optimized, and output to the resulting object file. In addition, if
> > +@option{-ffat-lto-objects} is specified the binary code is produced for future
> > +non-lto linking. The object file produced by incremental linking will be smaller
> > +than a static library produced from the same object files.  At link-time the
> > +result of incremental linking will also load faster to compiler than a static
> > +library assuming that majority of objects in the library are used.
> > +
> > +Finally @samp{nolto-rel} configure compiler to for incremental linking where
> > +code generation is forced, final binary is produced and the intermediate code
> > +for later link-time optimization is stripped. When multiple object files are
> > +linked together the resulting code will be optimized better than with link time
> > +optimizations disabled (for example, the cross-module inlining will happen),
> > +most of benefits of whole program optimizations are however lost. 
> > +
> > +During the incremental link (by @option{-r}) the linker plugin will default to
> > +@option{rel}. With current interfaces to GNU Binutils it is however not
> > +possible to link incrementally LTO objects and non-LTO objects into a single
> > +mixed object file.  In the case any of object files in incremental link can not
> > +be used for link-time optimization the linker plugin will output warning and
> > +use @samp{nolto-rel}. To maintain the whole program optimization it is
> > +recommended to link such objects into static library instead. Alternatively it
> > +is possible to use H.J. Lu's binutils with support for mixed objects.
> > +
> 
> I wonder whether this will be still true and what will people make of
> this two years from now.  Perhaps add a reference to the current
> binutils version?

Current GCC won't work better with newer binutils without changes at GCC side.
It does not seem that HJ's and GNU binutils will converge and if they do, we
will need to support it at lto-plugin time and while adding the support we will
also update the docs :)
> 
> Martin

Re: Incremental LTO linking part 7: documentation

[Gerald Pfeifer]

On Tue, 8 May 2018, Jan Hubicka wrote:
> this patch adds documentation of -flinker-output.
> 
> 	* doc/invoke.texi (-flinker-output): Document

I found a follow-up patch to this in a local tree that had been
sitting there for a while, had a another look, and now committed
it.

Sandra, if you could have another pass that would be really good;
I'm sure you'll be able to further improve.

Gerald

2019-05-26  Gerald Pfeifer  <gerald@pfeifer.com>

	* doc/invoke.texi (Link Options): Many editorial changes around
	-flinker-output.

Index: doc/invoke.texi
===================================================================
--- doc/invoke.texi	(revision 271632)
+++ doc/invoke.texi	(working copy)
@@ -13141,44 +13141,48 @@ Options}.
 
 @item -flinker-output=@var{type}
 @opindex flinker-output
-This option controls the code generation of the link time optimizer.  By
-default the linker output is determined by the linker plugin automatically. For
-debugging the compiler and in the case of incremental linking to non-lto object
-file is desired, it may be useful to control the type manually.
+This option controls code generation of the link time optimizer.  By
+default the linker output is automatically determined by the linker
+plugin.  For debugging the compiler and if incremental linking with a 
+non-LTO object file is desired, it may be useful to control the type
+manually.
 
-If @var{type} is @samp{exec} the code generation is configured to produce static
-binary. In this case @option{-fpic} and @option{-fpie} are both disabled.
+If @var{type} is @samp{exec} code generation produces a static
+binary. In this case @option{-fpic} and @option{-fpie} are both
+disabled.
 
-If @var{type} is @samp{dyn} the code generation is configured to produce shared
-library. In this case @option{-fpic} or @option{-fPIC} is preserved, but not
-enabled automatically.  This makes it possible to build shared libraries without
-position independent code on architectures this is possible, i.e.@: on x86.
+If @var{type} is @samp{dyn} code generation produces a shared
+library.  In this case @option{-fpic} or @option{-fPIC} is preserved,
+but not enabled automatically.  This allows to build shared libraries
+without position independent code on architectures where this is
+possible, i.e.@: on x86.
 
-If @var{type} is @samp{pie} the code generation is configured to produce
-@option{-fpie} executable. This result in similar optimizations as @samp{exec}
-except that @option{-fpie} is not disabled if specified at compilation time.
+If @var{type} is @samp{pie} code generation produces an @option{-fpie}
+executable. This results in similar optimizations as @samp{exec}
+except that @option{-fpie} is not disabled if specified at compilation
+time.
 
 If @var{type} is @samp{rel} the compiler assumes that incremental linking is
 done.  The sections containing intermediate code for link-time optimization are
 merged, pre-optimized, and output to the resulting object file. In addition, if
 @option{-ffat-lto-objects} is specified the binary code is produced for future
-non-lto linking. The object file produced by incremental linking will be smaller
-than a static library produced from the same object files.  At link-time the
+non-LTO linking. The object file produced by incremental linking will be smaller
+than a static library produced from the same object files.  At link time the
 result of incremental linking will also load faster to compiler than a static
-library assuming that majority of objects in the library are used.
+library assuming that the majority of objects in the library are used.
 
-Finally @samp{nolto-rel} configure compiler to for incremental linking where
-code generation is forced, final binary is produced and the intermediate code
-for later link-time optimization is stripped. When multiple object files are
-linked together the resulting code will be optimized better than with link time
-optimizations disabled (for example, the cross-module inlining will happen),
-most of benefits of whole program optimizations are however lost. 
+Finally @samp{nolto-rel} configures the compiler for incremental linking where
+code generation is forced, a final binary is produced and the intermediate
+code for later link-time optimization is stripped. When multiple object files
+are linked together the resulting code will be optimized better than with
+link-time optimizations disabled (for example, cross-module inlining will
+happen), most of benefits of whole program optimizations are however lost. 
 
 During the incremental link (by @option{-r}) the linker plugin will default to
 @option{rel}. With current interfaces to GNU Binutils it is however not
-possible to link incrementally LTO objects and non-LTO objects into a single
+possible to incrementally link LTO objects and non-LTO objects into a single
 mixed object file.  In the case any of object files in incremental link cannot
-be used for link-time optimization the linker plugin will output warning and
+be used for link-time optimization the linker plugin will issue a warning and
 use @samp{nolto-rel}. To maintain the whole program optimization it is
 recommended to link such objects into static library instead. Alternatively it
 is possible to use H.J. Lu's binutils with support for mixed objects.

Re: Incremental LTO linking part 7: documentation

[Sandra Loosemore]

[-- Attachment #1: Type: text/plain, Size: 1054 bytes --]

On 5/26/19 11:35 AM, Gerald Pfeifer wrote:
> On Tue, 8 May 2018, Jan Hubicka wrote:
>> this patch adds documentation of -flinker-output.
>>
>> 	* doc/invoke.texi (-flinker-output): Document
> 
> I found a follow-up patch to this in a local tree that had been
> sitting there for a while, had a another look, and now committed
> it.
> 
> Sandra, if you could have another pass that would be really good;
> I'm sure you'll be able to further improve.

I've checked in the attached patch, which I've also had sitting around 
for a few weeks now.  I kept getting sidetracked when working on this... 
  first I noticed that we were not consistently spelling "link-time 
optimization" like that, then I noticed that terms like "WPA" were not 
defined in the user documentation, there are some LTO-related options 
that are listed in the option summary in the user manual but only 
documented in the internals manual, etc.  I've finally decided I should 
just commit this piece as-is to get it out of my tree; it's an 
incremental improvement, anyway.

-Sandra

[-- Attachment #2: gccdoc.log --]
[-- Type: text/x-log, Size: 150 bytes --]

2019-07-01  Sandra Loosemore  <sandra@codesourcery.com>

	gcc/
	* doc/invoke.texi (Link Options): Further editorial changes to
	-flinker-output docs.

[-- Attachment #3: gccdoc.patch --]
[-- Type: text/x-patch, Size: 5063 bytes --]

Index: gcc/doc/invoke.texi
===================================================================
--- gcc/doc/invoke.texi	(revision 272886)
+++ gcc/doc/invoke.texi	(working copy)
@@ -6007,8 +6007,9 @@ Warn about types with virtual methods wh
 if the type were declared with the C++11 @code{final} specifier, 
 or, if possible,
 declared in an anonymous namespace. This allows GCC to more aggressively
-devirtualize the polymorphic calls. This warning is more effective with link
-time optimization, where the information about the class hierarchy graph is
+devirtualize the polymorphic calls. This warning is more effective with 
+link-time optimization, 
+where the information about the class hierarchy graph is
 more complete.
 
 @item -Wsuggest-final-methods
@@ -13190,49 +13191,49 @@ Options}.
 
 @item -flinker-output=@var{type}
 @opindex flinker-output
-This option controls code generation of the link time optimizer.  By
+This option controls code generation of the link-time optimizer.  By
 default the linker output is automatically determined by the linker
 plugin.  For debugging the compiler and if incremental linking with a 
 non-LTO object file is desired, it may be useful to control the type
 manually.
 
-If @var{type} is @samp{exec} code generation produces a static
+If @var{type} is @samp{exec}, code generation produces a static
 binary. In this case @option{-fpic} and @option{-fpie} are both
 disabled.
 
-If @var{type} is @samp{dyn} code generation produces a shared
+If @var{type} is @samp{dyn}, code generation produces a shared
 library.  In this case @option{-fpic} or @option{-fPIC} is preserved,
 but not enabled automatically.  This allows to build shared libraries
-without position independent code on architectures where this is
+without position-independent code on architectures where this is
 possible, i.e.@: on x86.
 
-If @var{type} is @samp{pie} code generation produces an @option{-fpie}
+If @var{type} is @samp{pie}, code generation produces an @option{-fpie}
 executable. This results in similar optimizations as @samp{exec}
 except that @option{-fpie} is not disabled if specified at compilation
 time.
 
-If @var{type} is @samp{rel} the compiler assumes that incremental linking is
+If @var{type} is @samp{rel}, the compiler assumes that incremental linking is
 done.  The sections containing intermediate code for link-time optimization are
 merged, pre-optimized, and output to the resulting object file. In addition, if
-@option{-ffat-lto-objects} is specified the binary code is produced for future
-non-LTO linking. The object file produced by incremental linking will be smaller
+@option{-ffat-lto-objects} is specified, binary code is produced for future
+non-LTO linking. The object file produced by incremental linking is smaller
 than a static library produced from the same object files.  At link time the
-result of incremental linking will also load faster to compiler than a static
+result of incremental linking also loads faster than a static
 library assuming that the majority of objects in the library are used.
 
 Finally @samp{nolto-rel} configures the compiler for incremental linking where
-code generation is forced, a final binary is produced and the intermediate
+code generation is forced, a final binary is produced, and the intermediate
 code for later link-time optimization is stripped. When multiple object files
-are linked together the resulting code will be optimized better than with
-link-time optimizations disabled (for example, cross-module inlining will
-happen), most of benefits of whole program optimizations are however lost. 
+are linked together the resulting code is better optimized than with
+link-time optimizations disabled (for example, cross-module inlining 
+happens), but most of benefits of whole program optimizations are lost. 
 
-During the incremental link (by @option{-r}) the linker plugin will default to
+During the incremental link (by @option{-r}) the linker plugin defaults to
 @option{rel}. With current interfaces to GNU Binutils it is however not
 possible to incrementally link LTO objects and non-LTO objects into a single
-mixed object file.  In the case any of object files in incremental link cannot
-be used for link-time optimization the linker plugin will issue a warning and
-use @samp{nolto-rel}. To maintain the whole program optimization it is
+mixed object file.  If any of object files in incremental link cannot
+be used for link-time optimization, the linker plugin issues a warning and
+uses @samp{nolto-rel}. To maintain whole program optimization, it is
 recommended to link such objects into static library instead. Alternatively it
 is possible to use H.J. Lu's binutils with support for mixed objects.
 
@@ -15379,8 +15380,8 @@ Disabled by default.
 
 @item -flto-report-wpa
 @opindex flto-report-wpa
-Like @option{-flto-report}, but only print for the WPA phase of Link
-Time Optimization.
+Like @option{-flto-report}, but only print for the WPA phase of link-time
+optimization.
 
 @item -fmem-report
 @opindex fmem-report