[wwwdocs] Update changes.html for LTO and IPA

[wwwdocs] Update changes.html for LTO and IPA

[Jan Hubicka]

Hi,
this patch mentiones few user visible changes I can think of.  I will
add some quality data on firefox once stage3 closes.

It seems that the optimization section of changes.html deserve some work :)

Honza

Index: changes.html
===================================================================
RCS file: /cvs/gcc/wwwdocs/htdocs/gcc-6/changes.html,v
retrieving revision 1.46
diff -u -r1.46 changes.html
--- changes.html	22 Dec 2015 19:23:31 -0000	1.46
+++ changes.html	19 Jan 2016 15:42:56 -0000
@@ -43,6 +43,64 @@
 	of array bounds.  In particular, it enables
 	<code>-fsanitize=bounds</code> as well as instrumentation of
 	flexible array member-like arrays.</li>
+    <li>Type based alias analysis now disambiguate accesses to different
+	pointers. This improve precision of the alias oracle by about 20-30%
+	on higher-level C++ programs. Programs doing invalid type punning
+	of pointer types may now need <code>-fno-strict-aliasing</code>
+	to work correctly.</li>
+    <li>Alias oracle now correctly supports <code>weakref</code> and
+	<code>alias</code> attributes. This makes it possible to access
+	both variable and its alias in one translation unit which is common
+	with link-time optimization.</li>
+    <li>Value range propagation now assume that <code>this</code> pointer
+	of C++ methods is non-NULL.  This eliminates many NULL pointer checks
+	but also breaks some non-conforming code-bases (such as Qt-5, Chromium,
+	KDevelop). As a termporary work-around
+	<code>-fno-delete-null-pointer-checks</code> can be used. Wrong
+	code can be identified <code>-fsanitize=undefined</code>.</li>
+    <li>Link-time optimization improvements:
+    <ul>
+      <li><code>warning</code> and <code>error</code> attributes are now
+	  correctly preserved by the declaration linking and thus
+	  <code>-D_FORTIFY_SOURCE=2</code> is now supported with <code>-flto</code>.</li>
+      <li><p>Type merging was fixed to handle C and Fortran interoperability
+	  rules as defined by the Fortran2005 language standard.</p>
+	  <p>
+	  As an exception, <code>CHARACTER(KIND=C_CHAR)</code> is not inter-operable
+	  with <code>char</code> in all cases because it is an array while
+	  <code>char</code> is scalar.
+	  <code>INTEGER(KIND=C_SIGNED_CHAR)</code> should be used instead.
+	  In general, this inter-operability can not be implemented, for
+	  example, on targets where function passing conventions of arrays
+	  differs from scalars.</p>
+      <li>More of type information is now preserved at link-time reducing
+	  the loss of accuracy of the type based alias analysis oracle compared
+	  to builds without link time optimization.</li>
+      <li>Invalid type punning on global variables and declarations is now
+	  reported with <code>-Wodr-type-mismatch</code>.</li>
+      <li>The size of LTO object files was reduced by about 11% (measured
+	  by compiling Firefox 46.0).</li>
+      <li>Link-time parallelization (enabled using <code>-flto=n</code>)
+	  was significantly improved by decreasing the size of streamed
+	  data when partitioning program.  The size of streamed
+	  IL while compiling Firefox 46.0 was reduced by 66%.</li>
+      <li>Linker plugin was extended to pass information about type of
+	  binary produced to GCC back-end (that can be also manually controlled
+	  by <code>-flinker-output</code>).  This makes it possible to
+	  properly confiugre code generator and support incremental
+	  linking. Inremental linking of LTO objects by <code>gcc -r</code> is
+	  now supported on plugin enabled setups. Because code generation happens
+	  during the incremnetal linking step, the whole program optimization
+	  is not performed. GCC 7 will support incremental IL linking.</li>
+    </ul></li>
+    <li>Inter-procedural optimization improvements:
+    <ul>
+      <li>Basic jump threading is now performed before profile construction
+	  and inline analysis resulting in more realistic size and time estimates
+	  that drive heuristics of inliner and function cloning passes.</li>
+      <li>Function cloning now more aggressively eliminate unused function
+	  parameters.</li>
+    </ul></li>
   </ul>
 
 <!-- .................................................................. -->

Re: [wwwdocs] Update changes.html for LTO and IPA

[Sandra Loosemore]

On 01/19/2016 08:45 AM, Jan Hubicka wrote:
> Hi,
> this patch mentiones few user visible changes I can think of.  I will
> add some quality data on firefox once stage3 closes.
>
> It seems that the optimization section of changes.html deserve some work :)
>
> Honza
>
> Index: changes.html
> ===================================================================
> RCS file: /cvs/gcc/wwwdocs/htdocs/gcc-6/changes.html,v
> retrieving revision 1.46
> diff -u -r1.46 changes.html
> --- changes.html	22 Dec 2015 19:23:31 -0000	1.46
> +++ changes.html	19 Jan 2016 15:42:56 -0000
> @@ -43,6 +43,64 @@
>   	of array bounds.  In particular, it enables
>   	<code>-fsanitize=bounds</code> as well as instrumentation of
>   	flexible array member-like arrays.</li>
> +    <li>Type based alias analysis now disambiguate accesses to different

s/Type based/Type-based/
s/disambiguate/disambiguates/

> +	pointers. This improve precision of the alias oracle by about 20-30%

s/improve/improves/

> +	on higher-level C++ programs. Programs doing invalid type punning
> +	of pointer types may now need <code>-fno-strict-aliasing</code>
> +	to work correctly.</li>
> +    <li>Alias oracle now correctly supports <code>weakref</code> and

Are users supposed to know what "Alias oracle" is?  (I don't....)

> +	<code>alias</code> attributes. This makes it possible to access
> +	both variable and its alias in one translation unit which is common

s/variable/a variable/

> +	with link-time optimization.</li>
> +    <li>Value range propagation now assume that <code>this</code> pointer

s/assume/assumes/

> +	of C++ methods is non-NULL.  This eliminates many NULL pointer checks
> +	but also breaks some non-conforming code-bases (such as Qt-5, Chromium,
> +	KDevelop). As a termporary work-around
> +	<code>-fno-delete-null-pointer-checks</code> can be used. Wrong
> +	code can be identified <code>-fsanitize=undefined</code>.</li>

s/identified/identified by using/

> +    <li>Link-time optimization improvements:
> +    <ul>
> +      <li><code>warning</code> and <code>error</code> attributes are now
> +	  correctly preserved by the declaration linking and thus

s/the declaration/declaration/

> +	  <code>-D_FORTIFY_SOURCE=2</code> is now supported with <code>-flto</code>.</li>
> +      <li><p>Type merging was fixed to handle C and Fortran interoperability
> +	  rules as defined by the Fortran2005 language standard.</p>

"Fortran2005" seems unlikely to be the correct spelling, and Wikipedia 
doesn't think there was a 2005 version of the Fortran standard at all. 
I'm confused.

> +	  <p>
> +	  As an exception, <code>CHARACTER(KIND=C_CHAR)</code> is not inter-operable
> +	  with <code>char</code> in all cases because it is an array while
> +	  <code>char</code> is scalar.
> +	  <code>INTEGER(KIND=C_SIGNED_CHAR)</code> should be used instead.
> +	  In general, this inter-operability can not be implemented, for
> +	  example, on targets where function passing conventions of arrays
> +	  differs from scalars.</p>
> +      <li>More of type information is now preserved at link-time reducing

s/More of/More/
s/link-time/link time,/

> +	  the loss of accuracy of the type based alias analysis oracle compared

s/type based/type-based/

Again, is "oracle" implementor-speak?

> +	  to builds without link time optimization.</li>

s/link time/link-time/

> +      <li>Invalid type punning on global variables and declarations is now
> +	  reported with <code>-Wodr-type-mismatch</code>.</li>
> +      <li>The size of LTO object files was reduced by about 11% (measured
> +	  by compiling Firefox 46.0).</li>
> +      <li>Link-time parallelization (enabled using <code>-flto=n</code>)
> +	  was significantly improved by decreasing the size of streamed
> +	  data when partitioning program.  The size of streamed

s/program/programs/

> +	  IL while compiling Firefox 46.0 was reduced by 66%.</li>

"IL" == implementor-speak....  Do users care about this statistic, or 
only about increased parallelism?

> +      <li>Linker plugin was extended to pass information about type of

s/Linker plugin/The linker plugin

> +	  binary produced to GCC back-end (that can be also manually controlled

s/GCC back-end/the GCC back end/

> +	  by <code>-flinker-output</code>).  This makes it possible to
> +	  properly confiugre code generator and support incremental

s/confiugre/configure/

> +	  linking. Inremental linking of LTO objects by <code>gcc -r</code> is

s/Iremental/Incremental/

> +	  now supported on plugin enabled setups. Because code generation happens

s/plugin enabled/plugin-enabled/

> +	  during the incremnetal linking step, the whole program optimization

s/incremnetal/incremental/

> +	  is not performed. GCC 7 will support incremental IL linking.</li>

"IL" again....  what does this mean to users?

> +    </ul></li>
> +    <li>Inter-procedural optimization improvements:
> +    <ul>
> +      <li>Basic jump threading is now performed before profile construction
> +	  and inline analysis resulting in more realistic size and time estimates

s/analysis/analysis,/

> +	  that drive heuristics of inliner and function cloning passes.</li>

s/heuristics of/the heuristics of the/

> +      <li>Function cloning now more aggressively eliminate unused function

s/eliminate/eliminates/

> +	  parameters.</li>
> +    </ul></li>
>     </ul>
>
>   <!-- .................................................................. -->
>

-Sandra the nit-picky

Re: [wwwdocs] Update changes.html for LTO and IPA

[Jan Hubicka]

> >+	  is not performed. GCC 7 will support incremental IL linking.</li>
> 
> "IL" again....  what does this mean to users?

Thanks for corrections, I will apply them and post updated patch.  Here I
wanted to explain that gcc -r should now give a correct code (while with
earlier GCC releases it will produce code that often works but sometimes
doesn't).

Doing incremental link however will prevent whole program optimization from
happening because it produces final assembly at gcc -r time.  To get this
working as expected you want gcc -r to incrementally link into the LTO IL again
that is implemented by patch 
https://gcc.gnu.org/ml/gcc-patches/2015-12/msg00000.html
but it seems it will only get in next stage 1.

Honza

Re: [wwwdocs] Update changes.html for LTO and IPA

[Jonathan Wakely]

On 19/01/16 16:45 +0100, Jan Hubicka wrote:
>Index: changes.html
>===================================================================
>RCS file: /cvs/gcc/wwwdocs/htdocs/gcc-6/changes.html,v
>retrieving revision 1.46
>diff -u -r1.46 changes.html
>--- changes.html	22 Dec 2015 19:23:31 -0000	1.46
>+++ changes.html	19 Jan 2016 15:42:56 -0000
>@@ -43,6 +43,64 @@
> 	of array bounds.  In particular, it enables
> 	<code>-fsanitize=bounds</code> as well as instrumentation of
> 	flexible array member-like arrays.</li>
>+    <li>Type based alias analysis now disambiguate accesses to different
>+	pointers. This improve precision of the alias oracle by about 20-30%
>+	on higher-level C++ programs. Programs doing invalid type punning
>+	of pointer types may now need <code>-fno-strict-aliasing</code>
>+	to work correctly.</li>
>+    <li>Alias oracle now correctly supports <code>weakref</code> and
>+	<code>alias</code> attributes. This makes it possible to access
>+	both variable and its alias in one translation unit which is common
>+	with link-time optimization.</li>
>+    <li>Value range propagation now assume that <code>this</code> pointer
>+	of C++ methods is non-NULL.  This eliminates many NULL pointer checks

s/of C++ methods/in C++ member functions/

>+	but also breaks some non-conforming code-bases (such as Qt-5, Chromium,
>+	KDevelop). As a termporary work-around

s/termporary/temporary/

Re: [wwwdocs] Update changes.html for LTO and IPA

[Gerald Pfeifer]

On Wed, 20 Jan 2016, Jan Hubicka wrote:
> this is updated patch.  I tried to explain better the situation WRT
> incremental linking.

Thank you, Jan.  I had a couple of editorial changes on top of
this, which I finally managed to commit.  (See the patch below.)

And one question: "declaration linking" is used in the description
of Link-time optimization improvements, alas that string does not 
appear anywhere in either the source tree or documentation?

On Wed, 3 Feb 2016, Jonathan Wakely wrote:
> s/of C++ methods/in C++ member functions/

This one got changed only partially in the patch applied last year, 
and I have adjusted this now to read "in C++ member functions" as 
opposed to "of C++ member functions".

Gerald

Index: changes.html
===================================================================
RCS file: /cvs/gcc/wwwdocs/htdocs/gcc-6/changes.html,v
retrieving revision 1.95
diff -u -r1.95 changes.html
--- changes.html	25 Feb 2017 17:46:38 -0000	1.95
+++ changes.html	27 Feb 2017 22:29:33 -0000
@@ -59,12 +59,12 @@
 	on higher-level C++ programs. Programs doing invalid type punning
 	of pointer types may now need <code>-fno-strict-aliasing</code>
 	to work correctly.</li>
-    <li>Alias analysis now correctly supports <code>weakref</code> and
-	<code>alias</code> attributes. This makes it possible to access
+    <li>Alias analysis now correctly supports the <code>weakref</code> and
+	<code>alias</code> attributes. This allows accessing
 	both a variable and its alias in one translation unit which is common
 	with link-time optimization.</li>
     <li>Value range propagation now assumes that the <code>this</code> pointer
-	of C++ member functions is non-null.  This eliminates
+	in C++ member functions is non-null.  This eliminates
 	common null pointer checks
 	but also breaks some non-conforming code-bases (such as Qt-5, Chromium,
 	KDevelop). As a temporary work-around
@@ -74,7 +74,8 @@
     <ul>
       <li><code>warning</code> and <code>error</code> attributes are now
 	  correctly preserved by declaration linking and thus
-	  <code>-D_FORTIFY_SOURCE=2</code> is now supported with <code>-flto</code>.</li>
+	  <code>-D_FORTIFY_SOURCE=2</code> is now supported with
+	  <code>-flto</code>.</li>
       <li><p>Type merging was fixed to handle C and Fortran interoperability
 	  rules as defined by the Fortran 2008 language standard.</p>
 	  <p>
@@ -83,10 +84,10 @@
 	  <code>char</code> is scalar.
 	  <code>INTEGER(KIND=C_SIGNED_CHAR)</code> should be used instead.
 	  In general, this inter-operability cannot be implemented, for
-	  example, on targets where function passing conventions of arrays
-	  differs from scalars.</p></li>
-      <li>More type information is now preserved at link time reducing
-	  the loss of accuracy of the type based alias analysis compared
+	  example on targets where the argument passing convention for
+	  arrays differs from scalars.</p></li>
+      <li>More type information is now preserved at link time, reducing
+	  the loss of accuracy of the type-based alias analysis compared
 	  to builds without link-time optimization.</li>
       <li>Invalid type punning on global variables and declarations is now
 	  reported with <code>-Wodr-type-mismatch</code>.</li>
@@ -96,30 +97,35 @@
 	  was significantly improved by decreasing the size of streamed
 	  data when partitioning programs.  The size of streamed
 	  IL while compiling Firefox 46.0 was reduced by 66%.</li>
-      <li><p>The linker plugin was extended to pass information about type of
-	  binary produced to GCC back end (that can be also manually controlled
-	  by <code>-flinker-output</code>).  This makes it possible to
+      <li><p>The linker plugin was extended to pass information about the
+	  type of binary produced to the GCC back end. (That can also be
+	  controlled manually by <code>-flinker-output</code>.)
+	  This makes it possible to
 	  properly configure the code generator and support incremental
-	  linking. Incremental linking of LTO objects by <code>gcc -r</code> is
-	  now supported on plugin-enabled setups.</p>
+	  linking. Incremental linking of LTO objects by <code>gcc -r</code>
+	  is now supported for plugin-enabled setups.</p>
 	  <p>There are two ways to perform incremental linking:</p>
 	  <ol>
 	     <li>Linking by <code>ld -r</code> will result in an object file
 	     with all sections from individual object files mechanically merged.
-	     This delays the actual link time optimization to final linking step
-	     and thus permits whole program optimization.  Linking final binary
+	     This delays the actual link-time optimization to the final
+	     linking step and thus permits whole program optimization.
+	     Linking the final binary
 	     with such object files is however slower.</li>
-	     <li>Linking by <code>gcc -r</code> will lead to link time optimization
-	     and produce final binary into the object file.  Linking such object
-	     file is fast but avoids any benefits from whole program optimization.</li>
+	     <li>Linking by <code>gcc -r</code> will lead to link-time
+	     optimization and emit the final binary into the object file.
+	     Linking such an object file is fast but avoids any benefits
+	     from whole program optimization.</li>
 	  </ol>
-	  GCC 7 will support incremental link-time optimization with <code>gcc -r</code>.</li>
+	  GCC 7 will support incremental link-time optimization with
+	  <code>gcc -r</code>.</li>
     </ul></li>
     <li>Inter-procedural optimization improvements:
     <ul>
       <li>Basic jump threading is now performed before profile construction
-	  and inline analysis, resulting in more realistic size and time estimates
-	  that drive the heuristics of the of inliner and function cloning passes.</li>
+	  and inline analysis, resulting in more realistic size and time
+	  estimates that drive the heuristics of the inliner and function
+	  cloning passes.</li>
       <li>Function cloning now more aggressively eliminates unused function
 	  parameters.</li>
     </ul></li>

Re: [wwwdocs] Update changes.html for LTO and IPA

[Jan Hubicka]

> On Wed, 20 Jan 2016, Jan Hubicka wrote:
> > this is updated patch.  I tried to explain better the situation WRT
> > incremental linking.
> 
> Thank you, Jan.  I had a couple of editorial changes on top of
> this, which I finally managed to commit.  (See the patch below.)
> 
> And one question: "declaration linking" is used in the description
> of Link-time optimization improvements, alas that string does not 
> appear anywhere in either the source tree or documentation?

It is my own name indeed. lto-symtab.c does merge declarations
when they refer to same symbol name. Newly we do not merge when the
symbol names does not appear semantically similar enough.
We can also call it declaration merging, but it is internal name anyway
and most link-time optimization papers reffer to these merging processes
as linking.

Honza
> 
> On Wed, 3 Feb 2016, Jonathan Wakely wrote:
> > s/of C++ methods/in C++ member functions/
> 
> This one got changed only partially in the patch applied last year, 
> and I have adjusted this now to read "in C++ member functions" as 
> opposed to "of C++ member functions".
> 
> Gerald
> 
> Index: changes.html
> ===================================================================
> RCS file: /cvs/gcc/wwwdocs/htdocs/gcc-6/changes.html,v
> retrieving revision 1.95
> diff -u -r1.95 changes.html
> --- changes.html	25 Feb 2017 17:46:38 -0000	1.95
> +++ changes.html	27 Feb 2017 22:29:33 -0000
> @@ -59,12 +59,12 @@
>  	on higher-level C++ programs. Programs doing invalid type punning
>  	of pointer types may now need <code>-fno-strict-aliasing</code>
>  	to work correctly.</li>
> -    <li>Alias analysis now correctly supports <code>weakref</code> and
> -	<code>alias</code> attributes. This makes it possible to access
> +    <li>Alias analysis now correctly supports the <code>weakref</code> and
> +	<code>alias</code> attributes. This allows accessing
>  	both a variable and its alias in one translation unit which is common
>  	with link-time optimization.</li>
>      <li>Value range propagation now assumes that the <code>this</code> pointer
> -	of C++ member functions is non-null.  This eliminates
> +	in C++ member functions is non-null.  This eliminates
>  	common null pointer checks
>  	but also breaks some non-conforming code-bases (such as Qt-5, Chromium,
>  	KDevelop). As a temporary work-around
> @@ -74,7 +74,8 @@
>      <ul>
>        <li><code>warning</code> and <code>error</code> attributes are now
>  	  correctly preserved by declaration linking and thus
> -	  <code>-D_FORTIFY_SOURCE=2</code> is now supported with <code>-flto</code>.</li>
> +	  <code>-D_FORTIFY_SOURCE=2</code> is now supported with
> +	  <code>-flto</code>.</li>
>        <li><p>Type merging was fixed to handle C and Fortran interoperability
>  	  rules as defined by the Fortran 2008 language standard.</p>
>  	  <p>
> @@ -83,10 +84,10 @@
>  	  <code>char</code> is scalar.
>  	  <code>INTEGER(KIND=C_SIGNED_CHAR)</code> should be used instead.
>  	  In general, this inter-operability cannot be implemented, for
> -	  example, on targets where function passing conventions of arrays
> -	  differs from scalars.</p></li>
> -      <li>More type information is now preserved at link time reducing
> -	  the loss of accuracy of the type based alias analysis compared
> +	  example on targets where the argument passing convention for
> +	  arrays differs from scalars.</p></li>
> +      <li>More type information is now preserved at link time, reducing
> +	  the loss of accuracy of the type-based alias analysis compared
>  	  to builds without link-time optimization.</li>
>        <li>Invalid type punning on global variables and declarations is now
>  	  reported with <code>-Wodr-type-mismatch</code>.</li>
> @@ -96,30 +97,35 @@
>  	  was significantly improved by decreasing the size of streamed
>  	  data when partitioning programs.  The size of streamed
>  	  IL while compiling Firefox 46.0 was reduced by 66%.</li>
> -      <li><p>The linker plugin was extended to pass information about type of
> -	  binary produced to GCC back end (that can be also manually controlled
> -	  by <code>-flinker-output</code>).  This makes it possible to
> +      <li><p>The linker plugin was extended to pass information about the
> +	  type of binary produced to the GCC back end. (That can also be
> +	  controlled manually by <code>-flinker-output</code>.)
> +	  This makes it possible to
>  	  properly configure the code generator and support incremental
> -	  linking. Incremental linking of LTO objects by <code>gcc -r</code> is
> -	  now supported on plugin-enabled setups.</p>
> +	  linking. Incremental linking of LTO objects by <code>gcc -r</code>
> +	  is now supported for plugin-enabled setups.</p>
>  	  <p>There are two ways to perform incremental linking:</p>
>  	  <ol>
>  	     <li>Linking by <code>ld -r</code> will result in an object file
>  	     with all sections from individual object files mechanically merged.
> -	     This delays the actual link time optimization to final linking step
> -	     and thus permits whole program optimization.  Linking final binary
> +	     This delays the actual link-time optimization to the final
> +	     linking step and thus permits whole program optimization.
> +	     Linking the final binary
>  	     with such object files is however slower.</li>
> -	     <li>Linking by <code>gcc -r</code> will lead to link time optimization
> -	     and produce final binary into the object file.  Linking such object
> -	     file is fast but avoids any benefits from whole program optimization.</li>
> +	     <li>Linking by <code>gcc -r</code> will lead to link-time
> +	     optimization and emit the final binary into the object file.
> +	     Linking such an object file is fast but avoids any benefits
> +	     from whole program optimization.</li>
>  	  </ol>
> -	  GCC 7 will support incremental link-time optimization with <code>gcc -r</code>.</li>
> +	  GCC 7 will support incremental link-time optimization with
> +	  <code>gcc -r</code>.</li>
>      </ul></li>
>      <li>Inter-procedural optimization improvements:
>      <ul>
>        <li>Basic jump threading is now performed before profile construction
> -	  and inline analysis, resulting in more realistic size and time estimates
> -	  that drive the heuristics of the of inliner and function cloning passes.</li>
> +	  and inline analysis, resulting in more realistic size and time
> +	  estimates that drive the heuristics of the inliner and function
> +	  cloning passes.</li>
>        <li>Function cloning now more aggressively eliminates unused function
>  	  parameters.</li>
>      </ul></li>

Re: [wwwdocs] Update changes.html for LTO and IPA

[Gerald Pfeifer]

On Tue, 28 Feb 2017, Jan Hubicka wrote:
>> And one question: "declaration linking" is used in the description
>> of Link-time optimization improvements, alas that string does not 
>> appear anywhere in either the source tree or documentation?
> It is my own name indeed. lto-symtab.c does merge declarations
> when they refer to same symbol name. Newly we do not merge when 
> the symbol names does not appear semantically similar enough.
> We can also call it declaration merging, but it is internal name 
> anyway and most link-time optimization papers reffer to these merging 
> processes as linking.

Thanks for the explanation, Honza.

In that case, I guess leave it as is unless someone else wants to
see this changed.


As another note, "declaration merging" once occurs in our codebase,
in varpool.c:

      /* When doing declaration merging we have duplicate
         entries for given decl.  Do not attempt to remove
         the boides, or we will end up remiving
         wrong one.  */

Should this read "Do not attempt to address this, or we risk removing
the wrong one" or something like that?

Unless