shared libraries + lto ?

shared libraries + lto ?

[Alain Meunier]

Hello,

I would like to know if one can use lto with shared libraries and leverage all the goodness of both worlds ?

My tests show that it works but not sure if lto brang something or not in the game.

gcc -O3 -shared -fPIC my_shared.c -o libmy_shared.so -lto

and linking

gcc -O3 my_app.c -o my_binary -lmy_shared -lto

I would like to keep the ability to have shared libraries. Will gcc make something out of it ?

Thanks 		 	   		  

Re: shared libraries + lto ?

[Vincent Lefevre]

On 2014-08-01 11:09:38 +0200, Alain Meunier wrote:
> I would like to know if one can use lto with shared libraries and
> leverage all the goodness of both worlds ?
> 
> My tests show that it works but not sure if lto brang something or
> not in the game.

I did some timings with MPFR + GMP two years ago and I found that it
was useless to use LTO with the shared library (I even wonder whether
this can make sense at all). Here are the results:

Precision 10:
               shared        static
             arg   macro   arg   macro
Default     3.480  3.470  2.670  2.690
LTO paths   4.000  3.980  2.640  2.660
With LTO    4.110  3.970  2.320  2.410

Precision 80:
               shared        static
             arg   macro   arg   macro
Default     5.520  5.470  4.950  5.000
LTO paths   5.510  5.500  4.440  4.470
With LTO    5.540  5.520  4.040  4.120

Precision 300:
               shared        static
             arg   macro   arg   macro
Default     6.770  6.560  5.950  5.960
LTO paths   6.140  5.980  5.060  5.020
With LTO    5.980  5.960  4.280  4.400

Conclusion (on these examples):
  * There isn't much difference between a precision given in argument
    and a fixed precision given via a macro (known at compile time of
    the main program).
  * Using a static library instead of a shared library can yield a
    speedup of up to 44% (this happens with LTO enabled), i.e. that's
    almost twice as fast!
  * LTO should be used only with -static (for performance, but also
    when considering practical use, it is pointless to use LTO with
    shared libraries).
  * The LTO speedup ("With LTO" compared to "LTO paths" in static) can
    be up to 15% (28% if we compare to the default static library, but
    we are not just measuring LTO in this case).
  * The LTO speedup compared to traditional linking (shared library
    from the vendor, here Debian/unstable) can be up to 37%.

Note: The versions of MPFR in "Default" (Debian packages providing
MPFR 3.1.0-p10) and with LTO paths (MPFR 3.1.1-p2) are not exactly
the same, but the differences consist only of bug fixes, so that the
tested source code should be the same.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

RE: shared libraries + lto ?

[Alain Meunier]

Thanks Vincent,
 
I think I will not use shared libraries in this case.
I will stick static ones.
 
But on the whole net there are many different expressions of correctness when related to lto.
Could you clarify :
 
say I have a static library libfoo.a:
void function cool(int * restrict a,int * restrict b){
       //do something useful
}
 
I also have  libbar.a:
void function eatTheWorld(int * restrict a,int * restrict b){
       //do something useful
}
Both compiled with gcc 
gcc /*optim. flags here*/ -fPIC foo.c -o libfoo.a -flto
gcc /*optim. flags here*/ -fPIC bar.c -o libbar.a -flto
 
and a main program my_app.c :
 
#include here
int main(){
    int f1 = 5;
    int f2 = 3;
    cool(&f1,&f2);
    eatTheWorld(&f1,&f2)
 
return 0;
}
 
I will compile it with
gcc /*optim. flags here*/  my_app.c -lfoo -lbar -flto
 
Is that it ? Nothing more ?
 
This article suggests otherwise : http://hubicka.blogspot.fr/2014/04/linktime-optimization-in-gcc-2-firefox.html
 
No plugin or all this mess ?
I am on Debian testing 64 bits.
 
The lto best use is still a bit unclear to yield the best performance, at least to me.
----------------------------------------
> Date: Fri, 1 Aug 2014 12:47:16 +0200
> From: vincent+gcc@vinc17.org
> To: gcc-help@gcc.gnu.org
> Subject: Re: shared libraries + lto ?
>
> On 2014-08-01 11:09:38 +0200, Alain Meunier wrote:
>> I would like to know if one can use lto with shared libraries and
>> leverage all the goodness of both worlds ?
>>
>> My tests show that it works but not sure if lto brang something or
>> not in the game.
>
> I did some timings with MPFR + GMP two years ago and I found that it
> was useless to use LTO with the shared library (I even wonder whether
> this can make sense at all). Here are the results:
>
> Precision 10:
> shared static
> arg macro arg macro
> Default 3.480 3.470 2.670 2.690
> LTO paths 4.000 3.980 2.640 2.660
> With LTO 4.110 3.970 2.320 2.410
>
> Precision 80:
> shared static
> arg macro arg macro
> Default 5.520 5.470 4.950 5.000
> LTO paths 5.510 5.500 4.440 4.470
> With LTO 5.540 5.520 4.040 4.120
>
> Precision 300:
> shared static
> arg macro arg macro
> Default 6.770 6.560 5.950 5.960
> LTO paths 6.140 5.980 5.060 5.020
> With LTO 5.980 5.960 4.280 4.400
>
> Conclusion (on these examples):
> * There isn't much difference between a precision given in argument
> and a fixed precision given via a macro (known at compile time of
> the main program).
> * Using a static library instead of a shared library can yield a
> speedup of up to 44% (this happens with LTO enabled), i.e. that's
> almost twice as fast!
> * LTO should be used only with -static (for performance, but also
> when considering practical use, it is pointless to use LTO with
> shared libraries).
> * The LTO speedup ("With LTO" compared to "LTO paths" in static) can
> be up to 15% (28% if we compare to the default static library, but
> we are not just measuring LTO in this case).
> * The LTO speedup compared to traditional linking (shared library
> from the vendor, here Debian/unstable) can be up to 37%.
>
> Note: The versions of MPFR in "Default" (Debian packages providing
> MPFR 3.1.0-p10) and with LTO paths (MPFR 3.1.1-p2) are not exactly
> the same, but the differences consist only of bug fixes, so that the
> tested source code should be the same.
>
> --
> Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
> 100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
> Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
 		 	   		  

Re: shared libraries + lto ?

[Vincent Lefevre]

On 2014-08-01 13:59:28 +0200, Alain Meunier wrote:
[...]
> Both compiled with gcc 
> gcc /*optim. flags here*/ -fPIC foo.c -o libfoo.a -flto
> gcc /*optim. flags here*/ -fPIC bar.c -o libbar.a -flto
>  
> and a main program my_app.c :
>  
> #include here
> int main(){
>     int f1 = 5;
>     int f2 = 3;
>     cool(&f1,&f2);
>     eatTheWorld(&f1,&f2)
>  
> return 0;
> }
>  
> I will compile it with
> gcc /*optim. flags here*/  my_app.c -lfoo -lbar -flto
>  
> Is that it ? Nothing more ?

I compiled everything with "-flto=jobserve -fuse-linker-plugin"
(that was two years ago), but I don't know whether this is still
necessary.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)