Test with an lto-build of libgfortran.

Test with an lto-build of libgfortran.

[Toon Moene]

Hi all,

During the GNU Tools Cauldron we discussed (at the BoF: IPA & LTO) the 
possibility (and hazards) of building the run time libraries for various 
compilers with -flto, enabling an -flto -static linking of programs with 
the run time library available during link time optimizations.

Today I tried that on my (AMD Ryzen 7 5800U) laptop with

gcc version 14.0.0 20230926 (experimental) [master 
r14-4282-g53daf67fd55] (GCC)

with the following "quick hack":

diff --git a/libgfortran/configure b/libgfortran/configure
index cd176b04a14..69a2b4a8881 100755
--- a/libgfortran/configure
+++ b/libgfortran/configure
@@ -5959,11 +5959,11 @@ fi
  # Add -Wall -fno-repack-arrays -fno-underscoring if we are using GCC.
  have_real_17=no
  if test "x$GCC" = "xyes"; then
-  AM_FCFLAGS="-I . -Wall -Werror -fimplicit-none -fno-repack-arrays 
-fno-underscoring"
+  AM_FCFLAGS="-I . -Wall -Werror -fimplicit-none -fno-repack-arrays 
-fno-underscoring -flto"
    ## We like to use C11 and C99 routines when available.  This makes
    ## sure that
    ## __STDC_VERSION__ is set such that libc includes make them available.
-  AM_CFLAGS="-std=gnu11 -Wall -Wstrict-prototypes -Wmissing-prototypes 
-Wold-style-definition -Wextra -Wwrite-strings 
-Werror=implicit-function-declaration -Werror=vla"
+  AM_CFLAGS="-std=gnu11 -Wall -Wstrict-prototypes -Wmissing-prototypes 
-Wold-style-definition -Wextra -Wwrite-strings 
-Werror=implicit-function-declaration -Werror=vla -flto"
    ## Compile the following tests with the same system header contents
    ## that we'll encounter when compiling our own source files.
    CFLAGS="-std=gnu11 $CFLAGS"

The build of this compiler (languages=fortran) completed without 
problems (no test results - not enough time).

I then proceeded to build LAPACK with the following build options:

CFLAGS = -O3 -flto -flto-partition=none -static
and
FFLAGS = -O3 -flto -flto-partition=none -static

This gave the same test results of the LAPACK test suite as the build 
with the same compiler, but without an lto'd libgfortran.

The lto-ing of libgfortran did succeed, because I did get a new warning:

gfortran -O3 -flto -flto-partition=none -static  -o xlintstrfz zchkrfp.o 
zdrvrfp.o zdrvrf1.o zdrvrf2.o zdrvrf3.o zdrvrf4.o zerrrfp.o zlatb4.o 
zlaipd.o zlarhs.o zsbmv.o zget04.o zpot01.o zpot03.o zpot02.o chkxer.o 
xerbla.o alaerh.o aladhd.o alahd.o alasvm.o ../../libtmglib.a 
../../liblapack.a ../../librefblas.a
In function 'xtoa_big',
     inlined from 'write_z' at 
/home/toon/compilers/gcc/libgfortran/io/write.c:1296:11,
     inlined from 'formatted_transfer_scalar_write' at 
/home/toon/compilers/gcc/libgfortran/io/transfer.c:2136:4:
/home/toon/compilers/gcc/libgfortran/io/write.c:1222:6: warning: writing 
1 byte into a region of size 0 [-Wstringop-overflow=]
  1222 |   *q = '\0';
       |      ^
/home/toon/compilers/gcc/libgfortran/io/write.c: In function 
'formatted_transfer_scalar_write':
/home/toon/compilers/gcc/libgfortran/io/write.c:1291:8: note: at offset 
[34, 4294967294] into destination object 'itoa_buf' of size 33
  1291 |   char itoa_buf[GFC_XTOA_BUF_SIZE];
       |        ^

which was (of course) not given with a non-lto libgfortran.

The full question of "lto-ing" run time libraries is more complicated 
than just "whether it works" as those who attended the BoF will recall.

Hope this helps,

-- 
Toon Moene - e-mail: toon@moene.org - phone: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands

Re: Test with an lto-build of libgfortran.

[Jeff Law]

On 9/27/23 12:21, Toon Moene wrote:

> 
> The lto-ing of libgfortran did succeed, because I did get a new warning:
> 
> gfortran -O3 -flto -flto-partition=none -static  -o xlintstrfz zchkrfp.o 
> zdrvrfp.o zdrvrf1.o zdrvrf2.o zdrvrf3.o zdrvrf4.o zerrrfp.o zlatb4.o 
> zlaipd.o zlarhs.o zsbmv.o zget04.o zpot01.o zpot03.o zpot02.o chkxer.o 
> xerbla.o alaerh.o aladhd.o alahd.o alasvm.o ../../libtmglib.a 
> ../../liblapack.a ../../librefblas.a
> In function 'xtoa_big',
>      inlined from 'write_z' at 
> /home/toon/compilers/gcc/libgfortran/io/write.c:1296:11,
>      inlined from 'formatted_transfer_scalar_write' at 
> /home/toon/compilers/gcc/libgfortran/io/transfer.c:2136:4:
> /home/toon/compilers/gcc/libgfortran/io/write.c:1222:6: warning: writing 
> 1 byte into a region of size 0 [-Wstringop-overflow=]
>   1222 |   *q = '\0';
>        |      ^
> /home/toon/compilers/gcc/libgfortran/io/write.c: In function 
> 'formatted_transfer_scalar_write':
> /home/toon/compilers/gcc/libgfortran/io/write.c:1291:8: note: at offset 
> [34, 4294967294] into destination object 'itoa_buf' of size 33
>   1291 |   char itoa_buf[GFC_XTOA_BUF_SIZE];
>        |        ^
> 
> which was (of course) not given with a non-lto libgfortran.
Yea.  This certainly can happen with LTO.  These warnings would 
definitely be something worth investigating.

Essentially the inlining enabled by LTO can expose a different set of 
diagnostics.

Jeff

Re: Test with an lto-build of libgfortran.

[Thomas Koenig]

Hi Toon,

> During the GNU Tools Cauldron we discussed (at the BoF: IPA & LTO) the 
> possibility (and hazards) of building the run time libraries for various 
> compilers with -flto, enabling an -flto -static linking of programs with 
> the run time library available during link time optimizations.

This would be a big win for libgfortran, especially the array functions,
knowing that stride==1 can be a _big_ win for optimization.  This is
why LTO is such an excellent idea for Fortran in general, and for the
library in particular.

There is a PR on this with quite some discussion already,
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77278 .

I've put you in CC of that bug, maybe we can discuss there more
in detail.

One point about the array functions: In the library, we use sort of a
ripple carry algorithm to step through the arrays.  This saves space
an is general, but the performance (esp for the most common one-and
two-dimensional arrays) can suffer.

[...]

> The full question of "lto-ing" run time libraries is more complicated 
> than just "whether it works" as those who attended the BoF will recall.

I didn't attend the Cauldron (but that discussion would have been
very interesting).  I think for libgfortran, a first step would be
additional work to get declarations on both sides to agree (which is
worth doing anyway).

Best regards

	Thomas

Re: Test with an lto-build of libgfortran.

[Richard Biener]

On Wed, Sep 27, 2023 at 11:48 PM Jeff Law via Fortran
<fortran@gcc.gnu.org> wrote:
>
>
>
> On 9/27/23 12:21, Toon Moene wrote:
>
> >
> > The lto-ing of libgfortran did succeed, because I did get a new warning:
> >
> > gfortran -O3 -flto -flto-partition=none -static  -o xlintstrfz zchkrfp.o
> > zdrvrfp.o zdrvrf1.o zdrvrf2.o zdrvrf3.o zdrvrf4.o zerrrfp.o zlatb4.o
> > zlaipd.o zlarhs.o zsbmv.o zget04.o zpot01.o zpot03.o zpot02.o chkxer.o
> > xerbla.o alaerh.o aladhd.o alahd.o alasvm.o ../../libtmglib.a
> > ../../liblapack.a ../../librefblas.a
> > In function 'xtoa_big',
> >      inlined from 'write_z' at
> > /home/toon/compilers/gcc/libgfortran/io/write.c:1296:11,
> >      inlined from 'formatted_transfer_scalar_write' at
> > /home/toon/compilers/gcc/libgfortran/io/transfer.c:2136:4:
> > /home/toon/compilers/gcc/libgfortran/io/write.c:1222:6: warning: writing
> > 1 byte into a region of size 0 [-Wstringop-overflow=]
> >   1222 |   *q = '\0';
> >        |      ^
> > /home/toon/compilers/gcc/libgfortran/io/write.c: In function
> > 'formatted_transfer_scalar_write':
> > /home/toon/compilers/gcc/libgfortran/io/write.c:1291:8: note: at offset
> > [34, 4294967294] into destination object 'itoa_buf' of size 33
> >   1291 |   char itoa_buf[GFC_XTOA_BUF_SIZE];
> >        |        ^
> >
> > which was (of course) not given with a non-lto libgfortran.
> Yea.  This certainly can happen with LTO.  These warnings would
> definitely be something worth investigating.
>
> Essentially the inlining enabled by LTO can expose a different set of
> diagnostics.

This particular place in libgfortran has

  /* write_z, which calls xtoa_big, is called from transfer.c,
     formatted_transfer_scalar_write.  There it is passed the kind as
     argument, which means a maximum of 16.  The buffer is large
     enough, but the compiler does not know that, so shut up the
     warning here.  */
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wstringop-overflow"
  *q = '\0';
#pragma GCC diagnostic pop

so obviously the #pragma doesn't survive through LTO.  Somehow I think
this is a known bug, but maybe I misremember (I think we are not streaming
any of the ad-hoc location parts).

Richard.

>
> Jeff

Re: Test with an lto-build of libgfortran.

[Andrew Pinski]

On Wed, Sep 27, 2023 at 11:28 PM Richard Biener via Fortran
<fortran@gcc.gnu.org> wrote:
>
> On Wed, Sep 27, 2023 at 11:48 PM Jeff Law via Fortran
> <fortran@gcc.gnu.org> wrote:
> >
> >
> >
> > On 9/27/23 12:21, Toon Moene wrote:
> >
> > >
> > > The lto-ing of libgfortran did succeed, because I did get a new warning:
> > >
> > > gfortran -O3 -flto -flto-partition=none -static  -o xlintstrfz zchkrfp.o
> > > zdrvrfp.o zdrvrf1.o zdrvrf2.o zdrvrf3.o zdrvrf4.o zerrrfp.o zlatb4.o
> > > zlaipd.o zlarhs.o zsbmv.o zget04.o zpot01.o zpot03.o zpot02.o chkxer.o
> > > xerbla.o alaerh.o aladhd.o alahd.o alasvm.o ../../libtmglib.a
> > > ../../liblapack.a ../../librefblas.a
> > > In function 'xtoa_big',
> > >      inlined from 'write_z' at
> > > /home/toon/compilers/gcc/libgfortran/io/write.c:1296:11,
> > >      inlined from 'formatted_transfer_scalar_write' at
> > > /home/toon/compilers/gcc/libgfortran/io/transfer.c:2136:4:
> > > /home/toon/compilers/gcc/libgfortran/io/write.c:1222:6: warning: writing
> > > 1 byte into a region of size 0 [-Wstringop-overflow=]
> > >   1222 |   *q = '\0';
> > >        |      ^
> > > /home/toon/compilers/gcc/libgfortran/io/write.c: In function
> > > 'formatted_transfer_scalar_write':
> > > /home/toon/compilers/gcc/libgfortran/io/write.c:1291:8: note: at offset
> > > [34, 4294967294] into destination object 'itoa_buf' of size 33
> > >   1291 |   char itoa_buf[GFC_XTOA_BUF_SIZE];
> > >        |        ^
> > >
> > > which was (of course) not given with a non-lto libgfortran.
> > Yea.  This certainly can happen with LTO.  These warnings would
> > definitely be something worth investigating.
> >
> > Essentially the inlining enabled by LTO can expose a different set of
> > diagnostics.
>
> This particular place in libgfortran has
>
>   /* write_z, which calls xtoa_big, is called from transfer.c,
>      formatted_transfer_scalar_write.  There it is passed the kind as
>      argument, which means a maximum of 16.  The buffer is large
>      enough, but the compiler does not know that, so shut up the
>      warning here.  */
> #pragma GCC diagnostic push
> #pragma GCC diagnostic ignored "-Wstringop-overflow"
>   *q = '\0';
> #pragma GCC diagnostic pop
>
> so obviously the #pragma doesn't survive through LTO.  Somehow I think
> this is a known bug, but maybe I misremember (I think we are not streaming
> any of the ad-hoc location parts).

Yes it is a known bug.
See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80922 .

Thanks,
Andrew


>
> Richard.
>
> >
> > Jeff

Re: Test with an lto-build of libgfortran.

[Tobias Burnus]

[-- Attachment #1: Type: text/plain, Size: 1399 bytes --]

Hi all,

the following works for me. I have only tried a normal build (where it
does silence the same warning) and not an LTO build and I just believed
the comment - see attached patch. Comments?

On 28.09.23 08:25, Richard Biener via Fortran wrote:

> This particular place in libgfortran has
>
>    /* write_z, which calls xtoa_big, is called from transfer.c,
>       formatted_transfer_scalar_write.  There it is passed the kind as
>       argument, which means a maximum of 16.  The buffer is large
>       enough, but the compiler does not know that, so shut up the
>       warning here.  */
> #pragma GCC diagnostic push
> #pragma GCC diagnostic ignored "-Wstringop-overflow"
>    *q = '\0';
> #pragma GCC diagnostic pop
>
> so obviously the #pragma doesn't survive through LTO.  Somehow I think
> this is a known bug, but maybe I misremember (I think we are not streaming
> any of the ad-hoc location parts).

I have replaced it now by the assert that "len <= 16", i.e.

+  if (len > 16)
+    __builtin_unreachable ();

Build + tested on x86-64-gnu-linux
Comment? OK for mainline?

Tobias
-----------------
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955

[-- Attachment #2: xtoa_big-silence-warning.diff --]
[-- Type: text/x-patch, Size: 1583 bytes --]

libgfortran: Use __builtin_unreachable() not -Wno-stringop-overflow to silence warning

libgfortran/
	* io/write.c (xtoa_big): Change a 'GCC diagnostic ignored
	"-Wstringop-overflow"' to an assumption (via __builtin_unreachable).

 libgfortran/io/write.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/libgfortran/io/write.c b/libgfortran/io/write.c
index 5d47a6d25f7..00c8fd2e288 100644
--- a/libgfortran/io/write.c
+++ b/libgfortran/io/write.c
@@ -1179,6 +1179,15 @@ xtoa_big (const char *s, char *buffer, int len, GFC_UINTEGER_LARGEST *n)
   uint8_t h, l;
   int i;
 
+  /* write_z, which calls xtoa_big, is called from transfer.c,
+     formatted_transfer_scalar_write.  There it is passed the kind as
+     'len' argument, which means a maximum of 16.  The buffer is large
+     enough, but the compiler does not know that, so shut up the
+     warning here.  */
+
+  if (len > 16)
+    __builtin_unreachable ();
+
   q = buffer;
 
   if (__BYTE_ORDER__ == __ORDER_BIG_ENDIAN__)
@@ -1212,15 +1221,7 @@ xtoa_big (const char *s, char *buffer, int len, GFC_UINTEGER_LARGEST *n)
 	}
     }
 
-  /* write_z, which calls xtoa_big, is called from transfer.c,
-     formatted_transfer_scalar_write.  There it is passed the kind as
-     argument, which means a maximum of 16.  The buffer is large
-     enough, but the compiler does not know that, so shut up the
-     warning here.  */
-#pragma GCC diagnostic push
-#pragma GCC diagnostic ignored "-Wstringop-overflow"
   *q = '\0';
-#pragma GCC diagnostic pop
 
   if (*n == 0)
     return "0";

Re: Test with an lto-build of libgfortran.

[Jakub Jelinek]

On Thu, Sep 28, 2023 at 09:29:02AM +0200, Tobias Burnus wrote:
> the following works for me. I have only tried a normal build (where it
> does silence the same warning) and not an LTO build and I just believed
> the comment - see attached patch. Comments?
> 
> On 28.09.23 08:25, Richard Biener via Fortran wrote:
> 
> > This particular place in libgfortran has
> > 
> >    /* write_z, which calls xtoa_big, is called from transfer.c,
> >       formatted_transfer_scalar_write.  There it is passed the kind as
> >       argument, which means a maximum of 16.  The buffer is large
> >       enough, but the compiler does not know that, so shut up the
> >       warning here.  */
> > #pragma GCC diagnostic push
> > #pragma GCC diagnostic ignored "-Wstringop-overflow"
> >    *q = '\0';
> > #pragma GCC diagnostic pop
> > 
> > so obviously the #pragma doesn't survive through LTO.  Somehow I think
> > this is a known bug, but maybe I misremember (I think we are not streaming
> > any of the ad-hoc location parts).
> 
> I have replaced it now by the assert that "len <= 16", i.e.
> 
> +  if (len > 16)
> +    __builtin_unreachable ();
> 
> Build + tested on x86-64-gnu-linux
> Comment? OK for mainline?

Is it just that in correct programs len can't be > 16, or that it is really
impossible for it being > 16?  I mean, we have that artificial kind 17 for
powerpc which better should be turned into length of 16, but isn't e.g.
_gfortran_transfer_integer etc. just called with a kind argument?  Does
anything error earlier if it is larger?  I mean, say user calling
_gfortan_transfer_integer by hand with kind 1024?

Sure, we could still say it is UB to do that kind of thing and
__builtin_unreachable () would be a way to turn that UB into manifestly
reproducable UB.

	Jakub

Re: Test with an lto-build of libgfortran.

[Tobias Burnus]

(replace gcc@ by gcc-patches@; see
https://gcc.gnu.org/pipermail/gcc/2023-September/242591.html
and other emails in that thread)

On 28.09.23 11:51, Jakub Jelinek wrote:
> On Thu, Sep 28, 2023 at 09:29:02AM +0200, Tobias Burnus wrote:
>> On 28.09.23 08:25, Richard Biener via Fortran wrote:
>>
>>> This particular place in libgfortran has
>>>
>>>     /* write_z, which calls xtoa_big, is called from transfer.c,
>>>        formatted_transfer_scalar_write.  There it is passed the kind as
>>>        argument, which means a maximum of 16.  The buffer is large
>>>        enough, but the compiler does not know that, so shut up the
>>>        warning here.  */
...
>> I have replaced it now by the assert that "len <= 16", i.e.
>> +  if (len > 16)
>> +    __builtin_unreachable ();
> Is it just that in correct programs len can't be > 16, or that it is really
> impossible for it being > 16?  I mean, we have that artificial kind 17 for
> powerpc which better should be turned into length of 16, but isn't e.g.
> _gfortran_transfer_integer etc.

My understanding is that kind=17 only pops up on PowerPC
for REAL variables as they represent __float128 in multiple ways.

Having said that, the current call tree is:

* xtoa_big: that's where the warning suppression
   was replaced by the unreachable.

* Only caller is 'write_z' with calls it by passing its
   last argument ('len') as last argument ('len')

* "internal_proto(write_z)" implies that it is not called from
   outside libgfortran. The internal only caller is:

*  formatted_transfer_scalar_write, which calls it as:

         case FMT_Z:
           ...
#ifdef HAVE_GFC_REAL_17
           if (type == BT_REAL && kind == 17)
             kind = 16;
#endif
           write_z (dtp, f, p, kind);

I am not aware of any logigal/integer/real(+comples)/character kind > 16,
except for this PPC one. And complex numbers are pairs of BT_REAL.

Thus, I think that patch should be fine - except:

> Does anything error earlier if it is larger?  I mean, say user calling
> _gfortan_transfer_integer by hand with kind 1024?

I think this will fail. We have various ways to deal with this in libgfortran;
I see some cases where the switch "default:" sets the length to 0; we have
other places where we use an "assert", I think we have other places were
we run into UB.

Thus, one option would be to either 'assert(len <= 16)' or
'assert((size_t)len < GFC_OTOA_BUF_SIZE - 1)' instead.

Or we could handle it as len=0 and silently ignore the output or ...

I am fine with either of the many options - except that I like something
explicit involving 'len' and a comparison (unreachable, assert, regarding as len = 0)
better than the existing warning suppression which is too indirect for
me. (Besides: it does not work for LTO.) Preferences? Tobias

-----------------
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955

Re: Test with an lto-build of libgfortran.

[Jakub Jelinek]

On Thu, Sep 28, 2023 at 01:00:41PM +0200, Tobias Burnus wrote:
> I am not aware of any logigal/integer/real(+comples)/character kind > 16,
> except for this PPC one. And complex numbers are pairs of BT_REAL.
> 
> Thus, I think that patch should be fine - except:
> 
> > Does anything error earlier if it is larger?  I mean, say user calling
> > _gfortan_transfer_integer by hand with kind 1024?
> 
> I think this will fail. We have various ways to deal with this in libgfortran;
> I see some cases where the switch "default:" sets the length to 0; we have
> other places where we use an "assert", I think we have other places were
> we run into UB.
> 
> Thus, one option would be to either 'assert(len <= 16)' or
> 'assert((size_t)len < GFC_OTOA_BUF_SIZE - 1)' instead.
> 
> Or we could handle it as len=0 and silently ignore the output or ...
> 
> I am fine with either of the many options - except that I like something
> explicit involving 'len' and a comparison (unreachable, assert, regarding as len = 0)
> better than the existing warning suppression which is too indirect for
> me. (Besides: it does not work for LTO.) Preferences? Tobias

Let's go with the __builtin_unreachable, ok for trunk.

	Jakub

Re: Test with an lto-build of libgfortran.

[Toon Moene]

On 9/28/23 07:33, Thomas Koenig wrote:

> Hi Toon,

[ I wrote: ]

>> The full question of "lto-ing" run time libraries is more complicated 
>> than just "whether it works" as those who attended the BoF will recall.
> 
> I didn't attend the Cauldron (but that discussion would have been
> very interesting).  I think for libgfortran, a first step would be
> additional work to get declarations on both sides to agree (which is
> worth doing anyway).
> 
> Best regards
> 
>      Thomas

The big problem in *distributing* GCC (i.e., the collection) with lto'd 
run-time libraries is that the format of the lto structure changes with 
releases. If a compiler (by accident) picks up a run time library with 
non-matching lto objects, it might crash (or "introduce subtle errors in 
a once working program").

I.e., like the problem the gfortran community had with the changing 
format of our .mod files.

But it would be a big win for Fortran ...

Kind regards,

-- 
Toon Moene - e-mail: toon@moene.org - phone: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands

Re: Test with an lto-build of libgfortran.

[Jakub Jelinek]

On Thu, Sep 28, 2023 at 09:03:39PM +0200, Toon Moene wrote:
> > > The full question of "lto-ing" run time libraries is more
> > > complicated than just "whether it works" as those who attended the
> > > BoF will recall.
> > 
> > I didn't attend the Cauldron (but that discussion would have been
> > very interesting).  I think for libgfortran, a first step would be
> > additional work to get declarations on both sides to agree (which is
> > worth doing anyway).
> > 
> > Best regards
> > 
> >      Thomas
> 
> The big problem in *distributing* GCC (i.e., the collection) with lto'd
> run-time libraries is that the format of the lto structure changes with
> releases. If a compiler (by accident) picks up a run time library with
> non-matching lto objects, it might crash (or "introduce subtle errors in a
> once working program").

It is worse than that, usually the LTO format changes e.g. any time any
option or parameter is added on a release branch (several times a year) and
at other times as well.
Though, admittedly GCC is the single package that actually could get away
with LTO in lib*.a libraries, at least in some packagings (if the static
libraries are in gcc specific subdirectories rather than say /usr/lib{,64}
or similar and if the packaging of gcc updates both the compiler and
corresponding static libraries in a lock-step.  Because in that case LTO
in there will be always used only by the same snapshot from the release
branch and so should be compatible with the LTO in it.

	Jakub

Re: Test with an lto-build of libgfortran.

[Toon Moene]

On 9/28/23 21:26, Jakub Jelinek wrote:

> It is worse than that, usually the LTO format changes e.g. any time any
> option or parameter is added on a release branch (several times a year) and
> at other times as well.
> Though, admittedly GCC is the single package that actually could get away
> with LTO in lib*.a libraries, at least in some packagings (if the static
> libraries are in gcc specific subdirectories rather than say /usr/lib{,64}
> or similar and if the packaging of gcc updates both the compiler and
> corresponding static libraries in a lock-step.  Because in that case LTO
> in there will be always used only by the same snapshot from the release
> branch and so should be compatible with the LTO in it.
This might be an argument to make it a configure option, e.g. 
--enable-lto-runtime.

Kind regards,

-- 
Toon Moene - e-mail: toon@moene.org - phone: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands

Re: Test with an lto-build of libgfortran.

[David Edelsohn]

[-- Attachment #1: Type: text/plain, Size: 1025 bytes --]

On Thu, Sep 28, 2023 at 4:00 PM Toon Moene <toon@moene.org> wrote:

> On 9/28/23 21:26, Jakub Jelinek wrote:
>
> > It is worse than that, usually the LTO format changes e.g. any time any
> > option or parameter is added on a release branch (several times a year)
> and
> > at other times as well.
> > Though, admittedly GCC is the single package that actually could get away
> > with LTO in lib*.a libraries, at least in some packagings (if the static
> > libraries are in gcc specific subdirectories rather than say
> /usr/lib{,64}
> > or similar and if the packaging of gcc updates both the compiler and
> > corresponding static libraries in a lock-step.  Because in that case LTO
> > in there will be always used only by the same snapshot from the release
> > branch and so should be compatible with the LTO in it.
> This might be an argument to make it a configure option, e.g.
> --enable-lto-runtime.
>
> Note that not all targets support LTO, so the option cannot be added
unilaterally.

David

Re: Test with an lto-build of libgfortran.

[Thomas Koenig]

Hi Jakub,

> It is worse than that, usually the LTO format changes e.g. any time any
> option or parameter is added on a release branch (several times a year) and
> at other times as well.

Hm, that makes LTO not very well suited for libraries...

> Though, admittedly GCC is the single package that actually could get away
> with LTO in lib*.a libraries, at least in some packagings (if the static
> libraries are in gcc specific subdirectories rather than say /usr/lib{,64}
> or similar and if the packaging of gcc updates both the compiler and
> corresponding static libraries in a lock-step.  Because in that case LTO
> in there will be always used only by the same snapshot from the release
> branch and so should be compatible with the LTO in it.

Maybe another approach: Instead of storing version-dependent LTO code in
the *.a files, we could just store preprocessed C code there, somewhat
shortened by removing whitespace, comments and unused declarations
(and/or store them in compressed format).

This would also allow the something like sanitizers to look into the
runtime libraries, and other instrumentation.

It would also be a much more extensive project, also involving
modifications to the linker. Not sure how realistic that would be.

Best regards

	Thomas

Re: Test with an lto-build of libgfortran.

[Andrew Stubbs]

On 28/09/2023 20:59, Toon Moene wrote:
> On 9/28/23 21:26, Jakub Jelinek wrote:
> 
>> It is worse than that, usually the LTO format changes e.g. any time any
>> option or parameter is added on a release branch (several times a 
>> year) and
>> at other times as well.
>> Though, admittedly GCC is the single package that actually could get away
>> with LTO in lib*.a libraries, at least in some packagings (if the static
>> libraries are in gcc specific subdirectories rather than say 
>> /usr/lib{,64}
>> or similar and if the packaging of gcc updates both the compiler and
>> corresponding static libraries in a lock-step.  Because in that case LTO
>> in there will be always used only by the same snapshot from the release
>> branch and so should be compatible with the LTO in it.
> This might be an argument to make it a configure option, e.g. 
> --enable-lto-runtime.

This sort of thing should definitely Just Work for cross compilers and 
embedded platforms where the libraries are bundled with the compiler.

Andrew