[PATCH,RFC] collect2 LTO for AIX

[PATCH,RFC] collect2 LTO for AIX

[David Edelsohn]

[-- Attachment #1: Type: text/plain, Size: 722 bytes --]

The attached patch is an incremental step toward GCC LTO on AIX.  The
recent Libiberty Simple Object improvements for XCOFF provide more
capabilities for operations on XCOFF object files, which are a
prerequisite for GCC LTO functionality.

This patch adds the basic LTO scanning pass to the COFF support in
collect2.  I don't believe that this change should affect other COFF
targets adversely (do they even use collect2?), but I wanted to give
people an opportunity to comment.

(Yes, the formatting is messy, but all of collect2.c is messy.)

Bootstrapped on powerpc-ibm-aix7.2.0.0

Thanks, David

* collect2.c (add_lto_object): Compile for OBJECT_COFF.
(scan_prog_file): Don't skip PASS_LTOINFO. Scan for LTO objects.

[-- Attachment #2: collect2-aix-lto.txt --]
[-- Type: text/plain, Size: 1886 bytes --]

Index: collect2.c
===================================================================
--- collect2.c	(revision 253736)
+++ collect2.c	(working copy)
@@ -614,7 +614,7 @@ static const char *const target_machine = TARGET_M
 
    Return 0 if not found, otherwise return its name, allocated with malloc.  */
 
-#ifdef OBJECT_FORMAT_NONE
+#if defined (OBJECT_FORMAT_NONE) || defined (OBJECT_FORMAT_COFF)
 
 /* Add an entry for the object file NAME to object file list LIST.
    New entries are added at the end of the list. The original pointer
@@ -634,7 +634,7 @@ add_lto_object (struct lto_object_list *list, cons
 
   list->last = n;
 }
-#endif /* OBJECT_FORMAT_NONE */
+#endif
 
 
 /* Perform a link-time recompilation and relink if any of the object
@@ -2750,8 +2750,10 @@ scan_prog_file (const char *prog_name, scanpass wh
   LDFILE *ldptr = NULL;
   int sym_index, sym_count;
   int is_shared = 0;
+  int found_lto = 0;
 
-  if (which_pass != PASS_FIRST && which_pass != PASS_OBJ)
+  if (which_pass != PASS_FIRST && which_pass != PASS_OBJ
+      && which_pass != PASS_LTOINFO)
     return;
 
 #ifdef COLLECT_EXPORT_LIST
@@ -2764,6 +2766,7 @@ scan_prog_file (const char *prog_name, scanpass wh
      eliminate scan_libraries() function.  */
   do
     {
+      found_lto = 0;
 #endif
       /* Some platforms (e.g. OSF4) declare ldopen as taking a
 	 non-const char * filename parameter, even though it will not
@@ -2806,6 +2809,19 @@ scan_prog_file (const char *prog_name, scanpass wh
 			++name;
 #endif
 
+                      if (which_pass == PASS_LTOINFO)
+                        {
+			  if (found_lto)
+			    continue;
+			  if (strncmp (name, "__gnu_lto_v1", 12) == 0)
+			    {
+			      add_lto_object (&lto_objects, prog_name);
+			      found_lto = 1;
+			      break;
+			    }
+			  continue;
+			}
+
 		      switch (is_ctor_dtor (name))
 			{
 #if TARGET_AIX_VERSION

Re: [PATCH,RFC] collect2 LTO for AIX

[Jeff Law]

On 10/13/2017 12:04 PM, David Edelsohn wrote:
> The attached patch is an incremental step toward GCC LTO on AIX.  The
> recent Libiberty Simple Object improvements for XCOFF provide more
> capabilities for operations on XCOFF object files, which are a
> prerequisite for GCC LTO functionality.
> 
> This patch adds the basic LTO scanning pass to the COFF support in
> collect2.  I don't believe that this change should affect other COFF
> targets adversely (do they even use collect2?), but I wanted to give
> people an opportunity to comment.
I honestly can't remember what the COFF targets do anymore :-)  Didn't
we delete them all a while back?  I see that the generic COFF targets
died back in gcc-4.4.


Jeff

Re: [PATCH,RFC] collect2 LTO for AIX

[David Edelsohn]

On Mon, Oct 30, 2017 at 3:13 PM, Jeff Law <law@redhat.com> wrote:
> On 10/13/2017 12:04 PM, David Edelsohn wrote:
>> The attached patch is an incremental step toward GCC LTO on AIX.  The
>> recent Libiberty Simple Object improvements for XCOFF provide more
>> capabilities for operations on XCOFF object files, which are a
>> prerequisite for GCC LTO functionality.
>>
>> This patch adds the basic LTO scanning pass to the COFF support in
>> collect2.  I don't believe that this change should affect other COFF
>> targets adversely (do they even use collect2?), but I wanted to give
>> people an opportunity to comment.
> I honestly can't remember what the COFF targets do anymore :-)  Didn't
> we delete them all a while back?  I see that the generic COFF targets
> died back in gcc-4.4.

Jim,

With your recent removal of SDB and -gcoff support, I would appreciate
your advice about my patch to incrementally add some preliminary LTO
support for AIX to collect2.c:

https://gcc.gnu.org/ml/gcc-patches/2017-10/msg00893.html

There don't seem to be any other COFF targets around that still use
the collect2 code to manually scan object and archive files. And I
don't think that they would be hurt by the additional search for the
LTO symbol.  I want to check if anyone is aware of a potential problem
before I commit the code.

Thanks, David

Re: [PATCH,RFC] collect2 LTO for AIX

[Jim Wilson]

On 10/30/2017 12:13 PM, Jeff Law wrote:
> On 10/13/2017 12:04 PM, David Edelsohn wrote:
>> This patch adds the basic LTO scanning pass to the COFF support in
>> collect2.  I don't believe that this change should affect other COFF
>> targets adversely (do they even use collect2?), but I wanted to give
>> people an opportunity to comment.
> I honestly can't remember what the COFF targets do anymore :-)  Didn't
> we delete them all a while back?  I see that the generic COFF targets
> died back in gcc-4.4.

We still have PE-COFF for Windows and XCOFF for AIX.  The SVR3 Unix 
COFF, embedded COFF, and ECOFF (MIPS/Alpha) support is gone.  Hopefully 
Microsoft and IBM will see the light someday and switch to ELF.

Jim

Re: [PATCH,RFC] collect2 LTO for AIX

[Jim Wilson]

On 10/31/2017 12:11 PM, David Edelsohn wrote:
> With your recent removal of SDB and -gcoff support, I would appreciate
> your advice about my patch to incrementally add some preliminary LTO
> support for AIX to collect2.c:

OK.  I can take a look.  I started a new job this week, so I'm a bit 
disorganized until I get my new work environment sorted out.

Jim

Re: [PATCH,RFC] collect2 LTO for AIX

[Jim Wilson]

On 10/31/2017 12:11 PM, David Edelsohn wrote:
> With your recent removal of SDB and -gcoff support, I would appreciate
> your advice about my patch to incrementally add some preliminary LTO
> support for AIX to collect2.c:
> 
> https://gcc.gnu.org/ml/gcc-patches/2017-10/msg00893.html
> 
> There don't seem to be any other COFF targets around that still use
> the collect2 code to manually scan object and archive files. And I
> don't think that they would be hurt by the additional search for the
> LTO symbol.  I want to check if anyone is aware of a potential problem
> before I commit the code.

The patch is OK.

cygwin uses collect2, but it is using the generic support, not the COFF 
specific support.

The generic support has a check for to verify that we have an object 
file and not an archive file.  See maybe_lto_object_file.  You might 
want to add equivalent code to the COFF support.

Jim

Re: [PATCH,RFC] collect2 LTO for AIX

[Jim Wilson]

On 10/31/2017 12:11 PM, David Edelsohn wrote:
> With your recent removal of SDB and -gcoff support, I would appreciate
> your advice about my patch to incrementally add some preliminary LTO
> support for AIX to collect2.c:
> 
> https://gcc.gnu.org/ml/gcc-patches/2017-10/msg00893.html
> 
> There don't seem to be any other COFF targets around that still use
> the collect2 code to manually scan object and archive files. And I
> don't think that they would be hurt by the additional search for the
> LTO symbol.  I want to check if anyone is aware of a potential problem
> before I commit the code.

The patch is OK.

cygwin uses collect2, but it is using the generic support, not the COFF 
specific support.

The generic support has a check for to verify that we have an object 
file and not an archive file.  See maybe_lto_object_file.  You might 
want to add equivalent code to the COFF support.

Jim

Re: [PATCH,RFC] collect2 LTO for AIX

[David Edelsohn]

On Mon, Nov 6, 2017 at 11:01 AM, Jim Wilson <jimw@sifive.com> wrote:
> On 10/31/2017 12:11 PM, David Edelsohn wrote:
>>
>> With your recent removal of SDB and -gcoff support, I would appreciate
>> your advice about my patch to incrementally add some preliminary LTO
>> support for AIX to collect2.c:
>>
>> https://gcc.gnu.org/ml/gcc-patches/2017-10/msg00893.html
>>
>> There don't seem to be any other COFF targets around that still use
>> the collect2 code to manually scan object and archive files. And I
>> don't think that they would be hurt by the additional search for the
>> LTO symbol.  I want to check if anyone is aware of a potential problem
>> before I commit the code.
>
>
> The patch is OK.
>
> cygwin uses collect2, but it is using the generic support, not the COFF
> specific support.
>
> The generic support has a check for to verify that we have an object file
> and not an archive file.  See maybe_lto_object_file.  You might want to add
> equivalent code to the COFF support.

On AIX, archive files can be processed as a series of object files and
collect2.c:scan_prog_file() explicitly utilizes this. LTO object files
can be in archives.

Thanks for checking the patch.

- David

Re: [PATCH,RFC] collect2 LTO for AIX

[Martin Liška]

Hi.

Recently I've introduced a new .gnu.lto_.lto section that
is supposed to provide meta information about a LTO bytecode.

As a further step, I'm planning to teach binutils about
existence of the section and I'll remove in the future
emission of __gnu_lto_slim and __gnu_lto_v1 symbols.
The former one is used by binutils to identify if
an object is a slim LTO object. The later one is currently
used only in gcc/collect2.c and was added by David's patch.

My question is: Can we remove __gnu_lto_v1 right now and
XCOFF will use something similar to ELF (has_lto_section)?
Can you David help me with that please?

Thank you,
Martin

Re: [PATCH,RFC] collect2 LTO for AIX

[David Edelsohn]

On Thu, Jul 4, 2019 at 10:38 AM Martin Liška <mliska@suse.cz> wrote:
>
> Hi.
>
> Recently I've introduced a new .gnu.lto_.lto section that
> is supposed to provide meta information about a LTO bytecode.
>
> As a further step, I'm planning to teach binutils about
> existence of the section and I'll remove in the future
> emission of __gnu_lto_slim and __gnu_lto_v1 symbols.
> The former one is used by binutils to identify if
> an object is a slim LTO object. The later one is currently
> used only in gcc/collect2.c and was added by David's patch.
>
> My question is: Can we remove __gnu_lto_v1 right now and
> XCOFF will use something similar to ELF (has_lto_section)?
> Can you David help me with that please?

LTO currently does not work on AIX. I added the __gnu_lto_v1 as a
test. You can rip it out and XCOFF will follow a different path when
implementing LTO.

Thanks David

Re: [PATCH,RFC] collect2 LTO for AIX

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 1046 bytes --]

On 7/4/19 5:03 PM, David Edelsohn wrote:
> On Thu, Jul 4, 2019 at 10:38 AM Martin Liška <mliska@suse.cz> wrote:
>>
>> Hi.
>>
>> Recently I've introduced a new .gnu.lto_.lto section that
>> is supposed to provide meta information about a LTO bytecode.
>>
>> As a further step, I'm planning to teach binutils about
>> existence of the section and I'll remove in the future
>> emission of __gnu_lto_slim and __gnu_lto_v1 symbols.
>> The former one is used by binutils to identify if
>> an object is a slim LTO object. The later one is currently
>> used only in gcc/collect2.c and was added by David's patch.
>>
>> My question is: Can we remove __gnu_lto_v1 right now and
>> XCOFF will use something similar to ELF (has_lto_section)?
>> Can you David help me with that please?
> 
> LTO currently does not work on AIX. I added the __gnu_lto_v1 as a
> test. You can rip it out and XCOFF will follow a different path when
> implementing LTO.

Great. Then I'm sending revert of the patch.

Ready to be installed?
Thanks,
Martin

> 
> Thanks David
> 


[-- Attachment #2: 0001-Revert-r254460-collect2-LTO-for-AIX.patch --]
[-- Type: text/x-patch, Size: 2377 bytes --]

From d34a25ca594e9b8c41af56b812faf2f8cdafd010 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Thu, 4 Jul 2019 17:38:33 +0200
Subject: [PATCH] Revert r254460 (collect2 LTO for AIX).

gcc/ChangeLog:

2019-07-04  Martin Liska  <mliska@suse.cz>

	* collect2.c (defined): Revert to before r254460.
	(scan_prog_file): Revert to before r254460.
---
 gcc/collect2.c | 22 +++-------------------
 1 file changed, 3 insertions(+), 19 deletions(-)

diff --git a/gcc/collect2.c b/gcc/collect2.c
index d6a7355e218..e25e33962fb 100644
--- a/gcc/collect2.c
+++ b/gcc/collect2.c
@@ -616,7 +616,7 @@ static const char *const target_machine = TARGET_MACHINE;
 
    Return 0 if not found, otherwise return its name, allocated with malloc.  */
 
-#if defined (OBJECT_FORMAT_NONE) || defined (OBJECT_FORMAT_COFF)
+#ifdef OBJECT_FORMAT_NONE
 
 /* Add an entry for the object file NAME to object file list LIST.
    New entries are added at the end of the list. The original pointer
@@ -636,7 +636,7 @@ add_lto_object (struct lto_object_list *list, const char *name)
 
   list->last = n;
 }
-#endif
+#endif /* OBJECT_FORMAT_NONE */
 
 
 /* Perform a link-time recompilation and relink if any of the object
@@ -2799,10 +2799,8 @@ scan_prog_file (const char *prog_name, scanpass which_pass,
   LDFILE *ldptr = NULL;
   int sym_index, sym_count;
   int is_shared = 0;
-  int found_lto = 0;
 
-  if (which_pass != PASS_FIRST && which_pass != PASS_OBJ
-      && which_pass != PASS_LTOINFO)
+  if (which_pass != PASS_FIRST && which_pass != PASS_OBJ)
     return;
 
 #ifdef COLLECT_EXPORT_LIST
@@ -2815,7 +2813,6 @@ scan_prog_file (const char *prog_name, scanpass which_pass,
      eliminate scan_libraries() function.  */
   do
     {
-      found_lto = 0;
 #endif
       /* Some platforms (e.g. OSF4) declare ldopen as taking a
 	 non-const char * filename parameter, even though it will not
@@ -2858,19 +2855,6 @@ scan_prog_file (const char *prog_name, scanpass which_pass,
 			++name;
 #endif
 
-                      if (which_pass == PASS_LTOINFO)
-                        {
-			  if (found_lto)
-			    continue;
-			  if (strncmp (name, "__gnu_lto_v1", 12) == 0)
-			    {
-			      add_lto_object (&lto_objects, prog_name);
-			      found_lto = 1;
-			      break;
-			    }
-			  continue;
-			}
-
 		      switch (is_ctor_dtor (name))
 			{
 #if TARGET_AIX_VERSION
-- 
2.22.0

Re: [PATCH,RFC] collect2 LTO for AIX

[David Edelsohn]

On Thu, Jul 4, 2019 at 11:43 AM Martin Liška <mliska@suse.cz> wrote:
>
> On 7/4/19 5:03 PM, David Edelsohn wrote:
> > On Thu, Jul 4, 2019 at 10:38 AM Martin Liška <mliska@suse.cz> wrote:
> >>
> >> Hi.
> >>
> >> Recently I've introduced a new .gnu.lto_.lto section that
> >> is supposed to provide meta information about a LTO bytecode.
> >>
> >> As a further step, I'm planning to teach binutils about
> >> existence of the section and I'll remove in the future
> >> emission of __gnu_lto_slim and __gnu_lto_v1 symbols.
> >> The former one is used by binutils to identify if
> >> an object is a slim LTO object. The later one is currently
> >> used only in gcc/collect2.c and was added by David's patch.
> >>
> >> My question is: Can we remove __gnu_lto_v1 right now and
> >> XCOFF will use something similar to ELF (has_lto_section)?
> >> Can you David help me with that please?
> >
> > LTO currently does not work on AIX. I added the __gnu_lto_v1 as a
> > test. You can rip it out and XCOFF will follow a different path when
> > implementing LTO.
>
> Great. Then I'm sending revert of the patch.
>
> Ready to be installed?

Okay.

Thanks, David