[GSOC] few question about Bypass assembler when generating LTO object files

[GSOC] few question about Bypass assembler when generating LTO object files

[Rishi Raj]

[-- Attachment #1: Type: text/plain, Size: 958 bytes --]

Hii Everyone,
I had already expressed my interest in the " Bypass assembler when
generating LTO object files" project and making a proposal for the same. I
know I should have done it earlier but I was admitted to the hospital for
past few days :(.
I have a few doubts.
1)

"One problem is that the object files produced by libiberty/simple-object.c
(which is the low-level API used by the LTO code)
are missing some information (such as the architecture info and symbol
table) and API of the simple object will need to be extended to handle
that" I found this in the previous mailing list discussion. So who
output this information currently in the object file, is it assembler?

Also in the current patch for this project by Jan Hubica, from where
are we getting these information from? Is it from crtbegin.o?

2)
"Support in driver to properly execute *1 binary." I found this on Jan
original patch's email. what does it mean

exactly?

Regards

Rishi Raj

Re: [GSOC] few question about Bypass assembler when generating LTO object files

[Rishi Raj]

[-- Attachment #1: Type: text/plain, Size: 2120 bytes --]

While going through the patch and simple-object.c I understood that the
file simple-object.c is used to handle the object file format. However,
this file does not contain all the architecture information required for
LTO object files, so the workaround used in the patch is to read the
crtbegin.o file and merge the missing attributes. While this workaround is
functional, it is not optimal, and the ideal solution would be to extend
simple-object.c to include the missing information.

Regarding the phrase "Support in the driver to properly execute *1 binary",
it is not entirely clear what it refers to. My interpretation is that the
compiler driver (the program that coordinates the compilation process)
needs to be modified to correctly output LTO object files instead of
assembler files (the current approach involves passing the -S and -o
<obj_file_name>.o options) and also skip the assembler option while using
-fbypass-asm option but I am not sure. Can Jan or Martin please shed some
light on this?

Thanks & Regards

Rishi Raj

On Sun, 2 Apr 2023 at 03:05, Rishi Raj <rishiraj45035@gmail.com> wrote:

> Hii Everyone,
> I had already expressed my interest in the " Bypass assembler when
> generating LTO object files" project and making a proposal for the same. I
> know I should have done it earlier but I was admitted to the hospital for
> past few days :(.
> I have a few doubts.
> 1)
>
> "One problem is that the object files produced by libiberty/simple-object.c
> (which is the low-level API used by the LTO code)
> are missing some information (such as the architecture info and symbol
> table) and API of the simple object will need to be extended to handle
> that" I found this in the previous mailing list discussion. So who output this information currently in the object file, is it assembler?
>
> Also in the current patch for this project by Jan Hubica, from where are we getting these information from? Is it from crtbegin.o?
>
> 2)
> "Support in driver to properly execute *1 binary." I found this on Jan original patch's email. what does it mean
>
> exactly?
>
> Regards
>
> Rishi Raj
>
>
>
>

Re: [GSOC] few question about Bypass assembler when generating LTO object files

[Jan Hubicka]

Hello,
> While going through the patch and simple-object.c I understood that the
> file simple-object.c is used to handle the object file format. However,
> this file does not contain all the architecture information required for
> LTO object files, so the workaround used in the patch is to read the
> crtbegin.o file and merge the missing attributes. While this workaround is
> functional, it is not optimal, and the ideal solution would be to extend
> simple-object.c to include the missing information.

Yes, simple-object.c simply uses architecture settings it read earlier
which is problem since at compile time we do not read any object files,
just parse sources). In my original patch the architecture flags were
simply left blank.  I am not sure if there is a version reading
crtbeing.o which would probably not a be that bad workaround, at least
for the start.  Having a way to specify this from the machine descriptions
would be better.

Besides the architecture bits, for simple-object files to work we need
to add the symbol table. For practically useful information we also need
to stream the debug info.
> 
> Regarding the phrase "Support in the driver to properly execute *1 binary",
> it is not entirely clear what it refers to. My interpretation is that the
> compiler driver (the program that coordinates the compilation process)
> needs to be modified to correctly output LTO object files instead of
> assembler files (the current approach involves passing the -S and -o
> <obj_file_name>.o options) and also skip the assembler option while using
> -fbypass-asm option but I am not sure. Can Jan or Martin please shed some
> light on this?
Yes, compiler drivers decides what to do and it needs to know that with
-flto it does not need to produce assembly file and then invoke gas.  If
we go the way of reading in crtbegin.o it will also need to pass correct
crtbegin to *1 binary.  This is generally not that hard to do, just
needs to be done :)

Honza
> 
> Thanks & Regards
> 
> Rishi Raj
> 
> On Sun, 2 Apr 2023 at 03:05, Rishi Raj <rishiraj45035@gmail.com> wrote:
> 
> > Hii Everyone,
> > I had already expressed my interest in the " Bypass assembler when
> > generating LTO object files" project and making a proposal for the same. I
> > know I should have done it earlier but I was admitted to the hospital for
> > past few days :(.
> > I have a few doubts.
> > 1)
> >
> > "One problem is that the object files produced by libiberty/simple-object.c
> > (which is the low-level API used by the LTO code)
> > are missing some information (such as the architecture info and symbol
> > table) and API of the simple object will need to be extended to handle
> > that" I found this in the previous mailing list discussion. So who output this information currently in the object file, is it assembler?
> >
> > Also in the current patch for this project by Jan Hubica, from where are we getting these information from? Is it from crtbegin.o?
> >
> > 2)
> > "Support in driver to properly execute *1 binary." I found this on Jan original patch's email. what does it mean
> >
> > exactly?
> >
> > Regards
> >
> > Rishi Raj
> >
> >
> >
> >

Re: [GSOC] few question about Bypass assembler when generating LTO object files

[Rishi Raj]

[-- Attachment #1: Type: text/plain, Size: 3657 bytes --]

Thanks, Jan for the Reply! I have completed a draft proposal for this
project. I will appreciate your's, Martin's, or anybody else feedback on
the same.
Here is the link to my proposal
https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing

On Tue, 4 Apr 2023 at 04:35, Jan Hubicka <hubicka@ucw.cz> wrote:

> Hello,
> > While going through the patch and simple-object.c I understood that the
> > file simple-object.c is used to handle the object file format. However,
> > this file does not contain all the architecture information required for
> > LTO object files, so the workaround used in the patch is to read the
> > crtbegin.o file and merge the missing attributes. While this workaround
> is
> > functional, it is not optimal, and the ideal solution would be to extend
> > simple-object.c to include the missing information.
>
> Yes, simple-object.c simply uses architecture settings it read earlier
> which is problem since at compile time we do not read any object files,
> just parse sources). In my original patch the architecture flags were
> simply left blank.  I am not sure if there is a version reading
> crtbeing.o which would probably not a be that bad workaround, at least
> for the start.  Having a way to specify this from the machine descriptions
> would be better.
>


>
> Besides the architecture bits, for simple-object files to work we need
> to add the symbol table. For practically useful information we also need
> to stream the debug info.
>
>
> > Regarding the phrase "Support in the driver to properly execute *1
> binary",
> > it is not entirely clear what it refers to. My interpretation is that the
> > compiler driver (the program that coordinates the compilation process)
> > needs to be modified to correctly output LTO object files instead of
> > assembler files (the current approach involves passing the -S and -o
> > <obj_file_name>.o options) and also skip the assembler option while using
> > -fbypass-asm option but I am not sure. Can Jan or Martin please shed some
> > light on this?
> Yes, compiler drivers decides what to do and it needs to know that with
> -flto it does not need to produce assembly file and then invoke gas.  If
> we go the way of reading in crtbegin.o it will also need to pass correct
> crtbegin to *1 binary.  This is generally not that hard to do, just
> needs to be done :)
>
Honza
> >
> > Thanks & Regards
> >
> > Rishi Raj
> >
> > On Sun, 2 Apr 2023 at 03:05, Rishi Raj <rishiraj45035@gmail.com> wrote:
> >
> > > Hii Everyone,
> > > I had already expressed my interest in the " Bypass assembler when
> > > generating LTO object files" project and making a proposal for the
> same. I
> > > know I should have done it earlier but I was admitted to the hospital
> for
> > > past few days :(.
> > > I have a few doubts.
> > > 1)
> > >
> > > "One problem is that the object files produced by
> libiberty/simple-object.c
> > > (which is the low-level API used by the LTO code)
> > > are missing some information (such as the architecture info and symbol
> > > table) and API of the simple object will need to be extended to handle
> > > that" I found this in the previous mailing list discussion. So who
> output this information currently in the object file, is it assembler?
> > >
> > > Also in the current patch for this project by Jan Hubica, from where
> are we getting these information from? Is it from crtbegin.o?
> > >
> > > 2)
> > > "Support in driver to properly execute *1 binary." I found this on Jan
> original patch's email. what does it mean
> > >
> > > exactly?
> > >
> > > Regards
> > >
> > > Rishi Raj
> > >
> > >
> > >
> > >
>

Fwd: [GSOC] few question about Bypass assembler when generating LTO object files

[Rishi Raj]

[-- Attachment #1: Type: text/plain, Size: 3952 bytes --]

---------- Forwarded message ---------
From: Rishi Raj <rishiraj45035@gmail.com>
Date: Tue, 4 Apr 2023 at 05:57
Subject: Re: [GSOC] Submission of draft proposal.
To: Jan Hubicka <hubicka@ucw.cz>
Cc: <gcc@gcc.gnu.org>, <mjambor@suse.cz>
oops, I forgot to change the subject in previous email :(

Thanks, Jan for the Reply! I have completed a draft proposal for this
project. I will appreciate your's, Martin's, or anybody else feedback on
the same.
Here is the link to my proposal
https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing

On Tue, 4 Apr 2023 at 04:35, Jan Hubicka <hubicka@ucw.cz> wrote:

> Hello,
> > While going through the patch and simple-object.c I understood that the
> > file simple-object.c is used to handle the object file format. However,
> > this file does not contain all the architecture information required for
> > LTO object files, so the workaround used in the patch is to read the
> > crtbegin.o file and merge the missing attributes. While this workaround
> is
> > functional, it is not optimal, and the ideal solution would be to extend
> > simple-object.c to include the missing information.
>
> Yes, simple-object.c simply uses architecture settings it read earlier
> which is problem since at compile time we do not read any object files,
> just parse sources). In my original patch the architecture flags were
> simply left blank.  I am not sure if there is a version reading
> crtbeing.o which would probably not a be that bad workaround, at least
> for the start.  Having a way to specify this from the machine descriptions
> would be better.
>


>
> Besides the architecture bits, for simple-object files to work we need
> to add the symbol table. For practically useful information we also need
> to stream the debug info.
>
>
> > Regarding the phrase "Support in the driver to properly execute *1
> binary",
> > it is not entirely clear what it refers to. My interpretation is that the
> > compiler driver (the program that coordinates the compilation process)
> > needs to be modified to correctly output LTO object files instead of
> > assembler files (the current approach involves passing the -S and -o
> > <obj_file_name>.o options) and also skip the assembler option while using
> > -fbypass-asm option but I am not sure. Can Jan or Martin please shed some
> > light on this?
> Yes, compiler drivers decides what to do and it needs to know that with
> -flto it does not need to produce assembly file and then invoke gas.  If
> we go the way of reading in crtbegin.o it will also need to pass correct
> crtbegin to *1 binary.  This is generally not that hard to do, just
> needs to be done :)
>
Honza
> >
> > Thanks & Regards
> >
> > Rishi Raj
> >
> > On Sun, 2 Apr 2023 at 03:05, Rishi Raj <rishiraj45035@gmail.com> wrote:
> >
> > > Hii Everyone,
> > > I had already expressed my interest in the " Bypass assembler when
> > > generating LTO object files" project and making a proposal for the
> same. I
> > > know I should have done it earlier but I was admitted to the hospital
> for
> > > past few days :(.
> > > I have a few doubts.
> > > 1)
> > >
> > > "One problem is that the object files produced by
> libiberty/simple-object.c
> > > (which is the low-level API used by the LTO code)
> > > are missing some information (such as the architecture info and symbol
> > > table) and API of the simple object will need to be extended to handle
> > > that" I found this in the previous mailing list discussion. So who
> output this information currently in the object file, is it assembler?
> > >
> > > Also in the current patch for this project by Jan Hubica, from where
> are we getting these information from? Is it from crtbegin.o?
> > >
> > > 2)
> > > "Support in driver to properly execute *1 binary." I found this on Jan
> original patch's email. what does it mean
> > >
> > > exactly?
> > >
> > > Regards
> > >
> > > Rishi Raj
> > >
> > >
> > >
> > >
>

Re: [GSOC] few question about Bypass assembler when generating LTO object files

[Martin Jambor]

Hi,

On Tue, Apr 04 2023, Rishi Raj wrote:
> Thanks, Jan for the Reply! I have completed a draft proposal for this
> project. I will appreciate your's, Martin's, or anybody else feedback on
> the same.
> Here is the link to my proposal
> https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing
>

in my opinion, the proposal looks rather good.  I don't know how to
significantly improve it, not in the remaining nine hours before the
deadline (just maybe fix the spelling of Honza's surname, it is Hubicka
:-). So please go ahead and submit it. (How the selection goes will then
depend on how many slots we get from Google).

Thanks for putting in the effort,

Martin


> On Tue, 4 Apr 2023 at 04:35, Jan Hubicka <hubicka@ucw.cz> wrote:
>
>> Hello,
>> > While going through the patch and simple-object.c I understood that the
>> > file simple-object.c is used to handle the object file format. However,
>> > this file does not contain all the architecture information required for
>> > LTO object files, so the workaround used in the patch is to read the
>> > crtbegin.o file and merge the missing attributes. While this workaround
>> is
>> > functional, it is not optimal, and the ideal solution would be to extend
>> > simple-object.c to include the missing information.
>>
>> Yes, simple-object.c simply uses architecture settings it read earlier
>> which is problem since at compile time we do not read any object files,
>> just parse sources). In my original patch the architecture flags were
>> simply left blank.  I am not sure if there is a version reading
>> crtbeing.o which would probably not a be that bad workaround, at least
>> for the start.  Having a way to specify this from the machine descriptions
>> would be better.
>>
>
>
>>
>> Besides the architecture bits, for simple-object files to work we need
>> to add the symbol table. For practically useful information we also need
>> to stream the debug info.
>>
>>
>> > Regarding the phrase "Support in the driver to properly execute *1
>> binary",
>> > it is not entirely clear what it refers to. My interpretation is that the
>> > compiler driver (the program that coordinates the compilation process)
>> > needs to be modified to correctly output LTO object files instead of
>> > assembler files (the current approach involves passing the -S and -o
>> > <obj_file_name>.o options) and also skip the assembler option while using
>> > -fbypass-asm option but I am not sure. Can Jan or Martin please shed some
>> > light on this?
>> Yes, compiler drivers decides what to do and it needs to know that with
>> -flto it does not need to produce assembly file and then invoke gas.  If
>> we go the way of reading in crtbegin.o it will also need to pass correct
>> crtbegin to *1 binary.  This is generally not that hard to do, just
>> needs to be done :)
>>
> Honza
>> >
>> > Thanks & Regards
>> >
>> > Rishi Raj
>> >
>> > On Sun, 2 Apr 2023 at 03:05, Rishi Raj <rishiraj45035@gmail.com> wrote:
>> >
>> > > Hii Everyone,
>> > > I had already expressed my interest in the " Bypass assembler when
>> > > generating LTO object files" project and making a proposal for the
>> same. I
>> > > know I should have done it earlier but I was admitted to the hospital
>> for
>> > > past few days :(.
>> > > I have a few doubts.
>> > > 1)
>> > >
>> > > "One problem is that the object files produced by
>> libiberty/simple-object.c
>> > > (which is the low-level API used by the LTO code)
>> > > are missing some information (such as the architecture info and symbol
>> > > table) and API of the simple object will need to be extended to handle
>> > > that" I found this in the previous mailing list discussion. So who
>> output this information currently in the object file, is it assembler?
>> > >
>> > > Also in the current patch for this project by Jan Hubica, from where
>> are we getting these information from? Is it from crtbegin.o?
>> > >
>> > > 2)
>> > > "Support in driver to properly execute *1 binary." I found this on Jan
>> original patch's email. what does it mean
>> > >
>> > > exactly?
>> > >
>> > > Regards
>> > >
>> > > Rishi Raj
>> > >
>> > >
>> > >
>> > >
>>

Re: [GSOC] few question about Bypass assembler when generating LTO object files

[Jan Hubicka]

Hello,
> Thanks, Jan for the Reply! I have completed a draft proposal for this
> project. I will appreciate your's, Martin's, or anybody else feedback on
> the same.
> Here is the link to my proposal
> https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing

Here are few comments on the proposal:

> The current Implementation of GCC first write the IL representation along with other section in an assembly file and then the assembler is used to convert it into LTO object files. Sections containing different IL representation is created and data is appended in lto-streamer-out.cc.I

The .o generated withh -flto file contains the IL (in different
sections), debug info, symbol table, etc.
"along with other section" sounds odd to me. Perhaps sections.

Second sentence seems bit odd too. Perhaps "Streaming of individual
sections is implemented in lto-streamer-out.cc which can either be used
to produce assembly code containing the section data (dumped
hexadecimally) or simple-object API provided by libiberty to produce
object files directly"

> In the slim object file (Default when using -flto, fat lto can be obtained using -ffat-lto-object) some section contains the IL and other contains the info related to architecture, command line options, symbol table, etc. 

Technically the architecture is part of ELF header and not section
itself (I think).

There are some other grammar errors, but I am not too good on fixing
these, so perhaps Martin can help.

The timeline looks reasonable.  It certianly makes sense to look into
non-ELF object files to understand what API we need, but implementation
wise I would suggest implementing ELF path first to get a working
implementation. Adding support for other object formats can be done
incrementally.

Honza
> 
> On Tue, 4 Apr 2023 at 04:35, Jan Hubicka <hubicka@ucw.cz> wrote:
> 
> > Hello,
> > > While going through the patch and simple-object.c I understood that the
> > > file simple-object.c is used to handle the object file format. However,
> > > this file does not contain all the architecture information required for
> > > LTO object files, so the workaround used in the patch is to read the
> > > crtbegin.o file and merge the missing attributes. While this workaround
> > is
> > > functional, it is not optimal, and the ideal solution would be to extend
> > > simple-object.c to include the missing information.
> >
> > Yes, simple-object.c simply uses architecture settings it read earlier
> > which is problem since at compile time we do not read any object files,
> > just parse sources). In my original patch the architecture flags were
> > simply left blank.  I am not sure if there is a version reading
> > crtbeing.o which would probably not a be that bad workaround, at least
> > for the start.  Having a way to specify this from the machine descriptions
> > would be better.
> >
> 
> 
> >
> > Besides the architecture bits, for simple-object files to work we need
> > to add the symbol table. For practically useful information we also need
> > to stream the debug info.
> >
> >
> > > Regarding the phrase "Support in the driver to properly execute *1
> > binary",
> > > it is not entirely clear what it refers to. My interpretation is that the
> > > compiler driver (the program that coordinates the compilation process)
> > > needs to be modified to correctly output LTO object files instead of
> > > assembler files (the current approach involves passing the -S and -o
> > > <obj_file_name>.o options) and also skip the assembler option while using
> > > -fbypass-asm option but I am not sure. Can Jan or Martin please shed some
> > > light on this?
> > Yes, compiler drivers decides what to do and it needs to know that with
> > -flto it does not need to produce assembly file and then invoke gas.  If
> > we go the way of reading in crtbegin.o it will also need to pass correct
> > crtbegin to *1 binary.  This is generally not that hard to do, just
> > needs to be done :)
> >
> Honza
> > >
> > > Thanks & Regards
> > >
> > > Rishi Raj
> > >
> > > On Sun, 2 Apr 2023 at 03:05, Rishi Raj <rishiraj45035@gmail.com> wrote:
> > >
> > > > Hii Everyone,
> > > > I had already expressed my interest in the " Bypass assembler when
> > > > generating LTO object files" project and making a proposal for the
> > same. I
> > > > know I should have done it earlier but I was admitted to the hospital
> > for
> > > > past few days :(.
> > > > I have a few doubts.
> > > > 1)
> > > >
> > > > "One problem is that the object files produced by
> > libiberty/simple-object.c
> > > > (which is the low-level API used by the LTO code)
> > > > are missing some information (such as the architecture info and symbol
> > > > table) and API of the simple object will need to be extended to handle
> > > > that" I found this in the previous mailing list discussion. So who
> > output this information currently in the object file, is it assembler?
> > > >
> > > > Also in the current patch for this project by Jan Hubica, from where
> > are we getting these information from? Is it from crtbegin.o?
> > > >
> > > > 2)
> > > > "Support in driver to properly execute *1 binary." I found this on Jan
> > original patch's email. what does it mean
> > > >
> > > > exactly?
> > > >
> > > > Regards
> > > >
> > > > Rishi Raj
> > > >
> > > >
> > > >
> > > >
> >

Re: [GSOC] few question about Bypass assembler when generating LTO object files

[Rishi Raj]

[-- Attachment #1: Type: text/plain, Size: 6137 bytes --]

Thanks to Martin, Honza, and Théo for your feedback. I have incorporated
almost all of it, updated my proposal accordingly, and submitted it.
Regarding grammar errors, I have fixed many, but there may still be some
left (I could be better at grammar, to be honest :( ).

On Tue, 4 Apr 2023 at 15:55, Jan Hubicka <hubicka@ucw.cz> wrote:

> Hello,
> > Thanks, Jan for the Reply! I have completed a draft proposal for this
> > project. I will appreciate your's, Martin's, or anybody else feedback on
> > the same.
> > Here is the link to my proposal
> >
> https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing
>
> Here are few comments on the proposal:
>
> > The current Implementation of GCC first write the IL representation
> along with other section in an assembly file and then the assembler is used
> to convert it into LTO object files. Sections containing different IL
> representation is created and data is appended in lto-streamer-out.cc.I
>
> The .o generated withh -flto file contains the IL (in different
> sections), debug info, symbol table, etc.
> "along with other section" sounds odd to me. Perhaps sections.
>
> Second sentence seems bit odd too. Perhaps "Streaming of individual
> sections is implemented in lto-streamer-out.cc which can either be used
> to produce assembly code containing the section data (dumped
> hexadecimally) or simple-object API provided by libiberty to produce
> object files directly"
>
> > In the slim object file (Default when using -flto, fat lto can be
> obtained using -ffat-lto-object) some section contains the IL and other
> contains the info related to architecture, command line options, symbol
> table, etc.
>
> Technically the architecture is part of ELF header and not section
> itself (I think).
>
> There are some other grammar errors, but I am not too good on fixing
> these, so perhaps Martin can help.
>
> The timeline looks reasonable.  It certianly makes sense to look into
> non-ELF object files to understand what API we need, but implementation
> wise I would suggest implementing ELF path first to get a working
> implementation. Adding support for other object formats can be done
> incrementally.
>
> Honza
> >
> > On Tue, 4 Apr 2023 at 04:35, Jan Hubicka <hubicka@ucw.cz> wrote:
> >
> > > Hello,
> > > > While going through the patch and simple-object.c I understood that
> the
> > > > file simple-object.c is used to handle the object file format.
> However,
> > > > this file does not contain all the architecture information required
> for
> > > > LTO object files, so the workaround used in the patch is to read the
> > > > crtbegin.o file and merge the missing attributes. While this
> workaround
> > > is
> > > > functional, it is not optimal, and the ideal solution would be to
> extend
> > > > simple-object.c to include the missing information.
> > >
> > > Yes, simple-object.c simply uses architecture settings it read earlier
> > > which is problem since at compile time we do not read any object files,
> > > just parse sources). In my original patch the architecture flags were
> > > simply left blank.  I am not sure if there is a version reading
> > > crtbeing.o which would probably not a be that bad workaround, at least
> > > for the start.  Having a way to specify this from the machine
> descriptions
> > > would be better.
> > >
> >
> >
> > >
> > > Besides the architecture bits, for simple-object files to work we need
> > > to add the symbol table. For practically useful information we also
> need
> > > to stream the debug info.
> > >
> > >
> > > > Regarding the phrase "Support in the driver to properly execute *1
> > > binary",
> > > > it is not entirely clear what it refers to. My interpretation is
> that the
> > > > compiler driver (the program that coordinates the compilation
> process)
> > > > needs to be modified to correctly output LTO object files instead of
> > > > assembler files (the current approach involves passing the -S and -o
> > > > <obj_file_name>.o options) and also skip the assembler option while
> using
> > > > -fbypass-asm option but I am not sure. Can Jan or Martin please shed
> some
> > > > light on this?
> > > Yes, compiler drivers decides what to do and it needs to know that with
> > > -flto it does not need to produce assembly file and then invoke gas.
> If
> > > we go the way of reading in crtbegin.o it will also need to pass
> correct
> > > crtbegin to *1 binary.  This is generally not that hard to do, just
> > > needs to be done :)
> > >
> > Honza
> > > >
> > > > Thanks & Regards
> > > >
> > > > Rishi Raj
> > > >
> > > > On Sun, 2 Apr 2023 at 03:05, Rishi Raj <rishiraj45035@gmail.com>
> wrote:
> > > >
> > > > > Hii Everyone,
> > > > > I had already expressed my interest in the " Bypass assembler when
> > > > > generating LTO object files" project and making a proposal for the
> > > same. I
> > > > > know I should have done it earlier but I was admitted to the
> hospital
> > > for
> > > > > past few days :(.
> > > > > I have a few doubts.
> > > > > 1)
> > > > >
> > > > > "One problem is that the object files produced by
> > > libiberty/simple-object.c
> > > > > (which is the low-level API used by the LTO code)
> > > > > are missing some information (such as the architecture info and
> symbol
> > > > > table) and API of the simple object will need to be extended to
> handle
> > > > > that" I found this in the previous mailing list discussion. So who
> > > output this information currently in the object file, is it assembler?
> > > > >
> > > > > Also in the current patch for this project by Jan Hubica, from
> where
> > > are we getting these information from? Is it from crtbegin.o?
> > > > >
> > > > > 2)
> > > > > "Support in driver to properly execute *1 binary." I found this on
> Jan
> > > original patch's email. what does it mean
> > > > >
> > > > > exactly?
> > > > >
> > > > > Regards
> > > > >
> > > > > Rishi Raj
> > > > >
> > > > >
> > > > >
> > > > >
> > >
>

Re: [GSOC] few question about Bypass assembler when generating LTO object files

[Jan Hubicka]

> Thanks to Martin, Honza, and Théo for your feedback. I have incorporated
> almost all of it, updated my proposal accordingly, and submitted it.
> Regarding grammar errors, I have fixed many, but there may still be some
> left (I could be better at grammar, to be honest :( ).

I could be better too, I think grammar is not critical here.
Thanks a lot for making and submitting the proposal.

Honza
> 
> On Tue, 4 Apr 2023 at 15:55, Jan Hubicka <hubicka@ucw.cz> wrote:
> 
> > Hello,
> > > Thanks, Jan for the Reply! I have completed a draft proposal for this
> > > project. I will appreciate your's, Martin's, or anybody else feedback on
> > > the same.
> > > Here is the link to my proposal
> > >
> > https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing
> >
> > Here are few comments on the proposal:
> >
> > > The current Implementation of GCC first write the IL representation
> > along with other section in an assembly file and then the assembler is used
> > to convert it into LTO object files. Sections containing different IL
> > representation is created and data is appended in lto-streamer-out.cc.I
> >
> > The .o generated withh -flto file contains the IL (in different
> > sections), debug info, symbol table, etc.
> > "along with other section" sounds odd to me. Perhaps sections.
> >
> > Second sentence seems bit odd too. Perhaps "Streaming of individual
> > sections is implemented in lto-streamer-out.cc which can either be used
> > to produce assembly code containing the section data (dumped
> > hexadecimally) or simple-object API provided by libiberty to produce
> > object files directly"
> >
> > > In the slim object file (Default when using -flto, fat lto can be
> > obtained using -ffat-lto-object) some section contains the IL and other
> > contains the info related to architecture, command line options, symbol
> > table, etc.
> >
> > Technically the architecture is part of ELF header and not section
> > itself (I think).
> >
> > There are some other grammar errors, but I am not too good on fixing
> > these, so perhaps Martin can help.
> >
> > The timeline looks reasonable.  It certianly makes sense to look into
> > non-ELF object files to understand what API we need, but implementation
> > wise I would suggest implementing ELF path first to get a working
> > implementation. Adding support for other object formats can be done
> > incrementally.
> >
> > Honza
> > >
> > > On Tue, 4 Apr 2023 at 04:35, Jan Hubicka <hubicka@ucw.cz> wrote:
> > >
> > > > Hello,
> > > > > While going through the patch and simple-object.c I understood that
> > the
> > > > > file simple-object.c is used to handle the object file format.
> > However,
> > > > > this file does not contain all the architecture information required
> > for
> > > > > LTO object files, so the workaround used in the patch is to read the
> > > > > crtbegin.o file and merge the missing attributes. While this
> > workaround
> > > > is
> > > > > functional, it is not optimal, and the ideal solution would be to
> > extend
> > > > > simple-object.c to include the missing information.
> > > >
> > > > Yes, simple-object.c simply uses architecture settings it read earlier
> > > > which is problem since at compile time we do not read any object files,
> > > > just parse sources). In my original patch the architecture flags were
> > > > simply left blank.  I am not sure if there is a version reading
> > > > crtbeing.o which would probably not a be that bad workaround, at least
> > > > for the start.  Having a way to specify this from the machine
> > descriptions
> > > > would be better.
> > > >
> > >
> > >
> > > >
> > > > Besides the architecture bits, for simple-object files to work we need
> > > > to add the symbol table. For practically useful information we also
> > need
> > > > to stream the debug info.
> > > >
> > > >
> > > > > Regarding the phrase "Support in the driver to properly execute *1
> > > > binary",
> > > > > it is not entirely clear what it refers to. My interpretation is
> > that the
> > > > > compiler driver (the program that coordinates the compilation
> > process)
> > > > > needs to be modified to correctly output LTO object files instead of
> > > > > assembler files (the current approach involves passing the -S and -o
> > > > > <obj_file_name>.o options) and also skip the assembler option while
> > using
> > > > > -fbypass-asm option but I am not sure. Can Jan or Martin please shed
> > some
> > > > > light on this?
> > > > Yes, compiler drivers decides what to do and it needs to know that with
> > > > -flto it does not need to produce assembly file and then invoke gas.
> > If
> > > > we go the way of reading in crtbegin.o it will also need to pass
> > correct
> > > > crtbegin to *1 binary.  This is generally not that hard to do, just
> > > > needs to be done :)
> > > >
> > > Honza
> > > > >
> > > > > Thanks & Regards
> > > > >
> > > > > Rishi Raj
> > > > >
> > > > > On Sun, 2 Apr 2023 at 03:05, Rishi Raj <rishiraj45035@gmail.com>
> > wrote:
> > > > >
> > > > > > Hii Everyone,
> > > > > > I had already expressed my interest in the " Bypass assembler when
> > > > > > generating LTO object files" project and making a proposal for the
> > > > same. I
> > > > > > know I should have done it earlier but I was admitted to the
> > hospital
> > > > for
> > > > > > past few days :(.
> > > > > > I have a few doubts.
> > > > > > 1)
> > > > > >
> > > > > > "One problem is that the object files produced by
> > > > libiberty/simple-object.c
> > > > > > (which is the low-level API used by the LTO code)
> > > > > > are missing some information (such as the architecture info and
> > symbol
> > > > > > table) and API of the simple object will need to be extended to
> > handle
> > > > > > that" I found this in the previous mailing list discussion. So who
> > > > output this information currently in the object file, is it assembler?
> > > > > >
> > > > > > Also in the current patch for this project by Jan Hubica, from
> > where
> > > > are we getting these information from? Is it from crtbegin.o?
> > > > > >
> > > > > > 2)
> > > > > > "Support in driver to properly execute *1 binary." I found this on
> > Jan
> > > > original patch's email. what does it mean
> > > > > >
> > > > > > exactly?
> > > > > >
> > > > > > Regards
> > > > > >
> > > > > > Rishi Raj
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > >
> >