Re: [GSOC] few question about Bypass assembler when generating LTO object files

[Jan Hubicka <hubicka@ucw.cz>]

> Thanks to Martin, Honza, and Théo for your feedback. I have incorporated
> almost all of it, updated my proposal accordingly, and submitted it.
> Regarding grammar errors, I have fixed many, but there may still be some
> left (I could be better at grammar, to be honest :( ).

I could be better too, I think grammar is not critical here.
Thanks a lot for making and submitting the proposal.

Honza
> 
> On Tue, 4 Apr 2023 at 15:55, Jan Hubicka <hubicka@ucw.cz> wrote:
> 
> > Hello,
> > > Thanks, Jan for the Reply! I have completed a draft proposal for this
> > > project. I will appreciate your's, Martin's, or anybody else feedback on
> > > the same.
> > > Here is the link to my proposal
> > >
> > https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing
> >
> > Here are few comments on the proposal:
> >
> > > The current Implementation of GCC first write the IL representation
> > along with other section in an assembly file and then the assembler is used
> > to convert it into LTO object files. Sections containing different IL
> > representation is created and data is appended in lto-streamer-out.cc.I
> >
> > The .o generated withh -flto file contains the IL (in different
> > sections), debug info, symbol table, etc.
> > "along with other section" sounds odd to me. Perhaps sections.
> >
> > Second sentence seems bit odd too. Perhaps "Streaming of individual
> > sections is implemented in lto-streamer-out.cc which can either be used
> > to produce assembly code containing the section data (dumped
> > hexadecimally) or simple-object API provided by libiberty to produce
> > object files directly"
> >
> > > In the slim object file (Default when using -flto, fat lto can be
> > obtained using -ffat-lto-object) some section contains the IL and other
> > contains the info related to architecture, command line options, symbol
> > table, etc.
> >
> > Technically the architecture is part of ELF header and not section
> > itself (I think).
> >
> > There are some other grammar errors, but I am not too good on fixing
> > these, so perhaps Martin can help.
> >
> > The timeline looks reasonable.  It certianly makes sense to look into
> > non-ELF object files to understand what API we need, but implementation
> > wise I would suggest implementing ELF path first to get a working
> > implementation. Adding support for other object formats can be done
> > incrementally.
> >
> > Honza
> > >
> > > On Tue, 4 Apr 2023 at 04:35, Jan Hubicka <hubicka@ucw.cz> wrote:
> > >
> > > > Hello,
> > > > > While going through the patch and simple-object.c I understood that
> > the
> > > > > file simple-object.c is used to handle the object file format.
> > However,
> > > > > this file does not contain all the architecture information required
> > for
> > > > > LTO object files, so the workaround used in the patch is to read the
> > > > > crtbegin.o file and merge the missing attributes. While this
> > workaround
> > > > is
> > > > > functional, it is not optimal, and the ideal solution would be to
> > extend
> > > > > simple-object.c to include the missing information.
> > > >
> > > > Yes, simple-object.c simply uses architecture settings it read earlier
> > > > which is problem since at compile time we do not read any object files,
> > > > just parse sources). In my original patch the architecture flags were
> > > > simply left blank.  I am not sure if there is a version reading
> > > > crtbeing.o which would probably not a be that bad workaround, at least
> > > > for the start.  Having a way to specify this from the machine
> > descriptions
> > > > would be better.
> > > >
> > >
> > >
> > > >
> > > > Besides the architecture bits, for simple-object files to work we need
> > > > to add the symbol table. For practically useful information we also
> > need
> > > > to stream the debug info.
> > > >
> > > >
> > > > > Regarding the phrase "Support in the driver to properly execute *1
> > > > binary",
> > > > > it is not entirely clear what it refers to. My interpretation is
> > that the
> > > > > compiler driver (the program that coordinates the compilation
> > process)
> > > > > needs to be modified to correctly output LTO object files instead of
> > > > > assembler files (the current approach involves passing the -S and -o
> > > > > <obj_file_name>.o options) and also skip the assembler option while
> > using
> > > > > -fbypass-asm option but I am not sure. Can Jan or Martin please shed
> > some
> > > > > light on this?
> > > > Yes, compiler drivers decides what to do and it needs to know that with
> > > > -flto it does not need to produce assembly file and then invoke gas.
> > If
> > > > we go the way of reading in crtbegin.o it will also need to pass
> > correct
> > > > crtbegin to *1 binary.  This is generally not that hard to do, just
> > > > needs to be done :)
> > > >
> > > Honza
> > > > >
> > > > > Thanks & Regards
> > > > >
> > > > > Rishi Raj
> > > > >
> > > > > On Sun, 2 Apr 2023 at 03:05, Rishi Raj <rishiraj45035@gmail.com>
> > wrote:
> > > > >
> > > > > > Hii Everyone,
> > > > > > I had already expressed my interest in the " Bypass assembler when
> > > > > > generating LTO object files" project and making a proposal for the
> > > > same. I
> > > > > > know I should have done it earlier but I was admitted to the
> > hospital
> > > > for
> > > > > > past few days :(.
> > > > > > I have a few doubts.
> > > > > > 1)
> > > > > >
> > > > > > "One problem is that the object files produced by
> > > > libiberty/simple-object.c
> > > > > > (which is the low-level API used by the LTO code)
> > > > > > are missing some information (such as the architecture info and
> > symbol
> > > > > > table) and API of the simple object will need to be extended to
> > handle
> > > > > > that" I found this in the previous mailing list discussion. So who
> > > > output this information currently in the object file, is it assembler?
> > > > > >
> > > > > > Also in the current patch for this project by Jan Hubica, from
> > where
> > > > are we getting these information from? Is it from crtbegin.o?
> > > > > >
> > > > > > 2)
> > > > > > "Support in driver to properly execute *1 binary." I found this on
> > Jan
> > > > original patch's email. what does it mean
> > > > > >
> > > > > > exactly?
> > > > > >
> > > > > > Regards
> > > > > >
> > > > > > Rishi Raj
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > >
> >