Re: [PATCH,RFC,V3 0/5] Support for CTF in GCC

[Richard Biener <richard.guenther@gmail.com>]

On Thu, Jul 4, 2019 at 2:36 AM Indu Bhagat <indu.bhagat@oracle.com> wrote:
>
>
> On 07/03/2019 05:31 AM, Richard Biener wrote:
> > On Wed, Jul 3, 2019 at 5:18 AM Jeff Law <law@redhat.com> wrote:
> >> On 7/2/19 11:54 AM, Indu Bhagat wrote:
> >>> Ping.
> >>> Can someone please review these patches ? We would like to get the
> >>> support for CTF integrated soon.
> >> I'm not sure there's really even consensus that we want CTF support in
> >> GCC.  Though I think that the changes you've made in the last several
> >> weeks do make it somewhat more palatable.  But ultimately the first step
> >> is to get that consensus.
> >>
> >> I'd hazard a guess that Jakub in particular isn't on board as he's been
> >> pushing to some degree for post-processing or perhaps doing it via a
> >> plug in.
> >>
> >> Richi has been guiding you a bit through how to make the changes easier
> >> to integrate, but I haven't seen him state one way or the other his
> >> preference on whether or not CTF support is something we want.
> > I'm mostly worried about the lack of a specification and the appearant
> > restriction on a subset of C (the patches have gcc_unreachable ()
> > in paths that can be reached by VECTOR_TYPE or COMPLEX_TYPE
> > not to mention FIXED_POINT_TYPE, etc...).
>
> RE lack of specification : I cannot agree more; This does need to absolutely exist
> if we envision CTF support in toolchain to be useful to the community.
> We plan on getting to this task once the Linker changes are scoped and closer
> to done (~ a couple of weeks from now). Will this work ?

Sure - just keep in mind that it's difficult to give feedback to
something without
a specification.

> RE subset of C : It is true that CTF format currently does leave out a very
> small subset of C like FIXED_POINT as you noted ( CTF does have representation
> for COMPLEX_TYPE, if my code paths culminate to gcc_unreachable () for that, I
> should fix them ).  The end goal is to make it support all of C, and not just a
> subset.

What about other languages?  GCC supports C++, Ada, Objective-C, Go, D,
Fortran, Modula-2, BRIG (this list is not necessarily complete and may change
in the future).

> Meanwhile, I intend to make the compiler skip types when a C construct is not
> supported instead of crashing because of gcc_unreachable (). (You may have also
> noted stubs with "TBD WARN instead" notes in the patch series I sent.)
>
>
> >
> > While CTF might be easy and fast to parse and small I fear it will
> > go the STABS way of being not extensible and bitrotten.
>
> FWIW, I can understand this. We will maintain it. And I hope it will also be a
> community effort thereafter with active consumers, so there is a positive
> feedback loop.
>
> >
> > Given it appears to generate only debug info for symbols and no locations
> > or whatnot it should be sufficient to introspect the compilation to generate
> > the CTF info on the side and then merge it in at link-time.  Which makes
> > me wonder if this shouldn't be a plugin for now until it is more complete
> > and can be evaluated better (comments in the patches indicate even the
> > on-disk format is in flux?).  Adding plugin hook invocations to the three
> > places the CTF info generation hooks off should be easy.
>
> Yes, some bits of the on-disk format are being adapted to make it easier to
> adopt the CTF format across the board. E.g., we recently added CU name in the
> CTF header. As another example, we added CTF_K_SLICE type because there existed
> no way in CTF to represent enum bitfields. For the most part though, CTF format
> has stayed as is.

I hope the format is versioned at least.

> Hmm...a GCC plugin for CTF generation at compile-time may work out for a single
> compilation unit.  But I am not sure how will LTO be supported in that case.
> Basically, for LTO and -gtLEVEL to work together, I need the lto-wrapper to be
> aware of the presence of .ctf sections (so I think). I will need to combine the
> .ctf sections from multiple compilation units into a CTF archive, which the
> linker can then de-duplicate.

True.  lto-wrapper does this kind of dancing for the much more complex set of
DWARF sections already.

> Even if I assume that the technical hurdle in the above paragraph is solvable
> within the purview of a plugin, I fear worse problems of adoption, maintenance
> and distribution in the long run, if CTF support unfortunately ever remains to be
> done via a plugin for reasons unforeseen.
>
> Going the plugin route for the short term, will continue to suffer similar
> problems of distribution and support.
>
> - Is the plugin infrastructure supported on most platforms ? Also, I see that
>    the plugin infrastructure supports all gcc versions from 4.5 onwards.
>    Can someone confirm ? ( We minimally want the toolchain support with
>    GCC 4.8.5 and GCC 8 and later, for now. )

The infrastructure is quite old but you'd need new invocation hooks so this
won't help.

> - How will the plugin be distributed for a variety of platforms and
>    architectures outside of what Oracle Linux commits to support ?
>
>    Unless you are suggesting that the GCC plugin be distributed within GCC,
>    meanwhile ? Well, that may be acceptable in the short term, depending on how
>    I resolve some points raised above.
>
> >
> > That said, the patch series isn't ready for integration since it will
> > crash left and right -- did you bootstrap and run the testsuite
> > with -gt?
> >
> >
> Bootstrap and Testsuite : Yes, I have.  On x86_64/linux, sparc64/linux,
>                            aarch64/linux.
> Run testsuite with -gt : Not yet. Believe me, it's on my plate. And I already
>                           regret not having done it sooner :)
> Bootstrap with -gt : Not yet. I should try soon.
>
> (I have compiled libdtrace-ctf with -gt and parsed the .ctf sections with the
> patch set.)
>
> About the patch being not ready for integration : Yes, you're right.
> That's why I chose to retain 'RFC' for this patch series as well. I am working
> on issues, testing the compiler, and closing on the open ends in the
> implementation.
>
> I will refresh the patch series when I have made a meaningful stride ahead. Any
> further suggestions on functional/performance testing will be helpful too.

What's the functional use of CTF?  Print nice backtraces (without showing
function argument values)?

Richard.

> Thanks again for your reviews.
>
> Indu
>