Re: [PATCH,RFC,V3 0/5] Support for CTF in GCC

[Indu Bhagat <indu.bhagat@oracle.com>]

On 07/03/2019 05:31 AM, Richard Biener wrote:
> On Wed, Jul 3, 2019 at 5:18 AM Jeff Law <law@redhat.com> wrote:
>> On 7/2/19 11:54 AM, Indu Bhagat wrote:
>>> Ping.
>>> Can someone please review these patches ? We would like to get the
>>> support for CTF integrated soon.
>> I'm not sure there's really even consensus that we want CTF support in
>> GCC.  Though I think that the changes you've made in the last several
>> weeks do make it somewhat more palatable.  But ultimately the first step
>> is to get that consensus.
>>
>> I'd hazard a guess that Jakub in particular isn't on board as he's been
>> pushing to some degree for post-processing or perhaps doing it via a
>> plug in.
>>
>> Richi has been guiding you a bit through how to make the changes easier
>> to integrate, but I haven't seen him state one way or the other his
>> preference on whether or not CTF support is something we want.
> I'm mostly worried about the lack of a specification and the appearant
> restriction on a subset of C (the patches have gcc_unreachable ()
> in paths that can be reached by VECTOR_TYPE or COMPLEX_TYPE
> not to mention FIXED_POINT_TYPE, etc...).

RE lack of specification : I cannot agree more; This does need to absolutely exist
if we envision CTF support in toolchain to be useful to the community.
We plan on getting to this task once the Linker changes are scoped and closer
to done (~ a couple of weeks from now). Will this work ?

RE subset of C : It is true that CTF format currently does leave out a very
small subset of C like FIXED_POINT as you noted ( CTF does have representation
for COMPLEX_TYPE, if my code paths culminate to gcc_unreachable () for that, I
should fix them ).  The end goal is to make it support all of C, and not just a
subset.

Meanwhile, I intend to make the compiler skip types when a C construct is not
supported instead of crashing because of gcc_unreachable (). (You may have also
noted stubs with "TBD WARN instead" notes in the patch series I sent.)	


>
> While CTF might be easy and fast to parse and small I fear it will
> go the STABS way of being not extensible and bitrotten.

FWIW, I can understand this. We will maintain it. And I hope it will also be a
community effort thereafter with active consumers, so there is a positive
feedback loop.

>
> Given it appears to generate only debug info for symbols and no locations
> or whatnot it should be sufficient to introspect the compilation to generate
> the CTF info on the side and then merge it in at link-time.  Which makes
> me wonder if this shouldn't be a plugin for now until it is more complete
> and can be evaluated better (comments in the patches indicate even the
> on-disk format is in flux?).  Adding plugin hook invocations to the three
> places the CTF info generation hooks off should be easy.

Yes, some bits of the on-disk format are being adapted to make it easier to
adopt the CTF format across the board. E.g., we recently added CU name in the
CTF header. As another example, we added CTF_K_SLICE type because there existed
no way in CTF to represent enum bitfields. For the most part though, CTF format
has stayed as is.

Hmm...a GCC plugin for CTF generation at compile-time may work out for a single
compilation unit.  But I am not sure how will LTO be supported in that case.
Basically, for LTO and -gtLEVEL to work together, I need the lto-wrapper to be
aware of the presence of .ctf sections (so I think). I will need to combine the
.ctf sections from multiple compilation units into a CTF archive, which the
linker can then de-duplicate.

Even if I assume that the technical hurdle in the above paragraph is solvable
within the purview of a plugin, I fear worse problems of adoption, maintenance
and distribution in the long run, if CTF support unfortunately ever remains to be
done via a plugin for reasons unforeseen.

Going the plugin route for the short term, will continue to suffer similar
problems of distribution and support.

- Is the plugin infrastructure supported on most platforms ? Also, I see that
   the plugin infrastructure supports all gcc versions from 4.5 onwards.
   Can someone confirm ? ( We minimally want the toolchain support with
   GCC 4.8.5 and GCC 8 and later, for now. )

- How will the plugin be distributed for a variety of platforms and
   architectures outside of what Oracle Linux commits to support ?

   Unless you are suggesting that the GCC plugin be distributed within GCC,
   meanwhile ? Well, that may be acceptable in the short term, depending on how
   I resolve some points raised above.


>
> That said, the patch series isn't ready for integration since it will
> crash left and right -- did you bootstrap and run the testsuite
> with -gt?
>
>
Bootstrap and Testsuite : Yes, I have.  On x86_64/linux, sparc64/linux,
                           aarch64/linux.
Run testsuite with -gt : Not yet. Believe me, it's on my plate. And I already
                          regret not having done it sooner :)
Bootstrap with -gt : Not yet. I should try soon.

(I have compiled libdtrace-ctf with -gt and parsed the .ctf sections with the
patch set.)

About the patch being not ready for integration : Yes, you're right.
That's why I chose to retain 'RFC' for this patch series as well. I am working
on issues, testing the compiler, and closing on the open ends in the
implementation.

I will refresh the patch series when I have made a meaningful stride ahead. Any
further suggestions on functional/performance testing will be helpful too.

Thanks again for your reviews.

Indu