Re: [pph] Adapting LTO streaming for front end AST saving

[Diego Novillo <dnovillo@google.com>]

This thread spilled into IRC chatter.  I think we stopped talking past
each other now:

(2011-03-23 12:51:34) froydnj: dnovillo: gimple gets rid of trees?
how does that work?
(2011-03-23 12:52:29) dnovillo: froydnj: we've been talking about
tuplifying more, but i don't think it makes sense past certain point
(say, _CST, IDENTIFIERS, DECLs, etc)
(2011-03-23 12:53:05) dnovillo: i wasn't clear in my initial message,
i think that's what confused honza and richi
(2011-03-23 12:58:26) honza: dnovillo: What would be really nice to
have is an sane (from middle end POV) type representation.  Tree types
makes life quite hard, especially at WPA time.
(2011-03-23 12:58:35) honza: because they tie way too many things together
(2011-03-23 12:59:03) honza: but indeed gimplified/tuplified types are
quite intrusive change.
(2011-03-23 12:59:18) dnovillo: honza: agreed.  but i want to make
sure my point is clear.  i want to reuse common streaming code.  it's
more than just pickling trees.
(2011-03-23 13:00:06) dnovillo: honza: there will be things we will
never change out of trees, so that will stay in common code.  if/when
we start getting rid of trees in gimple, it will make sense to take
that code out of lto-streamer*.c and move it into tree-streamer*c or
some such.
(2011-03-23 13:00:30) dnovillo: does that clarify things?
(2011-03-23 13:02:53) honza: dnovillo: Well, I am trying to get
cleaner idea where we want to go with streaming WRT LTO and WPA in
longer term.  It is obvious that gread deal of ineffecitvity coes from
fact that we stream blindly all the data we currently have in our tree
nodes = a lot of things that is never really used in middle-end.
(2011-03-23 13:03:34) honza: plus I really think we want defined
bytecode in longer run and trees makes this hard too.
(2011-03-23 13:03:35) matz: honza: I'm currently working quite hard to
do merging while reading in the global state, and to reduce the stuff
we have to put into global state.
(2011-03-23 13:03:44) dnovillo: honza: sure.  and we want to get rid
of as much as we can there.
(2011-03-23 13:04:06) matz: But now the IPA passes are quite a road-block :/
(2011-03-23 13:04:17) dnovillo: honza: but i don't want to simply toss
out that pickling code.  i want to move it to tree-streamer*
(2011-03-23 13:04:18) honza: matz: Why?
(2011-03-23 13:04:38) dnovillo: honza: so that we can use from FEs
that want to pickle trees.
(2011-03-23 13:04:40) matz: honza: I managed to get quite far with
merging symbols before reading in the cgraph.
(2011-03-23 13:04:58) honza: matz: and now you need to merge summaries, I see ;)
(2011-03-23 13:05:19) matz: honza: Exactly.  And the info in those
summaries is currently not always self-contained.
(2011-03-23 13:05:59) honza: dnovillo: hmm, I really don't know here.
On one hand clearly I would like to see more separation of gimple and
frontend ASTs. on the other side with current state of affairs I see
that not sharing streamer code is just impractical code duplication...
(2011-03-23 13:06:02) matz: honza: For instance the ipa-prop info
doesn't stream out the number of params, and also depends on the
callee list at stream-in to be exactly as when streaming out.
(2011-03-23 13:06:24) dnovillo: honza: now you lost me.  what?
(2011-03-23 13:08:11) honza: dnovillo: for LTO it makes a lot sense
when the way you stream function bodies is well defined VM (well, like
LLVM). We can't reach this when we will just keep saving our trees
without any more strict specification
(2011-03-23 13:08:38) honza: for AST in FE you have different
requirements, even when they happens to be represented by same tree *
structure in GCC.
(2011-03-23 13:08:51) dnovillo: honza: sure.  i agree with that completely.
(2011-03-23 13:09:18) honza: that is flawed design.  So i am just
concerned that merging the streamer implemetnation still goes
indirection of reusing tree for both purposes.
(2011-03-23 13:09:55) dnovillo: honza: i am not talking about merging
anything.  i don't know how to explain this.  let me try again.
(2011-03-23 13:10:04) dnovillo: honza: there's a bunch of streaming
code that has nothing to do with pickling.
(2011-03-23 13:10:05) honza: but I guess without going all the way and
designing gimple way of implementing tuples/debug info etc. there is
not much help.
(2011-03-23 13:10:28) dnovillo: honza: that code is commonable and
re-usable from various places.
(2011-03-23 13:10:30) dnovillo: right?
(2011-03-23 13:10:59) honza: OK. Depends on how high level of
streaming logic you think of.
(2011-03-23 13:11:21) dnovillo: (keeping of string tables, streams,
sections, hooks to read/write, etc)
(2011-03-23 13:11:35) honza: yep, that is fine for sure.
(2011-03-23 13:11:39) dnovillo: honza: the *pickling* code is what you
are worried about.
(2011-03-23 13:11:50) honza: OK, I guess we are in sync then :0
(2011-03-23 13:12:00) honza: Yes, getting this separated and with
clean API is good idea.
(2011-03-23 13:12:17) matz: And even the pickling code currently can
be shared.  It's not that we currently have anything else than the
tree pickling code we currently have :)
(2011-03-23 13:12:21) dnovillo: honza: what i propose there is to
reduce gimple's dependence on it.  move it to a different place (say
tree-pickle* or tree-streamer*)
(2011-03-23 13:12:31) dnovillo: for now i can just share it since
gimple also uses it
(2011-03-23 13:12:37) dnovillo: if gimple stops using it, that's fine.
(2011-03-23 13:12:48) dnovillo: all i ask is for it not to be tossed out.
mattyb matz
(2011-03-23 13:13:01) matz: We'd have to have a replacement for it first :)
(2011-03-23 13:13:19) dnovillo: matz: yes :)  and there is some tree
pickling that gimple will always want/need to do.
(2011-03-23 13:13:27) matz: most probably, yes.
(2011-03-23 13:13:30) honza: dnovillo: Yep, sharing these things and
crafting it somewhat better is definitely an improvmenet.
(2011-03-23 13:13:38) dnovillo: everything i add to it will be via
streamer hooks.
(2011-03-23 13:14:03) dnovillo: so, i will not be modifying LTO
behaviour in any way.
(2011-03-23 13:15:10) honza: well, OK. Current LTO behaviour is
definitely not very good, but as Micha mentions, we need replacement
for that first anyway.
(2011-03-23 13:15:33) matz: We're all at GROW, right?
(2011-03-23 13:15:45) dnovillo: OK, then i guess we are on the same
page now.  yes, i'll be at GROW.
(2011-03-23 13:16:02) matz: the suse bunch too.
(2011-03-23 13:16:07) dnovillo: great.