Serializing mixed gimple data from a plugin in LTO?

Serializing mixed gimple data from a plugin in LTO?

[Basile Starynkevitch]

Hello All,

Let's imagine someone is writing a plugin (or a MELT extension...) which computes some complex information around gimple in a pass before LTO serialization and re-uses that information in a later pass after LTO serialization. A possible way to associate data inside a plugin to some gimples would be to have hash tables (keys being Gimple, associated value being some complex information specific to the plugin). The the plugin would compute & serialize [i.e. write or output] data in a pass before LTO and deserialize [re-read or input] that data at LTO time.

The writing happens in the write_summary and write_optimization_summary hooks of struct ipa_opt_pass_d of file tree-pass.h. The reading happens in read_summary & read_optimization_summary hooks.

But what I cannot understand is how can such a plugin know what are the gimple involved. Since the plugin needs to persist (i.e. serialize into LTO stream & de-serialize) an hash table mapping some gimples to its data, the plugin need to know what gimples are serialized & deserialized, to restore at LTO streaming read the common gimple pointers (the same gimple pointer should appear in the function & the hash table)....

I would imagine gcc/lto-streamer-out.c would have a plugin hook inside output_gimple_stmt but this is not the case...

I admit I don't have a clear picture of LTO streams usage (& customization) from plugins. 
Or perhaps it is not possible yet???

Regards.
-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***

Re: Serializing mixed gimple data from a plugin in LTO?

[Richard Guenther]

On Fri, Mar 25, 2011 at 8:37 AM, Basile Starynkevitch
<basile@starynkevitch.net> wrote:
> Hello All,
>
> Let's imagine someone is writing a plugin (or a MELT extension...) which computes some complex information around gimple in a pass before LTO serialization and re-uses that information in a later pass after LTO serialization. A possible way to associate data inside a plugin to some gimples would be to have hash tables (keys being Gimple, associated value being some complex information specific to the plugin). The the plugin would compute & serialize [i.e. write or output] data in a pass before LTO and deserialize [re-read or input] that data at LTO time.
>
> The writing happens in the write_summary and write_optimization_summary hooks of struct ipa_opt_pass_d of file tree-pass.h. The reading happens in read_summary & read_optimization_summary hooks.
>
> But what I cannot understand is how can such a plugin know what are the gimple involved. Since the plugin needs to persist (i.e. serialize into LTO stream & de-serialize) an hash table mapping some gimples to its data, the plugin need to know what gimples are serialized & deserialized, to restore at LTO streaming read the common gimple pointers (the same gimple pointer should appear in the function & the hash table)....
>
> I would imagine gcc/lto-streamer-out.c would have a plugin hook inside output_gimple_stmt but this is not the case...
>
> I admit I don't have a clear picture of LTO streams usage (& customization) from plugins.
> Or perhaps it is not possible yet???

You are not allowed to piggy-back on statements but only global things
like cgraph and varpool.  There is no way to transition data from before
LTO to after associated with statements.  And there never will be.

Richard.

> Regards.
> --
> Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
> email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
> 8, rue de la Faiencerie, 92340 Bourg La Reine, France
> *** opinions {are only mine, sont seulement les miennes} ***
>

Re: Serializing mixed gimple data from a plugin in LTO?

[Diego Novillo]

On Fri, Mar 25, 2011 at 06:06, Richard Guenther
<richard.guenther@gmail.com> wrote:

> You are not allowed to piggy-back on statements but only global things
> like cgraph and varpool.  There is no way to transition data from before
> LTO to after associated with statements.  And there never will be.

But one could write out a table indexed by gimple statement uid.  That
would allow the pass to reconstruct the statement metadata on the way
in.


Diego.

Re: Serializing mixed gimple data from a plugin in LTO?

[Richard Guenther]

On Fri, Mar 25, 2011 at 12:52 PM, Diego Novillo <dnovillo@google.com> wrote:
> On Fri, Mar 25, 2011 at 06:06, Richard Guenther
> <richard.guenther@gmail.com> wrote:
>
>> You are not allowed to piggy-back on statements but only global things
>> like cgraph and varpool.  There is no way to transition data from before
>> LTO to after associated with statements.  And there never will be.
>
> But one could write out a table indexed by gimple statement uid.  That
> would allow the pass to reconstruct the statement metadata on the way
> in.

That's true.  You still have to shove your data through WPA stage though
(which doesn't have stmts).

Richard.