[Re: new plugin events]

[Re: new plugin events]

[Terrence Miller]

< Forwarded due to missing address>

-------- Original Message --------
Subject: 	Re: new plugin events
Date: 	Sun, 08 Nov 2009 18:25:21 +0100
From: 	Basile STARYNKEVITCH <basile@starynkevitch.net>
To: 	Terrence Miller <terrencem@sbcglobal.net>
References: 	<4AE72A4F.8000303@starynkevitch.net> 
<4AF28075.7020808@starynkevitch.net> 
<4AF291A6.7000600@starynkevitch.net> 
<38a0d8450911050824l838fd92ya9f3a08205c80a85@mail.gmail.com> 
<4AF33453.7090200@starynkevitch.net> 
<38a0d8450911060724h3c6f9ddh3e84c2c763ac4de0@mail.gmail.com> 
<b798aad50911060800w5bffaf50tdabbbf2d78be4d22@mail.gmail.com> 
<84fc9c000911060818s3462aff1r1ebfb298506b6939@mail.gmail.com> 
<b798aad50911060827r1105ad3fna194ff5b898ce95@mail.gmail.com> 
<4AF4634D.5050303@starynkevitch.net> 
<b798aad50911061036g3be83df2n8d8ca5c4144c3c8d@mail.gmail.com> 
<4AF47257.8040307@starynkevitch.net> <4AF6FB88.4030303@sbcglobal.net>



Terrence Miller wrote:
> I think this debate is missing one important issue. In order to generate 
> a patch to GCC
> you have to know a lot more about the compiler internals than is 
> required to create
> a plugin.  I am doing some casual experimentation with compiler based 
> source browsing
> and would love to have the DECL event(described below) added to the 
> plugin API.
> I would prefer to not have to figure where that plugin should be called.
> 

It is perhaps true for the DECL event you are talking about (which I did not understood fully), but I won't say that 
coding a plugin is *in general* easier than proposing a patch to core GCC.

My perception is that most *middle* end plugins are working on Gimple and other middle-end representations. Usually, 
they do that by adding a pass in the pass machinery. And knowing what pass to add (or replace) is as difficult when 
coding a plugin than when coding a patch to GCC core.

Conversely, I would suspect than many previous patches (which have now bee incorporated in GCC core) could have if a 
plugin machinery have been available at their time (which is false since plugin facilities is just coming in 4.5 which 
has not yet been released), first been developped as plugins -at least to experiment their viability & interest- and 
later one proposed as a patch.

This is why I believe that plugins will probably -at least for middle-end processing- have also the role of GCC branches 
today, with a very important difference. Almost nobody compiles branches today (in the general GCC user community - I am 
not talking of the smaller GCC developper community), and it could be the case that some plugins will be used.

For example, as far as I know, no common Linux distribution provides a package for any kind of GCC branch. I believe 
(perhaps I am too optimistic) that some Linux distributions will package some few GCC plugins.

Apparently you did not sent your reply to gcc@; feel free to forward this (or any reply to gcc@ if you want).

Regards.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: [Re: new plugin events]

[Richard Guenther]

On Sun, Nov 8, 2009 at 6:35 PM, Terrence Miller <terrencem@sbcglobal.net> wrote:
> < Forwarded due to missing address>
>
> -------- Original Message --------
> Subject:        Re: new plugin events
> Date:   Sun, 08 Nov 2009 18:25:21 +0100
> From:   Basile STARYNKEVITCH <basile@starynkevitch.net>
> To:     Terrence Miller <terrencem@sbcglobal.net>
> References:     <4AE72A4F.8000303@starynkevitch.net>
> <4AF28075.7020808@starynkevitch.net> <4AF291A6.7000600@starynkevitch.net>
> <38a0d8450911050824l838fd92ya9f3a08205c80a85@mail.gmail.com>
> <4AF33453.7090200@starynkevitch.net>
> <38a0d8450911060724h3c6f9ddh3e84c2c763ac4de0@mail.gmail.com>
> <b798aad50911060800w5bffaf50tdabbbf2d78be4d22@mail.gmail.com>
> <84fc9c000911060818s3462aff1r1ebfb298506b6939@mail.gmail.com>
> <b798aad50911060827r1105ad3fna194ff5b898ce95@mail.gmail.com>
> <4AF4634D.5050303@starynkevitch.net>
> <b798aad50911061036g3be83df2n8d8ca5c4144c3c8d@mail.gmail.com>
> <4AF47257.8040307@starynkevitch.net> <4AF6FB88.4030303@sbcglobal.net>
>
>
>
> Terrence Miller wrote:
>>
>> I think this debate is missing one important issue. In order to generate a
>> patch to GCC
>> you have to know a lot more about the compiler internals than is required
>> to create
>> a plugin.  I am doing some casual experimentation with compiler based
>> source browsing
>> and would love to have the DECL event(described below) added to the plugin
>> API.
>> I would prefer to not have to figure where that plugin should be called.
>>
>
> It is perhaps true for the DECL event you are talking about (which I did not
> understood fully), but I won't say that coding a plugin is *in general*
> easier than proposing a patch to core GCC.
>
> My perception is that most *middle* end plugins are working on Gimple and
> other middle-end representations. Usually, they do that by adding a pass in
> the pass machinery. And knowing what pass to add (or replace) is as
> difficult when coding a plugin than when coding a patch to GCC core.
>
> Conversely, I would suspect than many previous patches (which have now bee
> incorporated in GCC core) could have if a plugin machinery have been
> available at their time (which is false since plugin facilities is just
> coming in 4.5 which has not yet been released), first been developped as
> plugins -at least to experiment their viability & interest- and later one
> proposed as a patch.
>
> This is why I believe that plugins will probably -at least for middle-end
> processing- have also the role of GCC branches today, with a very important
> difference. Almost nobody compiles branches today (in the general GCC user
> community - I am not talking of the smaller GCC developper community), and
> it could be the case that some plugins will be used.
>
> For example, as far as I know, no common Linux distribution provides a
> package for any kind of GCC branch. I believe (perhaps I am too optimistic)
> that some Linux distributions will package some few GCC plugins.

You keep re-iterating this (IMHO bogus) argument.  I don't see how a plugin
in development is any different here - nobody will build or distribute it.
OTOH after a branch is mature it will be merged into the GCC core, so it
will be immediately available in distributed GCCs.

Richard.

Re: [Re: new plugin events]

[Terrence Miller]

IDE projects are an example of development that could make good use of a 
plugin that might never be integrated in the
compiler, indeed shouldn't ever be integrated in the compiler.

                                     Terrence Miller

Richard Guenther wrote:
> Basile STARYNKEVITCH  wrote:
>>
>> Terrence Miller wrote:
>>     
>>> I think this debate is missing one important issue. In order to generate a
>>> patch to GCC
>>> you have to know a lot more about the compiler internals than is required
>>> to create
>>> a plugin.  I am doing some casual experimentation with compiler based
>>> source browsing
>>> and would love to have the DECL event(described below) added to the plugin
>>> API.
>>> I would prefer to not have to figure where that plugin should be called.
>>>
>>>       
>> It is perhaps true for the DECL event you are talking about (which I did not
>> understood fully), but I won't say that coding a plugin is *in general*
>> easier than proposing a patch to core GCC.
>>
>> My perception is that most *middle* end plugins are working on Gimple and
>> other middle-end representations. Usually, they do that by adding a pass in
>> the pass machinery. And knowing what pass to add (or replace) is as
>> difficult when coding a plugin than when coding a patch to GCC core.
>>
>> Conversely, I would suspect than many previous patches (which have now bee
>> incorporated in GCC core) could have if a plugin machinery have been
>> available at their time (which is false since plugin facilities is just
>> coming in 4.5 which has not yet been released), first been developped as
>> plugins -at least to experiment their viability & interest- and later one
>> proposed as a patch.
>>
>> This is why I believe that plugins will probably -at least for middle-end
>> processing- have also the role of GCC branches today, with a very important
>> difference. Almost nobody compiles branches today (in the general GCC user
>> community - I am not talking of the smaller GCC developper community), and
>> it could be the case that some plugins will be used.
>>
>> For example, as far as I know, no common Linux distribution provides a
>> package for any kind of GCC branch. I believe (perhaps I am too optimistic)
>> that some Linux distributions will package some few GCC plugins.
>>     
>
> You keep re-iterating this (IMHO bogus) argument.  I don't see how a plugin
> in development is any different here - nobody will build or distribute it.
> OTOH after a branch is mature it will be merged into the GCC core, so it
> will be immediately available in distributed GCCs.
>
> Richard.
>
>   

Re: [Re: new plugin events]

[Joern Rennecke]

Quoting Richard Guenther <richard.guenther@gmail.com>:

> On Sun, Nov 8, 2009 at 6:35 PM, Terrence Miller
...
>> For example, as far as I know, no common Linux distribution provides a
>> package for any kind of GCC branch. I believe (perhaps I am too optimistic)
>> that some Linux distributions will package some few GCC plugins.
>
> You keep re-iterating this (IMHO bogus) argument.  I don't see how a plugin
> in development is any different here - nobody will build or distribute it.
> OTOH after a branch is mature it will be merged into the GCC core, so it
> will be immediately available in distributed GCCs.

It is not uncommon that a user complains about some missed optimization or
pessimization that a proposed new pass might fix.
At the moment, a developer might ask the user to download the latest
experimental GCC from trunk, apply his special, even more experimental
patch to it, build and install it (which might accidentally overwrite
the stable compiler if the user has more privileges on the machine than
sysadmin experience), and then check if his code gets better.
Or the developer might ask the user to send/post his/her code, which might
need manager approval, or be outright disallowed for confidentiality reasons.

With a plugin, the developer can simply point the user at the place where
he can download the plugin for his current version, and we can get quick
feedback on the usefulness of the new optimization.

Re: [Re: new plugin events]

[Richard Guenther]

On Sun, Nov 8, 2009 at 9:47 PM, Joern Rennecke <amylaar@spamcop.net> wrote:
> Quoting Richard Guenther <richard.guenther@gmail.com>:
>
>> On Sun, Nov 8, 2009 at 6:35 PM, Terrence Miller
>
> ...
>>>
>>> For example, as far as I know, no common Linux distribution provides a
>>> package for any kind of GCC branch. I believe (perhaps I am too
>>> optimistic)
>>> that some Linux distributions will package some few GCC plugins.
>>
>> You keep re-iterating this (IMHO bogus) argument.  I don't see how a
>> plugin
>> in development is any different here - nobody will build or distribute it.
>> OTOH after a branch is mature it will be merged into the GCC core, so it
>> will be immediately available in distributed GCCs.
>
> It is not uncommon that a user complains about some missed optimization or
> pessimization that a proposed new pass might fix.
> At the moment, a developer might ask the user to download the latest
> experimental GCC from trunk, apply his special, even more experimental
> patch to it, build and install it (which might accidentally overwrite
> the stable compiler if the user has more privileges on the machine than
> sysadmin experience), and then check if his code gets better.
> Or the developer might ask the user to send/post his/her code, which might
> need manager approval, or be outright disallowed for confidentiality
> reasons.
>
> With a plugin, the developer can simply point the user at the place where
> he can download the plugin for his current version, and we can get quick
> feedback on the usefulness of the new optimization.

It's not that simple if you are not suggesting that all plugin development
will happen against a stable branch.  And even then the plugin binary
needs an exactly mathching gcc version - how do you suppose the user
will get that?  By compiling both itself or by the developer being a
distributor of binary gcc versions alongside his plugin?

Note that with the same reasoning the developer could provide patches
against a released gcc instead of just gcc trunk.

Plugins don't make anything easier here.  Really.

Richard.

Re: [Re: new plugin events]

[Richard Guenther]

On Sun, Nov 8, 2009 at 10:03 PM, Richard Guenther
<richard.guenther@gmail.com> wrote:
> On Sun, Nov 8, 2009 at 9:47 PM, Joern Rennecke <amylaar@spamcop.net> wrote:
>> Quoting Richard Guenther <richard.guenther@gmail.com>:
>>
>>> On Sun, Nov 8, 2009 at 6:35 PM, Terrence Miller
>>
>> ...
>>>>
>>>> For example, as far as I know, no common Linux distribution provides a
>>>> package for any kind of GCC branch. I believe (perhaps I am too
>>>> optimistic)
>>>> that some Linux distributions will package some few GCC plugins.
>>>
>>> You keep re-iterating this (IMHO bogus) argument.  I don't see how a
>>> plugin
>>> in development is any different here - nobody will build or distribute it.
>>> OTOH after a branch is mature it will be merged into the GCC core, so it
>>> will be immediately available in distributed GCCs.
>>
>> It is not uncommon that a user complains about some missed optimization or
>> pessimization that a proposed new pass might fix.
>> At the moment, a developer might ask the user to download the latest
>> experimental GCC from trunk, apply his special, even more experimental
>> patch to it, build and install it (which might accidentally overwrite
>> the stable compiler if the user has more privileges on the machine than
>> sysadmin experience), and then check if his code gets better.
>> Or the developer might ask the user to send/post his/her code, which might
>> need manager approval, or be outright disallowed for confidentiality
>> reasons.
>>
>> With a plugin, the developer can simply point the user at the place where
>> he can download the plugin for his current version, and we can get quick
>> feedback on the usefulness of the new optimization.
>
> It's not that simple if you are not suggesting that all plugin development
> will happen against a stable branch.  And even then the plugin binary
> needs an exactly mathching gcc version - how do you suppose the user
> will get that?  By compiling both itself or by the developer being a
> distributor of binary gcc versions alongside his plugin?
>
> Note that with the same reasoning the developer could provide patches
> against a released gcc instead of just gcc trunk.
>
> Plugins don't make anything easier here.  Really.

OTOH you can simply fork the existing gcc 4.5 packages for openSUSE,
add a local patch and a custom suffix and get them built and distributed
using the openSUSE build service with is free to use for everyone.  You
can even build for Fedora or Debian there.

Richard.

Re: [Re: new plugin events]

[Joern Rennecke]

Quoting Richard Guenther <richard.guenther@gmail.com>:
> It's not that simple if you are not suggesting that all plugin development
> will happen against a stable branch.  And even then the plugin binary
> needs an exactly mathching gcc version - how do you suppose the user
> will get that?  By compiling both itself or by the developer being a
> distributor of binary gcc versions alongside his plugin?

No, rather the developer would back-port the plugin to the release
version.  Assuming (s)he is motivated enough to do that in order to get
feedback.
For the binary to match, it should be enough for it to be for the exact
release version and a matching target architecture and object file format.
The operating system should only matter if the plugin itself uses system
functions, or if there are differences in the dynamic library format -
the latter might be overcome with a final linking step on the host system.

With a good plugin interface, backporting should be less painful than
back-porting patches: When you add something to a list with a patch, it
breaks when anything else got added / removed.  If something is added
with a plugin hook, you are isolated from such textual conflicts both
in the main gcc codebase as well as from other plugins.

> Note that with the same reasoning the developer could provide patches
> against a released gcc instead of just gcc trunk.

Not only are there likely more textual conflicts for the developer to
fix, patches are not so easy to use by the user, so fewer people will
volunteer to be ginea pigs for experimental GCC changes.

Moreover, plugins should generally stack well, so we could see interesting
feedback about combining plugins.  multiple patches that affect the same
areas of the compiler can often not be combined by a user.

Re: [Re: new plugin events]

[Gerald Pfeifer]

On Sun, 8 Nov 2009, Joern Rennecke wrote:
> With a plugin, the developer can simply point the user at the place where
> he can download the plugin for his current version, and we can get quick
> feedback on the usefulness of the new optimization.

Except that, based on what Richard and Basile discussed, you may need
a different (binary) plugin for different minor versions of GCC (and, 
possibly, different vendor versions of GCC).

All of which terribly reminds me of the painful (for end users, ISVs, 
IHVs, OSVs,...) situation we have with the Linux kernel and out-of-tree 
modules.

Gerald

Re: [Re: new plugin events]

[Richard Guenther]

On Sun, Nov 8, 2009 at 10:13 PM, Gerald Pfeifer <gerald@pfeifer.com> wrote:
> On Sun, 8 Nov 2009, Joern Rennecke wrote:
>> With a plugin, the developer can simply point the user at the place where
>> he can download the plugin for his current version, and we can get quick
>> feedback on the usefulness of the new optimization.
>
> Except that, based on what Richard and Basile discussed, you may need
> a different (binary) plugin for different minor versions of GCC (and,
> possibly, different vendor versions of GCC).
>
> All of which terribly reminds me of the painful (for end users, ISVs,
> IHVs, OSVs,...) situation we have with the Linux kernel and out-of-tree
> modules.

Correct.  As I said - if there are specific kinds of out-of-tree plugins then
GCC should implement (properly) a high-level abstraction to support them.
In a way that would at least provide ABI compatibility within a stable release.
Not something I see anyone working ok and certainly the thing the FSF
didn't what to have in the first place.  I hope a new pass manager will
be designed with that in mind - even if it will not provide a binary ABI
but instead a stable scripting interface.

Richard.

Re: [Re: new plugin events]

[Basile STARYNKEVITCH]

Gerald Pfeifer wrote:
> On Sun, 8 Nov 2009, Joern Rennecke wrote:
>> With a plugin, the developer can simply point the user at the place where
>> he can download the plugin for his current version, and we can get quick
>> feedback on the usefulness of the new optimization.
> 
> Except that, based on what Richard and Basile discussed, you may need
> a different (binary) plugin for different minor versions of GCC (and, 
> possibly, different vendor versions of GCC).
> 
> All of which terribly reminds me of the painful (for end users, ISVs, 
> IHVs, OSVs,...) situation we have with the Linux kernel and out-of-tree 
> modules.

I do agree with the similarity. But is that situation [of today's linux kernel modules?] really so painful??

One the other hand, as an end-user, I tend to believe that the current kernel with many naughty modules is easier for 
users (than the situation in 1995, at the time of linux 1.2, when you didn't have modules: to profit from a new hardware 
that you just bought, at that time you *had to* configure & build your own kernel). Of course, I would believe it is a 
pain to linux distribution makers, etc. But I tend to believe that for Joe Random user, current kernel modules are more 
a blessing than a mess. At the time of linux 1.2 you needed to understand how to compile a kernel (& various other 
software) when adding new hardware; this is really not the case today. When you plug in some new hardware on a new 
Ubuntu or Debian distribution, is may happens to work without any kernel recompilation. This was not true in 1995, so 
from the user's point of view, I see some progress.

So perhaps GCC plugins are better than no plugins at all. Only time can tell.

Or perhaps I am entirely wrong, and plugins won't be used at all, and we are all losing our time. Nobody knows for sure 
(at least not me). It is only intuitive guesses! I still strongly believe today that plugins are a good thing, but I 
agree it is a bet on the future, and I may be entirely wrong!

And if plugins are not that important, adding more hooks (so perhaps removing some of them later) is not really 
important neither (so I am even more confused that we are debating a few new hooks so much, and putting more energy in 
discussions than in patches). If plugins are not a success, we could eventually remove entirely the plugin support in 
GCC 5.0 (or even 4.6). [I have no idea of who will decide that, and I have no idea of who decided that GCC can have 
plugins. Perhaps the Steering Commitee, or RMS himself??? Certainly not me Basile... :-) :-) and probably this decision 
has not been taken on the gcc@ mailing list.] Of course, discussion of plugin extensions or removal is a possibility of 
flameful heated exchanges for 2011.

Maybe a more realistic bet is that the gcc@ mailing list will have even more heated messages in end of 2011 than in end 
of 2009. :-) :-)

Are there any objective measures of the temperature of a mailing list :-) :-) ? ? ?

A couple of hours ago I was almost angry when reading & writing on this mailing list. Now my mood is that is is quite 
funny to discuss all that. I am enjoying it. :-) :-)

Cheers.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: [Re: new plugin events]

[Gerald Pfeifer]

On Sun, 8 Nov 2009, Basile STARYNKEVITCH wrote:
>> All of which terribly reminds me of the painful (for end users, ISVs, 
>> IHVs, OSVs,...) situation we have with the Linux kernel and out-of-tree 
>> modules.
> I do agree with the similarity. But is that situation [of today's linux 
> kernel modules?] really so painful??

Quite, as far as out-of-tree modules go.  In tree, things are different.

> So perhaps GCC plugins are better than no plugins at all. Only time can 
> tell.

Oh, I didn't say plugins (or kernel modules) are undesirable, let alone
bad.

> And if plugins are not that important, adding more hooks (so perhaps 
> removing some of them later) is not really important neither (so I am 
> even more confused that we are debating a few new hooks so much, and 
> putting more energy in discussions than in patches). If plugins are not 
> a success, we could eventually remove entirely the plugin support in GCC 
> 5.0 (or even 4.6). [I have no idea of who will decide that, and I have 
> no idea of who decided that GCC can have plugins. Perhaps the Steering 
> Commitee, or RMS himself???

Not RMS for sure, and pretty much not the steering committee unless there 
is strong disagreement among the primarily responsible parties (= the 
technical maintainers).

> Are there any objective measures of the temperature of a mailing list 
> :-) :-) ? ? ?

Not that I'd know of, but the GCC lists are extremely harmless nearly all 
of the time (and this discussion is still quite on the harmless side of 
things).

> Now my mood is that is is quite funny to discuss all that. I am enjoying 
> it. :-) :-)

Happy to hear that. :-)

Gerald