"plugin"-ifying the MELT branch.

"plugin"-ifying the MELT branch.

[Basile STARYNKEVITCH]

Hello All,

To make MELT more interesting & more easy to use, I want to make it 
become a (big & meta *) plugin. I also need to document & illustrate it 
much more. I actually did start to work on the pluginification of MELT: 
I mean making MELT a real GCC plugin, not needing any core GCC patch 
anymore. The current snapshot (rev148175 of the MELT branch) already 
have some call to register_callback (with PLUGIN_GGC_MARK, so I don"t 
need to hack the GGC collector anymore.), even if MELT is not yet a real 
plugin.

I think that would be doable once gengtype is able to work in plugin 
mode, i.e. once the 
http://gcc.gnu.org/ml/gcc-patches/2009-06/msg00410.html patch is 
accepted into the trunk (or an equivalent way of doing it). AFAIK, once 
such a facility is provided, I could make MELT a plugin quite easily (by 
dropping some minor MELT functionalities, probably the compiler probe 
and the libtool support - ie using dlopen instead of the lt_dlopenext 
wrapper.


This brings some questions.

Can a branch be simply a plugin, or should I close (soon) the 
melt-branch and start a melt-plugin-branch on the SVN. If I do that, do 
I need some authorization? from whom?

Will there be a list of FSF-owned plugins (outside of the trivial ones 
for testing)? MELT could be an example of a non-trivial FSF plugin.

Perhaps we need on the SVN an additional class of sub-repositories. Not 
only branches and the trunk (and old releases) but also plugins....



I won't be able to attend the summit this year.

Regards.

Note *: MELT is a meta plugin in the sense that it generate C code, 
compile it, and dynamically load it. However, the MELT generated plugins 
do not follow the plugin API; they have their own ones!

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: "plugin"-ifying the MELT branch.

[Tom Tromey]

>>>>> "Basile" == Basile STARYNKEVITCH <basile@starynkevitch.net> writes:

Basile> Can a branch be simply a plugin, or should I close (soon) the
Basile> melt-branch and start a melt-plugin-branch on the SVN. If I do that,
Basile> do I need some authorization? from whom?

I think what you do on your branch is up to you.  If you want to
repurpose it to be a plugin branch, I don't think there's any problem
with that.

You can also start a new branch.  Any maintainer can start a new
branch for any reason.

I don't know the answer to your other questions.  I assume there will
be at least one plugin on gcc trunk, for testing purposes if nothing
else.

Tom

Re: "plugin"-ifying the MELT branch.

[Basile STARYNKEVITCH]

Basile STARYNKEVITCH wrote:
>
>
> Can a branch be simply a plugin, or should I close (soon) the 
> melt-branch and start a melt-plugin-branch on the SVN. If I do that, 
> do I need some authorization? from whom? 


Apparently, nothing very special is required to start a new branch. So I 
intend to create a new melt-plugin-branch in a few hours (or days), 
unless there  is some objections to it;I will progressively move code 
from the current MELT branch melt-branch into this melt-plugin-branch to 
make MELT a big plugin.

Also, does such a thing should be called a branch. After all, it is not 
a derivative of an existing trunk! Maybe it should be called something 
different!

This also brings interesting questions: how should plugins in general be 
structured? Do they need some autoconf hackery?

My goal is of course to make MELT a big (meta) plugin suitable to work 
with the standard trunk (ie future 4.5).

Regards.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: "plugin"-ifying the MELT branch.

[Richard Guenther]

On Tue, Jun 16, 2009 at 1:13 PM, Basile
STARYNKEVITCH<basile@starynkevitch.net> wrote:
> Basile STARYNKEVITCH wrote:
>>
>>
>> Can a branch be simply a plugin, or should I close (soon) the melt-branch
>> and start a melt-plugin-branch on the SVN. If I do that, do I need some
>> authorization? from whom?
>
>
> Apparently, nothing very special is required to start a new branch. So I
> intend to create a new melt-plugin-branch in a few hours (or days), unless
> there  is some objections to it;I will progressively move code from the
> current MELT branch melt-branch into this melt-plugin-branch to make MELT a
> big plugin.
>
> Also, does such a thing should be called a branch. After all, it is not a
> derivative of an existing trunk! Maybe it should be called something
> different!

I do not think that plugins should be branches of trunk, nor should
they live in the FSF repository.

Richard.

>
> This also brings interesting questions: how should plugins in general be
> structured? Do they need some autoconf hackery?
>
> My goal is of course to make MELT a big (meta) plugin suitable to work with
> the standard trunk (ie future 4.5).
>
> Regards.
>
> --
> Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
> email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
> 8, rue de la Faiencerie, 92340 Bourg La Reine, France
> *** opinions {are only mines, sont seulement les miennes} ***
>
>

Re: "plugin"-ifying the MELT branch.

[Basile STARYNKEVITCH]

Richard Guenther wrote:
> On Tue, Jun 16, 2009 at 1:13 PM, Basile
> STARYNKEVITCH<basile@starynkevitch.net> wrote:
>   
>> Basile STARYNKEVITCH wrote:
>>     
>>> Can a branch be simply a plugin, or should I close (soon) the melt-branch
>>> and start a melt-plugin-branch on the SVN. If I do that, do I need some
>>> authorization? from whom?
>>>       
>> Apparently, nothing very special is required to start a new branch. So I
>> intend to create a new melt-plugin-branch in a few hours (or days), unless
>> there  is some objections to it;I will progressively move code from the
>> current MELT branch melt-branch into this melt-plugin-branch to make MELT a
>> big plugin.
>>
>> Also, does such a thing should be called a branch. After all, it is not a
>> derivative of an existing trunk! Maybe it should be called something
>> different!
>>     
>
> I do not think that plugins should be branches of trunk, nor should
> they live in the FSF repository.
>   

I thought on the contrary that is was expected that some code would 
become FSF owned plugins?

BTW, since MELT is a branch, its code is FSF owned. So to become a 
plugin, it should be an FSF owned plugin!

In other words, are plugins only for code outside of GCC?

Regards.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: "plugin"-ifying the MELT branch.

[Joseph S. Myers]

On Tue, 16 Jun 2009, Basile STARYNKEVITCH wrote:

> I thought on the contrary that is was expected that some code would become FSF
> owned plugins?

Not without a mechanism for linking plugins in statically on hosts for 
which we don't support dynamic loading of plugins, and even then it's 
doubtful.  Rather, we should watch out for things being implemented as 
plugins that are generally useful for GCC and seek to build them into GCC 
(unconditionally) where appropriate, while leaving cases such as checking 
project-specific coding rules as separate plugins.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: "plugin"-ifying the MELT branch.

[Basile STARYNKEVITCH]

I (Basile) very probably misunderstood what Joseph Myers or Richard 
Guenther meant. What I might have [mis]understood scares me. This is a 
request for clarification.


Joseph S. Myers wrote:


> On Tue, 16 Jun 2009, Basile STARYNKEVITCH wrote:
>
>   
>> I thought on the contrary that is was expected that some code would become FSF
>> owned plugins?
>>     
>
> Not without a mechanism for linking plugins in statically on hosts for 
> which we don't support dynamic loading of plugins, and even then it's 
> doubtful.  
That mechanism already exists in libltdl (the libtool wrapper of dlopen).

However, I am not sure to understand the logic here. Plugins are by 
definition optional stuff, and I understood from the beginning that 
plugins are considered only on machines which have a way of dynamically 
loading code (currently, the documented constraint is even stronger: 
dlopen & -rdynamic).

> Rather, we should watch out for things being implemented as 
> plugins that are generally useful for GCC and seek to build them into GCC 
> (unconditionally) where appropriate, while leaving cases such as checking 
> project-specific coding rules as separate plugins.
Again, I don't understand the rationale here.

My broad feeling was that plugin feature is for code which could 
interest some people, but does not interest every GCC user. (and MELT, 
or even ICI or TreeHydra, fits the definition).

In particular, there would probably be several plugins which give some 
extra feedback to the developers using them, but do not modify the code 
generation behavior of GCC.

Did I understood that in your view no branch hosted on GCC SubVersion 
should provide plugins? Why? Is it only your view, or some decision by 
some powerful guys (e.g. the Steering Committee)? Did the MELT branch 
[*] suddenly become illegal without me knowing about that? That would be 
ironical for a branch which happened -with other branches & people- to 
have pushed the idea of plugins!

Is there some [political?] impossibility for FSF copyrighted GPLv3 code 
(like those sitting in branches, e.g. the MELT one) to become plugins? I 
thought that becoming GPLv3/FSF plugins is an additional natural path 
for code sitting in branched to become accepted in the trunk!

I suppose these things has been discussed at the GCC summit a few days 
ago? What has been discussed & decided?

This surprises me a big lot. I thought on the contrary that specialized 
plugins would be used inside GCC in the future (for GCC development). To 
be more concrete, one could imagine a plugin to check all the error & 
warning messages inside GCC for validity (attribute printf is not fully 
adequate for that purpose). And my interpretation of GTY as attribute 
discussion was that someone is dreaming to replace gengtype, in a 
distant future, by some plugin providing the same behavior as gengtype 
(there is a bootstrap chicken&egg issue in that case, but one could 
easily store the generated gt-*.h file in the source tree, as it is 
already done for autoconf stuff today).

Is there some new prohibition on FSF copyrighted GPLv3 licenced code 
(inside branches) providing plugins? Or did I (hopefully) misunderstood?

Can a branch only (or mostly) provide a plugin? If not, why? If a branch 
cannot provide a plugin, who, when was decided such a major decision? I 
feel such a decision fully in contradiction with the idea of accepting 
plugins in GCC.

Please take time to explain, and remember that I am not an English 
native speaker, that I am not familiar with the US law system or the 
American corporate culture, and that MELT branch was always designed 
with meta-programming & dlopening generated code in mind. MELT has 
absolutely no sense on system without dlopen (or an equivalent 
functionality. So far, MELT is using ltdl).

Regards.

Note *: the MELT branch always provided a plugin mechanism. It is 
essential to MELT to generate C code and run it. I always said that 
dlopen is essential to MELT

PS. My understanding of the runtime license exception discussion last 
year was that the FSF & the SC wanted to promote the idea of GPLv3 
licencsd plugins [and of course restrain proprietary plugins] not to 
discourage them (and rejecting the idea of FSF copyrighted GPLv3 
licenced plugins might not be perceived as encouraging GPLv3 plugins).

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: "plugin"-ifying the MELT branch.

[Richard Guenther]

On Tue, Jun 16, 2009 at 2:22 PM, Basile
STARYNKEVITCH<basile@starynkevitch.net> wrote:
>
> I (Basile) very probably misunderstood what Joseph Myers or Richard Guenther
> meant. What I might have [mis]understood scares me. This is a request for
> clarification.
>
>
> Joseph S. Myers wrote:
>
>
>> On Tue, 16 Jun 2009, Basile STARYNKEVITCH wrote:
>>
>>
>>>
>>> I thought on the contrary that is was expected that some code would
>>> become FSF
>>> owned plugins?
>>>
>>
>> Not without a mechanism for linking plugins in statically on hosts for
>> which we don't support dynamic loading of plugins, and even then it's
>> doubtful.
>
> That mechanism already exists in libltdl (the libtool wrapper of dlopen).
>
> However, I am not sure to understand the logic here. Plugins are by
> definition optional stuff, and I understood from the beginning that plugins
> are considered only on machines which have a way of dynamically loading code
> (currently, the documented constraint is even stronger: dlopen & -rdynamic).
>
>> Rather, we should watch out for things being implemented as plugins that
>> are generally useful for GCC and seek to build them into GCC
>> (unconditionally) where appropriate, while leaving cases such as checking
>> project-specific coding rules as separate plugins.
>
> Again, I don't understand the rationale here.
>
> My broad feeling was that plugin feature is for code which could interest
> some people, but does not interest every GCC user. (and MELT, or even ICI or
> TreeHydra, fits the definition).
>
> In particular, there would probably be several plugins which give some extra
> feedback to the developers using them, but do not modify the code generation
> behavior of GCC.
>
> Did I understood that in your view no branch hosted on GCC SubVersion should
> provide plugins? Why? Is it only your view, or some decision by some
> powerful guys (e.g. the Steering Committee)? Did the MELT branch [*]
> suddenly become illegal without me knowing about that? That would be
> ironical for a branch which happened -with other branches & people- to have
> pushed the idea of plugins!

As you already said, a plugin is not a branch of trunk and so it should
not be a branch of trunk.  There is no way a user with usual SVN
write access can (or should) add a new repository to host a plugin
on gcc.gnu.org.  Instead I suggest that plugins be hosted at
whatever convenient public repository hosting site.

Just because it does not make technical sense.  No "powerful"
entities are involved here.

Richard.

Re: "plugin"-ifying the MELT branch.

[Joseph S. Myers]

On Tue, 16 Jun 2009, Basile STARYNKEVITCH wrote:

> > > I thought on the contrary that is was expected that some code would become
> > > FSF
> > > owned plugins?
> > >     
> > 
> > Not without a mechanism for linking plugins in statically on hosts for which
> > we don't support dynamic loading of plugins, and even then it's doubtful.  
> That mechanism already exists in libltdl (the libtool wrapper of dlopen).

As already discussed at length, libltdl is liable to make building GCC and 
making effectively redistributable binaries without problematic 
dependencies much harder and so using it would be a bad idea.

> However, I am not sure to understand the logic here. Plugins are by definition
> optional stuff, and I understood from the beginning that plugins are
> considered only on machines which have a way of dynamically loading code
> (currently, the documented constraint is even stronger: dlopen & -rdynamic).

Making features presently supported by GCC optional and completely 
unavailable on many host platforms would be a clear regression.

> > Rather, we should watch out for things being implemented as plugins that are
> > generally useful for GCC and seek to build them into GCC (unconditionally)
> > where appropriate, while leaving cases such as checking project-specific
> > coding rules as separate plugins.
> Again, I don't understand the rationale here.
> 
> My broad feeling was that plugin feature is for code which could interest some
> people, but does not interest every GCC user. (and MELT, or even ICI or
> TreeHydra, fits the definition).

The criterion for being appropriate to include in GCC is different from 
interesting every GCC user.  Nothing interests every GCC user.

> In particular, there would probably be several plugins which give some extra
> feedback to the developers using them, but do not modify the code generation
> behavior of GCC.

If the feedback relates to the use of a particular library not provided by 
GCC, for example, that is probably an appropriate case for a plugin 
provided by that library rather than by GCC.  If it is generally relevant 
to users of a particular language feature or feature in a language's 
standard library, like existing warnings in GCC, it is probably an 
appropriate case for code in GCC proper.

> Did I understood that in your view no branch hosted on GCC SubVersion should
> provide plugins? Why? Is it only your view, or some decision by some powerful

I did not say anything about what branches should provide.  My comments 
are about the question of whether feature X should go in trunk or in a 
separate plugin, given that there is an implementation that is or can be 
assigned to the FSF and that is technically ready to go in trunk.

> This surprises me a big lot. I thought on the contrary that specialized
> plugins would be used inside GCC in the future (for GCC development). To be
> more concrete, one could imagine a plugin to check all the error & warning
> messages inside GCC for validity (attribute printf is not fully adequate for
> that purpose). And my interpretation of GTY as attribute discussion was that

I could indeed imagine that those warnings that are present in GCC only 
for use when building GCC might usefully become plugins - but not without 
being able to build them in statically when bootstrapping on a platform on 
which plugins are not supported.  This would be most useful if the plugins 
supported multiple host GCC versions so that the host GCC 4.5 building a 
4.6 cross compiler could load the plugin for the version of checking 
relevant for building GCC 4.6.  This is the special case above of code 
only useful for compiling one particular piece of software (where that one 
piece of software just happens to be GCC).

> someone is dreaming to replace gengtype, in a distant future, by some plugin
> providing the same behavior as gengtype (there is a bootstrap chicken&egg
> issue in that case, but one could easily store the generated gt-*.h file in
> the source tree, as it is already done for autoconf stuff today).

It would be better to have fewer generated files in the source tree and 
fewer bootstrap dependencies or dependencies for modifying GCC and 
building a modified version, not more.  (I also don't know what host and 
target dependencies those files might have.)  Portable tools that can be 
built using any ISO C compiler are far better than depending on 
complicated host-specific features of a particular version of GCC.  If we 
move to C++ as implementation language I believe we should keep the 
ability to build with non-GCC ISO C++ compilers and with a wide range of 
GCC versions.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: "plugin"-ifying the MELT branch.

[Janis Johnson]

On Tue, 2009-06-16 at 14:22 +0200, Basile STARYNKEVITCH wrote:
> I (Basile) very probably misunderstood what Joseph Myers or Richard 
> Guenther meant. What I might have [mis]understood scares me. This is a 
> request for clarification.

> Did I understood that in your view no branch hosted on GCC SubVersion 
> should provide plugins? Why? Is it only your view, or some decision by 
> some powerful guys (e.g. the Steering Committee)? Did the MELT branch 
> [*] suddenly become illegal without me knowing about that? That would be 
> ironical for a branch which happened -with other branches & people- to 
> have pushed the idea of plugins!
> 
> Is there some [political?] impossibility for FSF copyrighted GPLv3 code 
> (like those sitting in branches, e.g. the MELT one) to become plugins? I 
> thought that becoming GPLv3/FSF plugins is an additional natural path 
> for code sitting in branched to become accepted in the trunk!

I don't think that the responses have answered Basile's concerns.

Basile, people are saying that MELT no longer belongs in a branch of
the GCC repository because now that plug-ins are supported, MELT no
longer needs to modify GCC itself and can be maintained independently.
That does not mean that MELT cannot be assignd to the FSF (you should
probably talk to the FSF about that) and certainly doesn't mean that
it can't use GPLv3.  It also doesn't mean that you should stop being
part of the GCC community.  All it means is that the MELT sources
should now be hosted elsewhere as a separate project.

Richi and Joseph, do you agree with that?

Janis

Re: "plugin"-ifying the MELT branch.

[Diego Novillo]

On Tue, Jun 16, 2009 at 13:10, Janis Johnson<janis187@us.ibm.com> wrote:

> Basile, people are saying that MELT no longer belongs in a branch of
> the GCC repository because now that plug-ins are supported, MELT no
> longer needs to modify GCC itself and can be maintained independently.
> That does not mean that MELT cannot be assignd to the FSF (you should
> probably talk to the FSF about that) and certainly doesn't mean that
> it can't use GPLv3.  It also doesn't mean that you should stop being
> part of the GCC community.  All it means is that the MELT sources
> should now be hosted elsewhere as a separate project.

Agreed.

Additionally, I chatted with Taras last week at the summit about the
various plugins that people have developed.  I will be creating a
landing wiki page where people can add links to plugins they have
found or developed.  This could be a good place where to add links to
MELT.


Diego.

Re: "plugin"-ifying the MELT branch.

[Basile STARYNKEVITCH]

Diego Novillo wrote:
> On Tue, Jun 16, 2009 at 13:10, Janis Johnson<janis187@us.ibm.com> wrote:
>
>   
>> Basile, people are saying that MELT no longer belongs in a branch of
>> the GCC repository because now that plug-ins are supported, MELT no
>> longer needs to modify GCC itself and can be maintained independently.

MELT is not pluginified yet. So it will stay some time as a branch, 
because today it cannot be a plugin (for various small technical details).

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: "plugin"-ifying the MELT branch.

[Joseph S. Myers]

On Tue, 16 Jun 2009, Janis Johnson wrote:

> Basile, people are saying that MELT no longer belongs in a branch of
> the GCC repository because now that plug-ins are supported, MELT no
> longer needs to modify GCC itself and can be maintained independently.
> That does not mean that MELT cannot be assignd to the FSF (you should
> probably talk to the FSF about that) and certainly doesn't mean that
> it can't use GPLv3.  It also doesn't mean that you should stop being
> part of the GCC community.  All it means is that the MELT sources
> should now be hosted elsewhere as a separate project.
> 
> Richi and Joseph, do you agree with that?

I have not been saying anything about whether or not MELT belongs in a 
branch of the GCC repository or where plugins should be hosted.  My 
comments have been about general questions of what should or should not be 
a plugin rather than about what should be hosted where.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: "plugin"-ifying the MELT branch.

[Richard Guenther]

On Tue, Jun 16, 2009 at 7:30 PM, Basile
STARYNKEVITCH<basile@starynkevitch.net> wrote:
> Diego Novillo wrote:
>>
>> On Tue, Jun 16, 2009 at 13:10, Janis Johnson<janis187@us.ibm.com> wrote:
>>
>>
>>>
>>> Basile, people are saying that MELT no longer belongs in a branch of
>>> the GCC repository because now that plug-ins are supported, MELT no
>>> longer needs to modify GCC itself and can be maintained independently.
>
> MELT is not pluginified yet. So it will stay some time as a branch, because
> today it cannot be a plugin (for various small technical details).

My comment was to the new "branch" you were about to create where the
MELT plugin will materialize.  And yes, Janis got it right.

Richard.

Re: "plugin"-ifying the MELT branch.

[Basile STARYNKEVITCH]

Richard Guenther wrote:
> On Tue, Jun 16, 2009 at 7:30 PM, Basile
> STARYNKEVITCH<basile@starynkevitch.net> wrote:
>   
>> Diego Novillo wrote:
>>     
>>> On Tue, Jun 16, 2009 at 13:10, Janis Johnson<janis187@us.ibm.com> wrote:
>>>
>>>
>>>       
>>>> Basile, people are saying that MELT no longer belongs in a branch of
>>>> the GCC repository because now that plug-ins are supported, MELT no
>>>> longer needs to modify GCC itself and can be maintained independently.
>>>>         
>> MELT is not pluginified yet. So it will stay some time as a branch, because
>> today it cannot be a plugin (for various small technical details).
>>     
>
> My comment was to the new "branch" you were about to create where the
> MELT plugin will materialize.  And yes, Janis got it right.

I understood that I should not create a new branch for MELT as a plugin. 
So I won't, and the pluginification of MELT becomes less a priority for 
me (at least for the moment; it might change in 2010.).

Honestly, I was surprised by the reactions to my email. Very probably a 
cultural problem, or perhaps a different perception of why plugins could 
be useful inside GCC.

Regards.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***