Re: GCC 5 & modularity

Re: GCC 5 & modularity

[Basile Starynkevitch]

[-- Attachment #1: Type: text/plain, Size: 2771 bytes --]

On Tue, 20 Mar 2012 18:39:40 +1000
Peter Dolding <oiaohm@gmail.com> wrote:
> 
> The top level modules already exist and are named. 


Not really. I see nowhere on the GCC site a picture as clear as 
the "plateform overview" figure on http://developer.gnome.org/
And I am not able to list and name corresponding modules.
If you do, please send a link or make an exhaustive list.

The internal executables like cc1 are too high.

For example, my feeling is that GCC (actually cc1) is made of three layers

front-ends  |  middle-end    |  back-ends

and I might consider that the middle-end contains perhaps the following modules

   the pass manager
   the simple gimple passes
   the IPA gimple passes

(but I am not even sure).

I really would like some global reviewer propose a whole list of (say a dozen or two, but
no more) modules to start the discussion.

If we had a picture like the platform overview picture for Gnome/GTK we could tell that
proeminently on our web site.

In some of the slides on http://gcc-melt.org/ you could find the attached picture (I
don't know if gcc@gcc.gnu.org accept attachments, apparently not in PDF) which you could
browse at http://gcc-melt.org/cc1-internals.pdf or http://gcc-melt.org/cc1-internals.svg
(CC BY SA license)

And this picture contain no module names, because I don't know of any. It is sadly not
enough precise because I am unable to list the set of modules making GCC (or just cc1)

Most importantly, no text inside our source distribution or our SVN repository seems to
list clearly an exhaustive list of *named* modules, and for each module the directory (or
set of source file names) making that module.

At last, I do feel important that every source file should be documented as part of a
given named module (e.g. by a comment just after the GPLv3 license paragraph), and
preferably that we would use some device -either a prefix à le GTK, or just a C++
namespace to remind to which module any public identifier belongs.

We have none of them. Actually, I even don't understand if we have a tentative list of
(one or two dozen) modules; with a name and a one sentence description or title. I am
not aware of such a list. If you have it, please publish it (or a link to it). And if
that list already exist, I strongly suggest that it should be added into the svn trunk
repository, as a README-GCC-MODULES file or whatever.


If you have an exhaustive list of modules making cc1, please tell.

Regards.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***

[-- Attachment #2: cc1-internals.svg --]
[-- Type: image/svg+xml, Size: 131101 bytes --]

Re: GCC 5 & modularity

[Peter Dolding]

On Wed, Mar 21, 2012 at 3:47 AM, Basile Starynkevitch
<basile@starynkevitch.net> wrote:
> On Tue, 20 Mar 2012 18:39:40 +1000
> Peter Dolding <oiaohm@gmail.com> wrote:
>>
>> The top level modules already exist and are named.
>
>
> Not really. I see nowhere on the GCC site a picture as clear as
> the "plateform overview" figure on http://developer.gnome.org/
> And I am not able to list and name corresponding modules.
> If you do, please send a link or make an exhaustive list.
>
> The internal executables like cc1 are too high.
>
> For example, my feeling is that GCC (actually cc1) is made of three layers
>
> front-ends  |  middle-end    |  back-ends

Back-ends are shared between all languages gcc supports but can be
target particular..  So they are already a kind of module in there own
right.

Middle-end is language neutral.  Yes these are basically already a
module just not broken down enough.

front-ends part in cc1 is the only part that is fully unique to cc1.
The middle-end and back-ends you will find used with other languages
as is.

This here is a very good read.
http://www.cse.iitb.ac.in/grc/intdocs/gcc-conceptual-structure.html

For the official harder to read form
http://gcc.gnu.org/onlinedocs/gccint.pdf   basically this really needs
some graphics added.  And a little sorting.  But the same information
is in there as the gcc conceptual structure write up.

Even better gccint.pdf goes into what c files should be in each module.

This is why you hitting the wall that gcc is already kinda module
design.  Everything is sorted to go into modules is just starting
cutting.

Note I said exist and named.  I did not say they were into a simple to
read graphic on the gcc site.

Peter Dolding

Re: GCC 5 & modularity

[Richard Guenther]

On Wed, Mar 21, 2012 at 10:36 AM, Peter Dolding <oiaohm@gmail.com> wrote:
> On Wed, Mar 21, 2012 at 3:47 AM, Basile Starynkevitch
> <basile@starynkevitch.net> wrote:
>> On Tue, 20 Mar 2012 18:39:40 +1000
>> Peter Dolding <oiaohm@gmail.com> wrote:
>>>
>>> The top level modules already exist and are named.
>>
>>
>> Not really. I see nowhere on the GCC site a picture as clear as
>> the "plateform overview" figure on http://developer.gnome.org/
>> And I am not able to list and name corresponding modules.
>> If you do, please send a link or make an exhaustive list.
>>
>> The internal executables like cc1 are too high.
>>
>> For example, my feeling is that GCC (actually cc1) is made of three layers
>>
>> front-ends  |  middle-end    |  back-ends
>
> Back-ends are shared between all languages gcc supports but can be
> target particular..  So they are already a kind of module in there own
> right.
>
> Middle-end is language neutral.  Yes these are basically already a
> module just not broken down enough.
>
> front-ends part in cc1 is the only part that is fully unique to cc1.
> The middle-end and back-ends you will find used with other languages
> as is.
>
> This here is a very good read.
> http://www.cse.iitb.ac.in/grc/intdocs/gcc-conceptual-structure.html
>
> For the official harder to read form
> http://gcc.gnu.org/onlinedocs/gccint.pdf   basically this really needs
> some graphics added.  And a little sorting.  But the same information
> is in there as the gcc conceptual structure write up.
>
> Even better gccint.pdf goes into what c files should be in each module.
>
> This is why you hitting the wall that gcc is already kinda module
> design.  Everything is sorted to go into modules is just starting
> cutting.
>
> Note I said exist and named.  I did not say they were into a simple to
> read graphic on the gcc site.

Indeed.  There is also different module hierarchies that overlap.  For
example ILs used in the different parts of the compiler.

I think Basile is mostly confused about what files belong to what module,
a question with not a single answer per file.  It's been suggested before,
but moving files into subdirectories might help here, at least for those
which have a clearly defined module.  The C frontend is one obvious
example, generator programs for machine descriptions another,
RTL optimization passes are harder, GIMPLE/SSA optimization
passes are somewhat hard, ... there is not always a good directory
to put things in.  For example I absolutely hate the suggested gimple/

Richard.

> Peter Dolding

Re: GCC 5 & modularity

[Peter Dolding]

On Wed, Mar 21, 2012 at 7:57 PM, Richard Guenther
<richard.guenther@gmail.com> wrote:

> Indeed.  There is also different module hierarchies that overlap.  For
> example ILs used in the different parts of the compiler.
>
> I think Basile is mostly confused about what files belong to what module,
> a question with not a single answer per file.  It's been suggested before,
> but moving files into subdirectories might help here, at least for those
> which have a clearly defined module.  The C frontend is one obvious
> example, generator programs for machine descriptions another,
> RTL optimization passes are harder, GIMPLE/SSA optimization
> passes are somewhat hard, ... there is not always a good directory
> to put things in.  For example I absolutely hate the suggested gimple/
>
> Richard.

I see all the IL as storage/modules.  That transfer data between modules.

Breaking into modules does not mean a module cannot be dependant on a
different one.

So we could have a stack of modules dependant on particular IL module.

So Generic IL, Gimple IL, RTL IL would have modules in there own right
containing the code shared between optimisers and the like.

SSA processing of Gimple if the functions it using is used by nothing
else its SSA code.

Taking what gcc is and making into modules is just takes obeying a few
simple rules.

1 imports into your section have to come from other modules not lossy
shared files.(current confusion is lossy shared files)
2 there is no such thing as grey.  If a file is shared between two
modules it must be in a third module.   If file exists in both modules
you have split incorrectly.

If there is a grey area you have broken into modules wrong.  Module
based code has no grey.  No grey equals less errors of person editing
a file they think is exclusive to some part that turns out to be
shared with another part so causing nasty failure.

Like Gimplification has to be dependant on two modules for C.  Generic
IL and Gimple IL.  Most diagrams currently try to put it either in
front end or middle end.  When in fact its the bridge module between
front end and back end and its accessing data from both sides.

Once you start breaking modules correctly.  SSA is simple its just SSA.

With correct modules knowing what you are going to ruin by touching a
file become simpler.

So yes there should be a gimple/ directory but it should only contain
the shared functions and the internal code those shared functions use
to drive the gimple IL.  Repeat the same for RTL and Simple.

If you have broken everything right you should basically be able to
pull off building all the modules as .so/.dll files bar the core
application and have application work.

There should be a single answer per file Richard Guenther.  It is
critical there is.  So you know what you are working on.

Of course there are going to be a few files as cutting starts where if
we do a detailed design straight up when cutting starts its going to
be o bugger we need to create another module here so the code base is
clean.

We have the basic overview.  The question is were to start cutting.
Cutting front end stuff that is obvious would be a very good starting
point.

I don't expect gcc hierarchies to be a nice clean shape.  But it
should be able to be made cleaner.  So if I want to adjust like a
front end I know I am adjusting just the front end not something else.
 Same with SSA that I am fixing a bug in SSA I know that the files I
am currently touching SSA only uses.

Finally I know if I am playing in any of the IL directories I am
playing with fire.

Of course with SSA it might have subdirectories under it for code
sections that can be edited without effecting everything.   This is
where breaking into modules is good you can see how much risk you are
taking.

Personally in my eyes it time to start cutting into directories and
shaping the code up to neat.  Remove the left over layout from prior
to IL being used a lot inside gcc.

Basically we have the overview roughly how the code should look when
modules.  The final overview of the code as modules can only be done
after its chopped up into modules.   There is no requirement to chop
it all at once.  Trying to chop it all at once may not give enough
time to work out how to correctly cut any overlapping parts that turn
up.

So realistic goal should be a particular area chopped correctly.  Each
chop equals altering makefiles and moving files.  Slow can careful is
required when doing this is required.  Each chop could bring failure.
A chop per point release most likely would be too careful.

Basically Gcc 5=modular should a little of an extrema goal when we
have not made a decent percentage modular yet.   We have to get to
proper half or more modules before we really should set a goal like
this.  Once there is a clearer idea of the issues that will happen.

Peter Dolding

Re: GCC 5 & modularity

[Basile Starynkevitch]

On Wed, Mar 21, 2012 at 10:57:08AM +0100, Richard Guenther wrote:
> 
> Indeed.  There is also different module hierarchies that overlap.  For
> example ILs used in the different parts of the compiler.
> 
> I think Basile is mostly confused about what files belong to what module,
> a question with not a single answer per file.  It's been suggested before,


Sorry people, we don't have any established list of named modules. I see
nowhere a list of one or two dozens of modules with for each of them:

  * a name

  * short description in one or two sentences

  * the entire set of files or directories implementing that module

  * the API of that module (perhaps as a public header file, with the strong
    requirement that internal functions should be declared in a private header
    file)

Again, I take as a good example GTK/Gnome. It has a documented set of named
modules, there is a figure explaining their organization and a table listing
all of them, and each module has a documentation and a set of header files
defining its interface. It has also a naming convention: in C, all the
public identifiers of a module starts by the same prefix (e.g.
GtkSourceView); in C++ the interface is inside a single C++ namespace. At
last, it has a crystal clear file heararchy which, for a given file, makes
finding to which module it belongs very easy (often the parent or
grand-parent directory gives the name of the module).

So far, I was not even able to find a list of modules names of GCC. and for
each module, the set of source files implementing it (and the set of public
header files defining its interface at the very least).

And no reply even mentionned any module names. We badly need that some
people with a *global* understanding of GCC (these are few, I am not of
them) propose a list of module *names* to be discussed. 


For exmpla, I have absolutely no idea if the register allocator is one
module (or perhaps two, and probably none) or if it is just part of the
backend RTL passes


I'm quite surprised no one mentionned how it is difficult to explain GCC to
a newscomer (e.g. some student wanting to work on GCC within a GSOC). This
is partly the case because we cannot make a short list of modules. And to
ease the understanding effort, that list should be reasonably short. We
should have a list of 7-30 modules at most (I agree that some of these
modules could be further defined as made of several sub-modules). And each
of these modules should have a unique name. Each of these modules should
also have a small "title" sentende describint what it does (in a dozen
English words). At last, such a description should be part of the internal
documentation. http://gcc.gnu.org/onlinedocs/gccint/ should have a chapter
(probably after "3. http://gcc.gnu.org/onlinedocs/gccint/Interface.html")
perhaps titled "organization of GCC in toplevel-modules" explaining all
this.

We are far from that. We first need to discuss a set of proposed module
names and titles. (Unfortunately, I am not able to propose it, because I
don't have a global knowledge of GCC; I expecte the few "global reviewers"
to have enough such knowledge.)

Once we have a proposal for modules names and titles we can discuss them,
and dive more deaper into a modular organization of GCC. (I really think we
cannot discuss of things which stay ill-defined and un-named).

Regards.

PS. Again, I don't have a large enough knowledge of GCC to even propose a
list of modules.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: GCC 5 & modularity

[Pedro Lamarão]

Em 21/03/2012 08:58, Peter Dolding escreveu:

> If there is a grey area you have broken into modules wrong.  Module
> based code has no grey.  No grey equals less errors of person editing
> a file they think is exclusive to some part that turns out to be
> shared with another part so causing nasty failure.

This seems a Game of Words [tm].

Of course a modularized system can have a grey area. You just pack all 
of your gray area files and call them "grey module" to satisfy the 
bureaucracy.

There is no perfect taxinomy in this world. Searching for one is 
interesting; demanding one is useless.

--
  P.

Re: GCC 5 & modularity

[Richard Guenther]

On Wed, Mar 21, 2012 at 4:02 PM, Pedro Lamarão
<pedro.lamarao@prodist.com.br> wrote:
> Em 21/03/2012 08:58, Peter Dolding escreveu:
>
>
>> If there is a grey area you have broken into modules wrong.  Module
>> based code has no grey.  No grey equals less errors of person editing
>> a file they think is exclusive to some part that turns out to be
>> shared with another part so causing nasty failure.
>
>
> This seems a Game of Words [tm].
>
> Of course a modularized system can have a grey area. You just pack all of
> your gray area files and call them "grey module" to satisfy the bureaucracy.
>
> There is no perfect taxinomy in this world. Searching for one is
> interesting; demanding one is useless.

Very well said.  Discussing about modules also makes no sense.  Figure out
the present state.

Richard.

> --
>  P.
>

Re: GCC 5 & modularity

[Gabriel Dos Reis]

On Wed, Mar 21, 2012 at 10:03 AM, Basile Starynkevitch
<basile@starynkevitch.net> wrote:

> Sorry people, we don't have any established list of named modules. I see
> nowhere a list of one or two dozens of modules with for each of them:
>
>  * a name
>
>  * short description in one or two sentences
>
>  * the entire set of files or directories implementing that module
>
>  * the API of that module (perhaps as a public header file, with the strong
>    requirement that internal functions should be declared in a private header
>    file)


You appear to have a better understanding of what a module in GCC should be.
Would you mind leading the pack by sending patches for evaluation and review?
I am afraid an abstract lecture would not be enough.

-- Gaby

Re: GCC 5 & modularity

[Basile Starynkevitch]

On Wed, Mar 21, 2012 at 10:24:54AM -0500, Gabriel Dos Reis wrote:
> On Wed, Mar 21, 2012 at 10:03 AM, Basile Starynkevitch
> <basile@starynkevitch.net> wrote:
> 
> > Sorry people, we don't have any established list of named modules. I see
> > nowhere a list of one or two dozens of modules with for each of them:
> >
> >  * a name
> >
> >  * short description in one or two sentences
> >
> >  * the entire set of files or directories implementing that module
> >
> >  * the API of that module (perhaps as a public header file, with the strong
> >    requirement that internal functions should be declared in a private header
> >    file)
> 
> 
> You appear to have a better understanding of what a module in GCC should be.
> Would you mind leading the pack by sending patches for evaluation and review?
> I am afraid an abstract lecture would not be enough.
> 

I am not sure what you expect from me. As I said many times, I have not a
global understanding of GCC (the "global reviewers" have a much better
global understanding than I do). So I cannot propose or initiate a list of
modules.

Or do you want me to just propose a documentation patch with a canvas or
frame for other poeple to fill?

I would be happy to help, but please understand that my understanding of GCC
is restricted to gengtype, ggc, and some parts of the middle-end. I know
nothing about the vast rest of the GCC compiler.

Cheers.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: GCC 5 & modularity

[Peter Dolding]

On Thu, Mar 22, 2012 at 1:03 AM, Basile Starynkevitch
<basile@starynkevitch.net> wrote:
> On Wed, Mar 21, 2012 at 10:57:08AM +0100, Richard Guenther wrote:
>>
>> Indeed.  There is also different module hierarchies that overlap.  For
>> example ILs used in the different parts of the compiler.
>>
>> I think Basile is mostly confused about what files belong to what module,
>> a question with not a single answer per file.  It's been suggested before,
>
>
> Sorry people, we don't have any established list of named modules. I see
> nowhere a list of one or two dozens of modules with for each of them:
>
>  * a name
>
>  * short description in one or two sentences
>
>  * the entire set of files or directories implementing that module
>
>  * the API of that module (perhaps as a public header file, with the strong
>    requirement that internal functions should be declared in a private header
>    file)
>

We have more than two dozen.

http://gcc.gnu.org/onlinedocs/gccint.pdf  again every header and dot
point in section 9 would be a module.

Then there is the supporting bits to those modules.like the IL's and
the runtime.

Section 9 is basically the abridged overview.  So there will be a lot
more modules than what is listed.

Yes what you describing is a clean up of gccint documentation to be
more readable and practical.

Also you are describing the chopping process.

Yes we do have a list of what is what in documentation.  Is is
currently really nice to use no its not.  Its it fully detailed as it
should be no its not.   Is the problem huge yes it is.  Breaking into
modules gcc will have a more complex layout than gnome does.   Due to
the way thing interact.  Most likely will not fit well into a 2d
graphic.  Most likely will fit well into a 3d graphic.

So yes gccint is the starting block.  Some are nicely described with
what functions own to the module some are not.  Imports and exports
are not well declared.

It is going to have to be a piece by peice prcoess

Peter Dolding

Re: GCC 5 & modularity

[Jonathan Wakely]

On 21 March 2012 15:35, Basile Starynkevitch wrote:
>>
>
> I am not sure what you expect from me. As I said many times, I have not a
> global understanding of GCC (the "global reviewers" have a much better
> global understanding than I do). So I cannot propose or initiate a list of
> modules.
>
> Or do you want me to just propose a documentation patch with a canvas or
> frame for other poeple to fill?

Why not start a wiki page?  There's no need for patch approval and
anyone can make edits to improve it gradually over time.  If the
initial version is wrong someone can change it, but at least it will
have been started.

Re: GCC 5 & modularity

[Robert Dewar]

> Very well said.  Discussing about modules also makes no sense.  Figure out
> the present state.

these kind of meta discussions are very rarely of value, this
one is no exception IMO
>
> Richard.
>
>> --
>>   P.
>>

Re: GCC 5 & modularity

[Robert Dewar]

On 3/21/2012 11:35 AM, Basile Starynkevitch wrote:

> I would be happy to help, but please understand that my understanding of GCC
> is restricted to gengtype, ggc, and some parts of the middle-end. I know
> nothing about the vast rest of the GCC compiler.

Perhaps suggestions about improvements in the modularity of
gcc would be better left up to those who DO have a global
understanding of the existing structure of gcc.

Re: GCC 5 & modularity

[Peter Dolding]

On Thu, Mar 22, 2012 at 8:51 AM, Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
> On 21 March 2012 15:35, Basile Starynkevitch wrote:
>>>
>>
>> I am not sure what you expect from me. As I said many times, I have not a
>> global understanding of GCC (the "global reviewers" have a much better
>> global understanding than I do). So I cannot propose or initiate a list of
>> modules.
>>
>> Or do you want me to just propose a documentation patch with a canvas or
>> frame for other poeple to fill?
>
> Why not start a wiki page?  There's no need for patch approval and
> anyone can make edits to improve it gradually over time.  If the
> initial version is wrong someone can change it, but at least it will
> have been started.

Really it must happen at code level.  The advantage of modules is for
safer coding.

We have got stacks of docs written from the outside looking in.
Modules have to start in the code base.  Once it clear what is in a
module the module shape of the code base can be become stable.

gccint and work out a cut into a clear define module.  Create a
description for what has been cut.  Repeat.

Sooner or latter it will become module design.

The people who can review if a cut is sane are the ones working on the
code base.

This is the problem you can try to write a global list but until it in
the code base the global list will be open to change.

The gccint gives enough over view to start trying to plan a cut.

Really that the thing we need to focus on what will be the first module to make.

Peter Dolding

Re: GCC 5 & modularity

[Gabriel Dos Reis]

On Mon, Mar 19, 2012 at 12:56 AM, Basile Starynkevitch
<basile@starynkevitch.net> wrote:
> On Sun, 18 Mar 2012 20:49:24 +0000
> Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
>
>> On 18 March 2012 16:56, Basile Starynkevitch wrote:
>> >
>> >   * a garbage collector. Even a modular GCC need some memory management policy (and
>> > ref-counting à la GTK, or à la std::shared_ptr is not enough IMHO inside a compiler
>> > because a compiler has much more complex and circular data structures, and much less
>> > hierarchically organized, that a graphical tookit has).
>>
>> As has been pointed out to you before, shared_ptr is designed to be
>> useful even with circular structures. Please read about weak_ptr.
>
>
> Then why has it not being used before?

by whom? where?

-- Gaby

Re: GCC 5 & modularity

[Jonathan Wakely]

On Mar 19, 2012 5:56 AM, "Basile Starynkevitch" wrote:
>
> On Sun, 18 Mar 2012 20:49:24 +0000
> Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
>
> > On 18 March 2012 16:56, Basile Starynkevitch wrote:
> > >
> > >   * a garbage collector. Even a modular GCC need some memory management policy (and
> > > ref-counting à la GTK, or à la std::shared_ptr is not enough IMHO inside a compiler
> > > because a compiler has much more complex and circular data structures, and much less
> > > hierarchically organized, that a graphical tookit has).
> >
> > As has been pointed out to you before, shared_ptr is designed to be
> > useful even with circular structures. Please read about weak_ptr.
>
>
> Then why has it not being used before?

I don't understand the question.  Are you asking why shared_ptr hasn't
been used in GCC before?  Surely that's obvious, the GCC code is still
written in C and no C++ templates have been introduced to the code
yet.

Maybe shared_ptr isn't the right tool for memory management in GCC,
but if it's rejected then I hope it will be for valid reasons, not
because of misinformation about being unsuitable for code with
circular references.

Re: GCC 5 & modularity

[Basile Starynkevitch]

On Sun, 18 Mar 2012 20:49:24 +0000
Jonathan Wakely <jwakely.gcc@gmail.com> wrote:

> On 18 March 2012 16:56, Basile Starynkevitch wrote:
> >
> >   * a garbage collector. Even a modular GCC need some memory management policy (and
> > ref-counting à la GTK, or à la std::shared_ptr is not enough IMHO inside a compiler
> > because a compiler has much more complex and circular data structures, and much less
> > hierarchically organized, that a graphical tookit has).
> 
> As has been pointed out to you before, shared_ptr is designed to be
> useful even with circular structures. Please read about weak_ptr.


Then why has it not being used before?


-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***

Re: GCC 5 & modularity

[Jonathan Wakely]

On 18 March 2012 16:56, Basile Starynkevitch wrote:
>
>   * a garbage collector. Even a modular GCC need some memory management policy (and
> ref-counting à la GTK, or à la std::shared_ptr is not enough IMHO inside a compiler
> because a compiler has much more complex and circular data structures, and much less
> hierarchically organized, that a graphical tookit has).

As has been pointed out to you before, shared_ptr is designed to be
useful even with circular structures. Please read about weak_ptr.

Re: GCC 5 & modularity

[Basile Starynkevitch]

On Sun, 18 Mar 2012 13:11:05 -0400
Robert Dewar <dewar@adacore.com> wrote:

> On 3/18/2012 12:56 PM, Basile Starynkevitch wrote:
> 
> >     * you can name and count the modules of a software
> 
> Well in a hierarchical system this is not so clear, since modules may
> exist at different levels of abstraction. 

Agreed. Modules could contain submodules (but then I view this more as a language
feature, e.g. Ocaml's modules). I would like first that the GCC community sort-of agree
on a list of a dozen or two modules (call them super-modules or top-level modules if you
like). Obviously such modules would be quite large (since GCC is 5 or 10MLOC, having it
made of 30 modules mean that modules are several hundred thousand lines of source code
each, which means dozens of source files each). I don't think that we should organize GCC
as a large set of non-hierarchical modules: describing GCC as made of 200 top-level
modules won't help newbies to understand it; the description should speak of a
few dozen top-level parts 


> >
> >     * given a source line, or function, you can decide at a glance to which one module it
> > belongs
> 
> This seems totally bogus, if you have something like
> 
>      n++;
> 

I expressed wrongly myself. If you know that 
    n++;
was in file foo/bar.c line 34 [which I meant by a source line, not an isolated set of
tokens newline terminated] you should know that it belongs to some module. Actually, I
believe that modules are a large set of source files (e.g. if GCC was made of two dozens
of modules, each module would probably be made of some dozens of files), so the
requirement is becoming: every source file belongs to exactly one module, and every
module is made of a set of source files.

> you can't tell what module that belongs to, and if your idea is that
> all variables should be long enough to know immediately what module
> something is in, I would regard that as plain horrible and highly
> undesirable.

Long names are relevant only for exported names, usable outside of the module.

> >
> >     * the interface between modules is well documented
> 
> Sure that's apple pie and motherhood, so it says nothing

No, it implies that something which is not documented should not be used outside of the
module (irrespective of any protected: or private: annotations in C++) and this is a
strong coding rule (which is probably not enforced in GCC today). The documentation can
(and should) be generated (à la doxygen), which translates that requirement that only
API defined by structured comments understood by doxygen can be used outside of a module

> >
> > I'm sorry to say that, but current GCC (ie 4.7 or today's trunk) is *not* modular.
> 
> Modularity is not a binary quality, so this is not a helpful statement

IMHO modularity is an all or nothing quality. Our terminology differs!

> 
> > Don't
> > feel injured by that fact. Indeed, GCC is a little less messy than it was a few years
> > ago, but being less messy is not being modular IMHO. And something cannot be
> > "half-modular".
> 
> Absolutely it can, parts of the system can be arranged nicely into 
> modules, and parts of the system may not be.
> 
> > So I would be delighted if GCC was made of modules. But I have no idea of how that can be
> > done.
> 
> Then your comments are not at all helpful, since they just
> reflect vague goals which everyone agrees on.

No, I was expressing the wish that qualified people having a global understanding of GCC
(which I have not) would propose a well defined and small set of modules

> 
> > I do believe that identifiers in GCC should be organized in such a way that the module
> > they belong to is visible at once. I think that a prefix (à la GTK) or a C++ namespace
> > should be great. In particular, this means that most GCC identifiers should change
> > (which means that any such evolution is not syntactically gradual; it has to be made by
> > huge, but "easy", patches).
> 
> I am of the opinion that this would severely damage readability, it's 
> the same sort of thing that leads people in Ada to avoid use clauses
> completely.

I have absolutely no idea if the Ada parts of GCC can be made of several modules or not

> 
> Since in any decent IDE it's just a single click to find out where a 
> variable is declared, it's just noise to include this information in
> every variable name. Of course global variables with wide visibility
> should have appropriate names, but the idea that all identifiers
> should be prefixed is horrible IMO

Namespaces in C++ gives you exactly that. If we go the C++ route, why should we not use
namespaces for modules? If we stay in C, then we can have prefixes, and GTK experience
shows that it is helpful and not horrible (any decent editor has good completion),
contrarily to your claim (of course I am speaking only of exported identifiers, usable
from other modules). When you code in Gtk in C you know at a glance to which module a
given public identifier belongs (and in which source files you can find its
implementation). This helps a lot.

Regards.


-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***

Re: GCC 5 & modularity

[Robert Dewar]

On 3/18/2012 12:56 PM, Basile Starynkevitch wrote:

>     * you can name and count the modules of a software

Well in a hierarchical system this is not so clear, since modules may
exist at different levels of abstraction. For instance in a compiler,
at one level of abstraction, the front end is a module, at another
level of abstraction, e.g. in the compiler, the semantic analysis for
chapter 7 constructs in the RM could be considered as a module.
>
>     * given a source line, or function, you can decide at a glance to which one module it
> belongs

This seems totally bogus, if you have something like

     n++;

you can't tell what module that belongs to, and if your idea is that
all variables should be long enough to know immediately what module
something is in, I would regard that as plain horrible and highly
undesirable.
>
>     * the interface between modules is well documented

Sure that's apple pie and motherhood, so it says nothing
>
> I'm sorry to say that, but current GCC (ie 4.7 or today's trunk) is *not* modular.

Modularity is not a binary quality, so this is not a helpful statement

> Don't
> feel injured by that fact. Indeed, GCC is a little less messy than it was a few years
> ago, but being less messy is not being modular IMHO. And something cannot be
> "half-modular".

Absolutely it can, parts of the system can be arranged nicely into 
modules, and parts of the system may not be.

> So I would be delighted if GCC was made of modules. But I have no idea of how that can be
> done.

Then your comments are not at all helpful, since they just
reflect vague goals which everyone agrees on.

> I do believe that identifiers in GCC should be organized in such a way that the module
> they belong to is visible at once. I think that a prefix (à la GTK) or a C++ namespace
> should be great. In particular, this means that most GCC identifiers should change
> (which means that any such evolution is not syntactically gradual; it has to be made by
> huge, but "easy", patches).

I am of the opinion that this would severely damage readability, it's 
the same sort of thing that leads people in Ada to avoid use clauses
completely.

Since in any decent IDE it's just a single click to find out where a 
variable is declared, it's just noise to include this information in
every variable name. Of course global variables with wide visibility
should have appropriate names, but the idea that all identifiers
should be prefixed is horrible IMO

GCC 5 & modularity

[Basile Starynkevitch]

On Fri, 16 Mar 2012 19:41:56 +0000 (UTC)
"Joseph S. Myers" <joseph@codesourcery.com> wrote:

> On Fri, 16 Mar 2012, David Malcolm wrote:
> 
> > Proposed outcome
> [...]
> > Current architectural issues
> [...]
> 
> Not many people commented on the architectural goals document Diego and I 
> posted at <http://gcc.gnu.org/ml/gcc/2011-12/msg00103.html>.  Many of your 
> ideas seem potentially useful additions to it.  I hope Diego will have 
> time soon to push more of the goals and development conventions through 
> community approval and move them to the main website; as we do this, 
> detailed feedback from people interested in these issues is certainly 
> welcome.


I wanted many times to comment on these issues, but I am very scared of hurting people.
Last time I spoke of GCC modularity (actually lack of it), some people felt insulted,
which is certainly not my aim. So for many months I was too scared to comment on these
issues.

So if you feel insulted by any mine comments on these issues, please accept my apologies
and recall than I am not a native English speaker, nor was I educated in north American
universities or schools [except that I spent 1 year in California, as an 8 year old kid,
in 1967; but this is not enough a cultural influence, and it was really a long time
ago]. (I'm sure most people here would have difficulties in participating to mailing lists
in French or in Russian, so please be kind with me.)

First, my understanding of modules is that:

   * you can name and count the modules of a software

   * given a source line, or function, you can decide at a glance to which one module it
belongs

   * the interface between modules is well documented

I'm sorry to say that, but current GCC (ie 4.7 or today's trunk) is *not* modular. Don't
feel injured by that fact. Indeed, GCC is a little less messy than it was a few years
ago, but being less messy is not being modular IMHO. And something cannot be
"half-modular". 

A good modular software is the Gnome/GTK "graphical interface" system (ok, it is not a
compiler), and probably the KDE/Qt one. When you start learning it, you get a nice figure
http://developer.gnome.org/ and sort of list of modules
http://developer.gnome.org/platform-overview/stable/ ; we have absolutely nothing of that
sort for GCC today. We cannot even think of a similar figure for GCC today.


So I would be delighted if GCC was made of modules. But I have no idea of how that can be
done. I would like global GCC experts to propose an organization of modules. This means
that "global reviewers" type of experts would suggest at first:

 * A set of (a dozen or two) modules, each having a *name* and a short *description*
(perhaps a single title phrase at first).

 * An imprecise mapping of functionalities or features, or preferably of current source
files to modules

Don't expect non-global GCC maintainers to provide such a list; you need to have an
overall view of GCC to think of such a list; and very few people have such a global view;
I (Basile) certainly don't have it: there are lots of parts of GCC I don't know about! 
Of course, I may have an opinion on a particular proposed set of modules (but I don't
feel knowledgeable enough about all GCC to propose one).


 And making the current GCC core base modular is not easy (it might be impossible),
because we cannot decide at a glance to what module a given current code should belong.
An important thing if we want to go modular is to define a road-map and probably to
accept the fact that, if GCC 5 is a modular GCC, is will have less functionalities, less
optimizations, less power than the current GCC 4.7: we probably would have to
temporarily drop some feature or power from GCC to make it modular (otherwise, that would
be too gigantic an effort). I have no idea if we can do that (this is why I am sometimes
pessimistic); probably most GCC contributors are paid by companies which might not afford
that.

I do believe that identifiers in GCC should be organized in such a way that the module
they belong to is visible at once. I think that a prefix (à la GTK) or a C++ namespace
should be great. In particular, this means that most GCC identifiers should change
(which means that any such evolution is not syntactically gradual; it has to be made by
huge, but "easy", patches). 

I would prefer much more that tree-s would be named foobar::tree in C++ or foobar_tree in
C, where foobar is the name of the module defining trees and providing a API to
manipulate them. I strongly want that all GCC names would change in a more organized way.
It is a real pain, even with today's tools, to understand where are tree-s or edge-s or
basic_block-s defined.

GCC is not only a big bunch of source code, but also a set of "meta-programming" tools ie
a set of generators. And it is good we have them (I certainly don't think we should
remove them, and I am not naive enough to believe or want them to be replaced by tricky
templates in C++). Again, all GCC internal code generators should be listed (it is still
difficult to find an up-to-date list) and well documented.

My belief is that GCC should aim to be hosted on today's machines. This means that we
should use notably: shared libraries (we have almost none of them inside GCC) which can
map nicely to modules (like GTK does), and probably a richer system interface than what
the language standards provide. Libiberty is not enough: it cannot be even used by
plugins. Perhaps we could choose some foundational module (like perhaps Glib or something
else) providing small abstractions of system facilities. We also need GCC to really have
plugins (ie give up the idea of a plugin-less GCC, which is useless). And we should have
enough modularity so that each module could be extended or perhaps even superseded by an
external plugin. If modules are plugins or shared libraries, then extending or adding them
is easier, and working on GCC is faster.

We should also define a set of non-modular, ie global, features or traits for GCC. I am
in particular thinking about:

   * Naming conventions

   * a garbage collector. Even a modular GCC need some memory management policy (and
ref-counting à la GTK, or à la std::shared_ptr is not enough IMHO inside a compiler
because a compiler has much more complex and circular data structures, and much less
hierarchically organized, that a graphical tookit has). We should define a memory policy
in garbage collector's terms. (and ggc is a very bad GC, it should be improved, not
removed).

   * a set of conventions and a module for "dumping" (like our -fdump... today) and for
program arguments (ie optimization or other flags). I have myself not a precise
understanding of what -fdump means today... (just a feeling of it).

   * a module and set of conventions for diagnostic reporting to the user

   * the requirement that each major internal representation (like Gimple, Tree, ...)
should be dumpable, serializable in a textual way (perhaps JSON or YAML like), and
loadable (so we would have a parser able to construct Gimple in memory from the
serialized textual representation in a faithful manner).

   * a set of conventions regarding our meta-programming code generators (gengtype,
genattr, ..)

   * Meta-data about core types and modules is IMHO very important. The Gobject
introspection machinery of GTK3 made interfacing GTK to external software (in particular,
gluing it to interpreters) much much easier. We should probably have an equivalent thing:
a machine-level formalization of the main GCC APIs and the major GCC internal data. (This
would also help more robust plugins: it could query a particular GCC installation about
the feature it has).

    * Some documentation should be generated from source code (with a comment
convention), like doxygen or something else. (I hope that the license issue is solved on
this).


My feeling is that making GCC modular is a huge task (and we'll have to
accept that some current features would be first dropped to make it feasible). I have no
idea if it is realistic. (But I feel that in the long run, if GCC is remaining
non-modular, it will attract fewer and fewer new developers and gradually will become
less and less relevant).


My first wish is that someone (a "global reviewer" probably) would propose a tentative
list of modules to discuss.


I hope I did not hurt anyone. If I did, please accept my apologies. Feel free to ignore
that email.

Regards.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***