OpenMP, HPC, and the future of GCC

OpenMP, HPC, and the future of GCC

[Scott Robert Ladd]

Steven Bosscher wrote:
> Fact is that GCC is not a high-performance compiler for numerical
> applications: It does not do autovectorizing, and it's not easy to
> create parallel programs, etc.  Result is that when I compile a simple
> 1D Riemann solver with Intel C++, it runs more than twice as fast on my
> dual-P4 compared with GCC.  Differences: Vectorizing compiler with
> OpenMP support.

I am of the opinion that gcc needs a "high performance computing" initiative
to address the constellation of issues that surround numerical and "super"
computer applications.

If gcc wishes to address the needs of programmers in science and
engineering, we need to look at the entire constellation of features
required. That includes autovectorization, OpenMP, possibly
auto-parallelization, complete C99 support, Fortran 95, and maybe a GNU-MPI
or GNU-PVM implementation. All these pieces come together to provide a set
of tools for numerical work -- and the lack of individual components
(autovectorization, for example) undermines the effectiveness of gcc as a
whole in such applications.

> Maybe one of the reasons for this is because people involved in GCC
> development mostly are computer scientists, and that such people are not
> well known for understanding computational scientists?

Gcc is biased toward the needs of the GNU Project: writing command-line
tools and operating systems. Computational science simply hasn't entered the
picture as a primary player; most scientists of my acquaintance use gcc
because it is free-as-in-beer (especially true when it comes to g77). I
don't know anyone who advocates gcc as the most effective tool for numerical
work.

People should be aware that numerical applications transcend scientific
work. Some of the number-crunching code in my life has been in the financial
industry -- code that designs real-time portfolios, for example, based on
historical data analysis and user-supplied criteria and boundaries.

> And those numerical experts saw a trend: Everybody goes multi-processor!
> So they identified a *need* for an easy-to-use interface to create
> explicitly parallel software for people who are not computer
> scientists.  OpenMP was born, and is now widely used and well
> established.
>
> If GCC does not want to be useful in a hpc environment, that's fine.
> But I would like to hear that *before* we start an effort to implement
> OpenMP in GCC.  If the GCC community has an attitude like, "just put it
> in a branch and we'll see what we do with it," then I can think of more
> important things that I actually should put time in.

I agree. Good words.

Is GNU interested in supporting the HPC community? There are those of us
willing to put in the work if there is reasonable expectation that the gcc
"core" will accept it. But if HPC is going to be dismissed automatically, I
don't see the point of making an effort to implement HPC features.

One purported goal of the GNU Project is to provide free alternatives to
commercial products. Is GCC willing to surrender HPC to commercial compiler
vendors?

..Scott

--
Scott Robert Ladd
Coyote Gulch Productions (http://www.coyotegulch.com)
Professional programming for science and engineering;
Interesting and unusual bits of very free code.

Re: OpenMP, HPC, and the future of GCC

[Jan Hubicka]

> I agree. Good words.
> 
> Is GNU interested in supporting the HPC community? There are those of us

I think the question should be reversed.  Is HPC comunity interested in
supporting GNU?  I don't think anyone working on GCC would not like it
to be usefull for more people than it is now and generate better code.
To make this happen, someone must realize it.  GNU can't do it, someone
who actually needs the feature has to make this happen.

> willing to put in the work if there is reasonable expectation that the gcc
> "core" will accept it. But if HPC is going to be dismissed automatically, I

This is problematic.  Since GCC maintainers can not implement all the
HPC support themselves, they can not give in advance an promise that
they will accept the code.  Every new feature needs to be implemented
first and then it needs to be judged on how it works, how maintainable
the code is before it can be accepted for mainline GCC.

I personally would like to see GCC usefull for numeric computing too, so
I would like to help with the project if it is created as my time
allows.

Honza
> don't see the point of making an effort to implement HPC features.
> 
> One purported goal of the GNU Project is to provide free alternatives to
> commercial products. Is GCC willing to surrender HPC to commercial compiler
> vendors?
> 
> ..Scott
> 
> --
> Scott Robert Ladd
> Coyote Gulch Productions (http://www.coyotegulch.com)
> Professional programming for science and engineering;
> Interesting and unusual bits of very free code.

Re: OpenMP, HPC, and the future of GCC

[David Edelsohn]

>>>>> Scott Robert Ladd writes:

Scott> Is GNU interested in supporting the HPC community? There are those of us
Scott> willing to put in the work if there is reasonable expectation that the gcc
Scott> "core" will accept it. But if HPC is going to be dismissed automatically, I
Scott> don't see the point of making an effort to implement HPC features.

Scott> One purported goal of the GNU Project is to provide free alternatives to
Scott> commercial products. Is GCC willing to surrender HPC to commercial compiler
Scott> vendors?

	The GCC community welcomes all developers and contributions
consistent with the GCC Project and GNU Project.  If volunteers want to
tackle HPC features for GCC, there are no inherent reasons for that work
would be rejected.

	No one can say that an arbitrary implementation of those features
automatically will be accepted -- we need to see the patches.  I would
recommend doing the development in public and discussing the evolving
design on the GCC mailinglists.  Open development allows the GCC community
to see the design, provide feedback and make eventually accept the work
for merge into the official GCC sources.

Thanks, David

	P.S. Approaching this with a demanding, confrontational "us vs
them" attitude is not productive.

RE: OpenMP, HPC, and the future of GCC

[Scott Robert Ladd]

Jan Hubicka wrote
> > Is GNU interested in supporting the HPC community?
>
> I think the question should be reversed.  Is HPC comunity interested in
> supporting GNU?  I don't think anyone working on GCC would not like it
> to be usefull for more people than it is now and generate better code.
> To make this happen, someone must realize it.  GNU can't do it, someone
> who actually needs the feature has to make this happen.

The question is truly bidirectional; it is a matter of cooperation.

HPC receives a lot of help and cooperation from companies with a vested
interest in HPC. My question is: How much cooperation can the HPC community
expect from GCC? If people like me are willing to put in some of the work,
can GCC meet us part-way? Or is the GCC comunity so disinterested in HPC
that any work on our part would be futile?

I'm asking because I don't know the answers.

> This is problematic.  Since GCC maintainers can not implement all the
> HPC support themselves, they can not give in advance an promise that
> they will accept the code.  Every new feature needs to be implemented
> first and then it needs to be judged on how it works, how maintainable
> the code is before it can be accepted for mainline GCC.

As I have said before, I am not looking for GCC to give blanket pre-approval
to anything; Lord knows, that would be stupid. What I *am* asking for is an
open mind -- and several messages in this thread (*not* yours!) suggest
slammed doors.

> I personally would like to see GCC usefull for numeric computing too, so
> I would like to help with the project if it is created as my time
> allows.

And that's exactly what I'm looking for!

..Scott

Re: OpenMP, HPC, and the future of GCC

[Per Bothner]

Lars Segerlund wrote:
 > Perhaps you should take the time to get informed, before wasting
 > other peoples time

My point is that it is that the Gcc community is not responsible
for implementing OpenMP, and we are not required to merge in
an extension in to the main sources.  It is up to the OpenMP
community to produce an implementation, and if they want it
merged in, that it be is minimally disruptive to the cleanliness
(such as it is!) of Gcc.

I am making a point about *process*, primarily.  The *design* of
OpenMP also raises concerns, but that is a different issue.

 > also conisder that here are people willing to make an effort
 > to support 'multi threading' or 'native parallallism' in gcc,
 > which if you had a look at most modern systems might be of
 > major importance to gcc and the free community.

It *might* be.  However, there have been hundreds of designs
for language-based 'multi threading' or 'native parallallism'.
That suggests caution before making any major changes to our
code-base to support it.

 > Perhaps something as simple as reading openmp's homepage
 > should give a hint as to which institutions have contributed
 > to the standard,

So what?  All that means it's a standard with a lot of backing;
it means nothing about its quality.  For that the *people*
who contributed would be more important.

 > You would discover that a lot of people working on scientific
 > computing have been involved, and they can not all be idiots

Have I suggested otherwise?

Steven Bosscher wrote:

 > Maybe one of the reasons for [lack of numerical performance in Gcc]
 > is because people involved in GCC development mostly are computer
 > scientists, and that such people are not well known for understanding
 > computational scientists?

I think it is more a question of priorities.  Some people involved in
GCC development understand computations sciences very well, but most
do not.  But that I think is mostly because people have been very busy
on implementing language correctness, support many kinds of chips,
and implementing generally-useful improvements, so there hasn't been
much time (i.e. money!) for some of the things you're interested in.

To emphasize:  efforts to make Gcc better for "computational"
domains is very welcome.  It is just that any design needs to
avoid further complicating and slowing down Gcc.

Right now, there is a lot of concern and focus about the
compilation speed of Gcc, especially when not opimizing.
So it is very important that any new vectorization support
not slow the compiler down further, at least by default.

 > If GCC does not want to be useful in a hpc environment, that's fine.
 > But I would like to hear that *before* we start an effort to implement
 > OpenMP in GCC.  If the GCC community has an attitude like, "just
 > put it in a branch and we'll see what we do with it," then I can
 > think of more important things that I actually should put time in.

See Jan Hubicka's response.

Scott Robert Ladd wrote:

> Is GNU interested in supporting the HPC community? There are those of us
> willing to put in the work if there is reasonable expectation that the gcc
> "core" will accept it. But if HPC is going to be dismissed automatically, I
> don't see the point of making an effort to implement HPC features.

Nobody is talking about dismissing anything automatically.
However, we cannot accept patches we haven't seen, either.
I agree with Jan Hubicka's response.

> One purported goal of the GNU Project is to provide free alternatives to
> commercial products. Is GCC willing to surrender HPC to commercial compiler
> vendors?

GCC is Free Software.  You're free to use it as you will.  You're
free to create a fork.  We're free to reject your changes.

We don't want to "surrender HPC to commercial compiler vendors", but
it must be up to the HPC community to *become part* of the Gcc
community - we don't have the resources otherwise.  And that means
understanding the many contrary goals we have.  Runtime performance
is only one, and (at this point) not the most important one.
Maintainable code, compile-time speed, and correctness are at
least as important.

 > If people like me are willing to put in some of the work,
 > can GCC meet us part-way?

Yes, but resources are limited.  We can evaluate and advise,
but we cannot promise any implementation or debugging work,
unless you come up with funding that would allow one or more
expierences Gcc people to spend a substantial amount of time
on the project.  Otherwise, you have to learn how gcc works,
both technically, and the process.

 > What I *am* asking for is an open mind -- and several messages
 > in this thread (*not* yours!) suggest slammed doors.

I think we're willing to hold our noses about design choices
of OpenMP, as long as the *implementation* is as clean as
you/we can make it.  For example vectorization improvements
should avoid being tied to OpenMP specifics - in fact it is
critical that *algorithms* and *syntax* be kept separate.
-- 
	--Per Bothner
per@bothner.com   http://www.bothner.com/per/

Re: OpenMP, HPC, and the future of GCC

[Neil Booth]

Per Bothner wrote:-

> To emphasize:  efforts to make Gcc better for "computational"
> domains is very welcome.  It is just that any design needs to
> avoid further complicating and slowing down Gcc.
> 
> Right now, there is a lot of concern and focus about the
> compilation speed of Gcc, especially when not opimizing.
> So it is very important that any new vectorization support
> not slow the compiler down further, at least by default.

I'd like to emphasize this too, particularly the first paragraph, if
I may.  The best way to get something into GCC like we're discussing
here is to make it as clean and as unintrusive as possible.

Discussions over the last few months on these lists have pointed out how
various architectural aspects of GCC need to change.  One thing I would
like to see happen is a cleaner separation of front end and middle end,
which is why I'm writing my parser with a C-specific IR that is not trees.
We must try as hard as possible to avoid making architectural improvements
harder than they already are (way too hard, really).  The recent addition
of PCH to GCC has raised the bar even further to design changes.

I think the OMP proponents would do best to get GCC's architecture
improved to the point that it is a fairly clean and orthogonal addition
to the compiler.  At that point it would be very hard to find reasonable
objections to its addition.  However, at the moment such objections are
easy to find: the last thing we need is something hackish embedded in
the parsers and middle end of GCC like was intimated in the opening post
of these threads.

Neil.

Re: OpenMP, HPC, and the future of GCC

[Neil Booth]

Scott Robert Ladd wrote:-

> Is GNU interested in supporting the HPC community? There are those of us
> willing to put in the work if there is reasonable expectation that the gcc
> "core" will accept it. But if HPC is going to be dismissed automatically, I
> don't see the point of making an effort to implement HPC features.

I'm making an effort with my parser, with no guarantee of acceptance,
simply because I want change badly enough.  If you want OMP badly
enough, you should be prepared to put effort in without a guarantee of
success.  The best way to get acceptance is to do it cleanly, and in
such a way that maintenance and performance of GCC is relatively
unaffected.

In my case I'm not sure I can do that, but I'm going to have a stab
anyway.  Something will always be rescuable from it even if it's not
accepted; maybe the intermediate representation, or parsing ideas,
or forcing performance improvements to match, or ....

Neil.

Re: OpenMP, HPC, and the future of GCC

[Diego Novillo]

On Mon, 10 Feb 2003, Steven Bosscher wrote:

> I think a homepage is in the making.  I am not sure where it should go:
> Savannah or gcc.gnu.org.
> 
We can always add a new page describing the gomp-branch at
http://gcc.gnu.org/projects/

The entry for gomp-01-branch at http://gcc.gnu.org/cvs.html will
also need changing, as well.


Diego.

Re: OpenMP, HPC, and the future of GCC

[Steven Bosscher]

Op ma 10-02-2003, om 18:33 schreef Michael S. Zick:
> On Monday 10 February 2003 11:27 am, Steven Bosscher wrote:
> >
> >
> > There's a GOMP project: http://savannah.nongnu.org/projects/gomp.
> >
> > Greetz
> > Steven
> Interesting.
> 
> I tried the "Public Areas" - "Project Homepage" link on that page
> and get a "Forbidden - you don't have permission to access /gomp/
> on this server."

Well yes and no.

It means that there's nothing there yet.  The project does not have a
homepage yet, just a mailing list.  Like I said, we were just exploring
an idea.

I think a homepage is in the making.  I am not sure where it should go:
Savannah or gcc.gnu.org.

Greetz
Steven

Re: OpenMP, HPC, and the future of GCC

[Biagio Lucini]

On Mon, 10 Feb 2003, Michael S. Zick wrote:

> On Monday 10 February 2003 11:27 am, Steven Bosscher wrote:
> >
> >
> > There's a GOMP project: http://savannah.nongnu.org/projects/gomp.
> >
> > Greetz
> > Steven
> Interesting.
> 
> I tried the "Public Areas" - "Project Homepage" link on that page
> and get a "Forbidden - you don't have permission to access /gomp/
> on this server."
> 
> Is that intentional?
> Mike
> 

No. Its just that we don't have anything to show for the moment. the
project is very young, it started ~ 45 days ago and we are mainly at a
design stage. There is a page you can visit (search for gnomp in projects)
but there isn't much of information as for yet. There is a public mailing
list

gomp-discuss (at) nongnu.org

that you can subscribe and where all the debate about design is taking
place. You are welcome to join, as anybody else. And don't be afraid of
the "nongnu": it is just temporary, since we don't have code we couldn't
get officially approved into the GNU project.

Biagio

Re: OpenMP, HPC, and the future of GCC

[Jan Hubicka]

> > This is problematic.  Since GCC maintainers can not implement 
> > all the HPC support themselves, they can not give in advance 
> > an promise that they will accept the code. 
> 
> Of course.  But isn't it reasonable to ask if there would be at least a
> serious review of the *technical* merits of the concept before a group
> of volunteers starts hacking on implementing it? 

It looks like everyone who wants to start someting bigger on GCC has
this problem.  Except for discussing it on the official mailing lists,
there is no other way to get kind of "approval" on the design of future
change. 

I am personally trying to split every plan into several incremental
steps usefull at their own and push them into mainline so I don't spend
too much time on developing something totaly bogus.  This probably does
not work for everything.  AST branch is good example of this.  Once
branch got past early design problems, many GCC developers examined it
and the overall design seems to be quite clear now.  I guess the same
thing would happen if you get the OpenMP implementation past the early
stages.

> 
> >From Per's mail this (European) morning, you'd say he's opposed to
> OpenMP just because it stinks from a language design POV.  Fine, OK,
> sure, its not Java :-) 
> 
> But if that is how the majority of the GCC developer community feels
> about OpenMP, then why would anyone spend time on developing the idea? 
> I know that I'd rather spend a few hours doing some contract research to
> fund my new vendor compiler if nobody will ever accept my patches for
> GCC. 

Because it is the only standard in the area?  I must say I don't like
the way pragmas are insterted into C as well, but still I think GCC
should adopt it.  I don't like many C++ features either and don't object
against them being integrated in GCC.
After all the parser part of about the easiest part of the project and
less of infrastructure can be independent on this.
> 
> > I personally would like to see GCC usefull for numeric computing 
> > too, so I would like to help with the project if it is created 
> > as my time allows. 
> 
> There's a GOMP project: http://savannah.nongnu.org/projects/gomp. 
> Active participants are (among others) Diego Novillo, Sebastian Pop,
> Scott Robert Ladd, Lars Segelund, Biagio Lucini and myself.  Or: GNU
> C/C++ hackers, GNU Fortran hackers, and numerical professionals.  Anyone
> is welcome to join of course. 

Yes, I was looking at it already.
> 
> We've put the project on a separate mailing list to see how this idea
> would evolve, but obviously now that it's getting more serious, we could
> (and should IMO) move some discussions to this mailing list. 
> 
> >  Every new feature 
> > needs to be implemented first and then it needs to be judged 
> > on how it works, how maintainable the code is before it can be 
> > accepted for mainline GCC. 
> 
> Right.  Those are the technical merits of the implementation.  But if
> whatever is implemented is already rejected, then there's no point. 

I think with some good will, it will be sorted out on the way as more
specific issues appear. 

Honza
> 
> Greetz 
> Steven 
> 

Re: OpenMP, HPC, and the future of GCC

[Michael S. Zick]

On Monday 10 February 2003 11:27 am, Steven Bosscher wrote:
>
>
> There's a GOMP project: http://savannah.nongnu.org/projects/gomp.
>
> Greetz
> Steven
Interesting.

I tried the "Public Areas" - "Project Homepage" link on that page
and get a "Forbidden - you don't have permission to access /gomp/
on this server."

Is that intentional?
Mike

Re: OpenMP, HPC, and the future of GCC

[Steven Bosscher]

> I think the question should be reversed.  Is HPC comunity 
> interested in supporting GNU? 

Financially, no :-)  CJB and Toon tried that once during a BOF for g77,
but the bigger part of the HPC community is in academia, and they're
just lousy funders. 

Manpower, yes.  Absolutely.  Are people interested in OpenMP for GCC?
Just count the number of questions about OpenMP on the mailing list over
the past, say, two years.  And whaddayaknow, some of them would also
like to help develop it. 

>   I don't think anyone working on GCC would not like it 
> to be usefull for more people than it is now and generate 
> better code. To make this happen, someone must realize it. 
> GNU can't do it, someone who actually needs the feature 
> has to make this happen. 

Which is why the "OpenMP for GCC" (Gomp) project now exists. 

> > willing to put in the work if there is reasonable expectation 
> > that the gcc "core" will accept it. But if HPC is going to be 
> >  dismissed automatically, I 
> 
> This is problematic.  Since GCC maintainers can not implement 
> all the HPC support themselves, they can not give in advance 
> an promise that they will accept the code. 

Of course.  But isn't it reasonable to ask if there would be at least a
serious review of the *technical* merits of the concept before a group
of volunteers starts hacking on implementing it? 

From Per's mail this (European) morning, you'd say he's opposed to
OpenMP just because it stinks from a language design POV.  Fine, OK,
sure, its not Java :-) 

But if that is how the majority of the GCC developer community feels
about OpenMP, then why would anyone spend time on developing the idea? 
I know that I'd rather spend a few hours doing some contract research to
fund my new vendor compiler if nobody will ever accept my patches for
GCC. 

> I personally would like to see GCC usefull for numeric computing 
> too, so I would like to help with the project if it is created 
> as my time allows. 

There's a GOMP project: http://savannah.nongnu.org/projects/gomp. 
Active participants are (among others) Diego Novillo, Sebastian Pop,
Scott Robert Ladd, Lars Segelund, Biagio Lucini and myself.  Or: GNU
C/C++ hackers, GNU Fortran hackers, and numerical professionals.  Anyone
is welcome to join of course. 

We've put the project on a separate mailing list to see how this idea
would evolve, but obviously now that it's getting more serious, we could
(and should IMO) move some discussions to this mailing list. 

>  Every new feature 
> needs to be implemented first and then it needs to be judged 
> on how it works, how maintainable the code is before it can be 
> accepted for mainline GCC. 

Right.  Those are the technical merits of the implementation.  But if
whatever is implemented is already rejected, then there's no point. 

Greetz 
Steven