Re: OpenCoarrays integration with gfortran

[Damian Rouson <damian@sourceryinstitute.org>]

On Fri, Sep 21, 2018 at 9:25 AM Jerry DeLisle <jvdelisle@charter.net> wrote:

> The actual library source code is contained mostly in one source file.

There are as many files as there are options for the underlying
parallel programming
model.  The default is MPI, but I've co-authored conference papers last year
and this year in which the OpenCoarrays OpenSHEM option outperformed MPI.
One paper even described a platform on which OpenSHMEM was the only option
beyond a few thousand cores because the required MPI features were immature on
that platform.  Early versions of OpenCoarrays also provided GASNet
and ARMCI options.
I recommend against tying gfortran to MPI only.

> After all the attempts to integrate into the GNU build systems without
> much success my thinking has shifted.

Thanks for all your efforts!

> Keep in mind that the OpenCoarrays
> implementation is quite dependent on gfortran and in fact has to do
> special things in the build dependent on the version of gcc/gfortran a
> user happens to use.  I dont think this is a good situation.

I agree.  Possibly OpenCoarrays could drop support for older gfortran versions
at some point to avoid maintaining code that exists solely to support compiler
versions that are several years old.

>
> 1) Focus on distribution packages such as Fedora, Debian, Ubuntu,
> Windows, etc. Building of these packages needs to be automated into the
> distributions.

This is the option that the OpenCoarrays documentation recommends as easiest for
most users.

> 2) Take the one source file, edit out all the macros that define
> prefixes to function calls, hard code the gfortran prefixes etc and fork
> it directly into the libgfortran library under GPL with attributions to
> the original developers as appropriate.

See above.   Also, this means that changes in the gfortran repository would not
propagate back upstream unless each gfortran developer agrees to
distribute his or her
work under both GPL and BSD.  Even that is only feasible if the copied
files stay cohesive
and don't reference code outside the copied file.  I think it's more
likely that copying the code
into gfortran would be a branch point, after which the relevant files
would diverge and
work on the GPL side would be harder to fund than the BSD side.

Most commercial entities are more likely to contribute to a
BSD-licensed project than a
GPL-licensed one.  Over the past several months, one commercial compiler vendor
authorized one of their developers to contribute to OpenCoarrays. and
another commercial
compiler vendor invited community input on whether to use OpenCoarrays
during a public
teleconference.  The prospect of commercial support is the motivation
for using BSD.

> Strategy 2 does have some advantages. For example, eliminating the need
> for separate CAF and CAFRUN scripts which are a wrapper on gfortran.

Even in the case of just one underlying parallel programming model,
this is tricky.  To wit, Cray uses
a compiler wrapper and a program launcher.  Intel was able to
eliminate the compiler wrapper,
but still required a program launcher for distributed-memory execution
until recently.  I don't
know the details, but I've heard it was not trivial for Intel to
accomplish this and I imagine it would be
even more complicated if they weren't hardwiring Intel MPI into their back-end.

> People would just have to stop thinking about it and just use it.

The same would be true if someone could coax the GCC build system to
build OpenCoarrays
just as it builds other prerequisites.  The big difference is that
OpenCoarrays is a prerequisite
for using gfortran rather than for building gfortran so it needs to be
built after gfortran rather
than before like other prerequisites.  The real problem is finding
anyone who can work the
proper magic in the GCC build system.

Thanks for your input.  I hope my response is helpful.

Damian