co-arrays and libgfortran

co-arrays and libgfortran

[Steve Kargl]

Dear All, 

At the risk of causing consternation among the few
active gfortran developers, I'll offer the observation
that libgfortran likely need to be built with -fcoarray=lib
if one wants to use OpenCoarrays.  I'm not sure what
the status of shared memory coarrays is.  Why, you ask?

J3 defined RANDOM_INIT() in a manner that requires it to be
aware of the backing coarray communication method.  That is,
RANDOM_INIT(repeatable=.false., image_distinct=.false.)
requires communication between image 1 and all other images
(if num_image > 1).  'repeatable=.false.' means that each
time a program is run, a different set of processor-dependent
seeds are used on image 1.  This is trivial to do with
Janne's reworking of random_seed() when he introduce gfortran 
to xshiro++.  Now, if num_images() > 1, then all other images
are required to use the same set of seeds as image 1 to initial
their PRNG.  Therefore, the other images must ask image 1 for
its seeds.  I don't use co-arrays, so I'm unsure of the
 semantics; but, I have used MPI.  With MPI, one would have
image 1 broadcast the seeds and all other images would receive
the seeds.   

Note, I've attached a patch that fixes random_init() for other
combinations of repeatable and image_distinct (if anyone case)
in https://gcc.gnu.org/pipermail/gcc-bugs/2020-December/723251.html

-- 
Steve

Re: co-arrays and libgfortran

[Thomas Koenig]

Am 18.12.20 um 05:28 schrieb Steve Kargl via Fortran:
> Dear All,
> 
> At the risk of causing consternation among the few
> active gfortran developers, I'll offer the observation
> that libgfortran likely need to be built with -fcoarray=lib
> if one wants to use OpenCoarrays.

I hope there can be another solution :-)

> I'm not sure what
> the status of shared memory coarrays is.

It is starting to be usable.  It works for Toon's "random weather"
test program, and there is a growing number of test cases that work
(see 
https://gcc.gnu.org/git/?p=gcc.git;a=tree;f=gcc/testsuite/gfortran.dg/caf-shared;hb=refs/heads/devel/coarray_native 
)
but there are still quite a large number of things that don't.

> Why, you ask?

> J3 defined RANDOM_INIT() in a manner that requires it to be
> aware of the backing coarray communication method.  That is,
> RANDOM_INIT(repeatable=.false., image_distinct=.false.)
> requires communication between image 1 and all other images
> (if num_image > 1).  'repeatable=.false.' means that each
> time a program is run, a different set of processor-dependent
> seeds are used on image 1.  This is trivial to do with
> Janne's reworking of random_seed() when he introduce gfortran
> to xshiro++.  Now, if num_images() > 1, then all other images
> are required to use the same set of seeds as image 1 to initial
> their PRNG.  Therefore, the other images must ask image 1 for
> its seeds.  I don't use co-arrays, so I'm unsure of the
>   semantics; but, I have used MPI.  With MPI, one would have
> image 1 broadcast the seeds and all other images would receive
> the seeds.

The best way is probably to call different random_init routines
depending on the value of flag_coarray.

However, I am seeing some problems with synchronization with that
approach. The standard does not say anything about needing SYNC ALL
after random_init, or about implied synchronization.  That means
copying could be problematic.

Hmm... I'll have to think about this somre more.

Regards

	Thomas

Re: co-arrays and libgfortran

[Steve Kargl]

On Fri, Dec 18, 2020 at 01:26:37PM +0100, Thomas Koenig wrote:
> 
> Am 18.12.20 um 05:28 schrieb Steve Kargl via Fortran:
> > Dear All,
> > 
> > At the risk of causing consternation among the few
> > active gfortran developers, I'll offer the observation
> > that libgfortran likely need to be built with -fcoarray=lib
> > if one wants to use OpenCoarrays.
> 
> I hope there can be another solution :-)
> 

One of these days, I'll figure a way to grab Nicolas and
your work.  My svn gcc tree is almost a year out of date.


> > I'm not sure what
> > the status of shared memory coarrays is.
> 
> It is starting to be usable.  It works for Toon's "random weather"
> test program, and there is a growing number of test cases that work
> (see https://gcc.gnu.org/git/?p=gcc.git;a=tree;f=gcc/testsuite/gfortran.dg/caf-shared;hb=refs/heads/devel/coarray_native
> )
> but there are still quite a large number of things that don't.
> 
> > Why, you ask?
> 
> > J3 defined RANDOM_INIT() in a manner that requires it to be
> > aware of the backing coarray communication method.  That is,
> > RANDOM_INIT(repeatable=.false., image_distinct=.false.)
> > requires communication between image 1 and all other images
> > (if num_image > 1).  'repeatable=.false.' means that each
> > time a program is run, a different set of processor-dependent
> > seeds are used on image 1.  This is trivial to do with
> > Janne's reworking of random_seed() when he introduce gfortran
> > to xshiro++.  Now, if num_images() > 1, then all other images
> > are required to use the same set of seeds as image 1 to initial
> > their PRNG.  Therefore, the other images must ask image 1 for
> > its seeds.  I don't use co-arrays, so I'm unsure of the
> >   semantics; but, I have used MPI.  With MPI, one would have
> > image 1 broadcast the seeds and all other images would receive
> > the seeds.
> 
> The best way is probably to call different random_init routines
> depending on the value of flag_coarray.

That's what random_init() does.  The code for libgfortran/intrinsic/
random_init.f90 in the patch I submitted yesterday is cleaner to 
read and describes what I'm doing.

In particular, conv_intrinsic_random_init converts

  call random_init(repeatable, image_distinct)

to (could be some pass-by-value in here)
 
  _gfortran_random_init(repeatable, image_distinct, hidden)

where hidden is the value returned by this_image().  When I wrote
the function, this_image() maps to the library function in OpenCoarrays
via gfor_fndecl_caf_this_image.

  /* Create the hidden argument.  For non-coarray codes and -fcoarray=single,
     simply set this to 0.  For -fcoarray=lib, generate a call to
     THIS_IMAGE() without arguments.  */
  arg3 = build_int_cst (gfc_get_int_type (4), 0);
  if (flag_coarray == GFC_FCOARRAY_LIB)
    {
      arg3 = build_call_expr_loc (input_location, gfor_fndecl_caf_this_image,
      1, arg3);
      se.expr = fold_convert (gfc_get_int_type (4), arg3);
    }

If  gfor_fndecl_caf_this_image is rerouted to a shared memory implementation
for this_image(), then we're done; otherwise, we need to add an 

  else if (flag_coarray == GFC_FCOARRAY_SHARED)
    {
       /* map hidden to shared memory coarray's this_image() */
    }

> However, I am seeing some problems with synchronization with that
> approach. The standard does not say anything about needing SYNC ALL
> after random_init, or about implied synchronization.  That means
> copying could be problematic.
> 
> Hmm... I'll have to think about this somre more.

You're probably aware that the images are created async.

   5.3.4 Program execution

   Execution of a program consists of the asynchronous execution of
   a fixed number (which may be one) of its images.  Each image has
   its own execution state, floating-point status (17.7), and set of
   data objects, input/output units, and procedure pointers.

with MPI, the psuedocode would be

   if (this_image() == 1) then
      call random_seed()  ! processor-dependent seeds
      call mpi_send(seeds, ..., mpi_comm_world, ...)
   else
      call mpi_recv(seeds, ..., mpi_comm_world, ...)
   end if

I don't know if memory barriers are needed. As random_init.f90 is
Fortran, the psuedocode might be

   if (hidden < 2) then  
      ! get seeds on image 1
      call random_seed()
      call random_seed(get=seeds)
   else
      call co_broadcast(seeds,1)
   end if

This, however, would require libgfortran to be compiled with -fcoarry=lib
or shared.

-- 
Steve

Re: co-arrays and libgfortran

[Steve Kargl]

On Fri, Dec 18, 2020 at 07:52:52AM -0800, Steve Kargl wrote:
> 
> I don't know if memory barriers are needed. As random_init.f90 is
> Fortran, the psuedocode might be
> 
>    if (hidden < 2) then  
>       ! get seeds on image 1
>       call random_seed()
>       call random_seed(get=seeds)
>    else
>       call co_broadcast(seeds,1)
>    end if
> 

Well, one possible solution would have conv_intrinsic_random_init()
convert random_init(repeatable, image_distinct) into 

_gfortran_random_init(repeatable, image_distinct, hidden, sub)

where hidden is the value of this_image() as before, and sub is the
name (or pointer?) to a subroutine.  For -fcoarrays=none or single,
sub could be a stub that is never called.  For -fcoarrays=lib, sub
would be whatever gfor_fndecl_co_broadcast is associated (opencoarray
is _gfortran_caf_co_broadcast).  The Fortran code in random_init.f90
would then be

    if (hidden < 2) then
       ! get seeds on image 1
       call random_seed()
       call random_seed(get=seeds)
    else
       call sub(seeds,1)
    end if

A second possibility is to use a weak reference.  libgfortran
can contain a weak reference to _gfortran_caf_co_broadcast
when linked with -fcoarrays=none or single the weak reference
will never be called.  With  -fcoarrays=lib, the linker should
resolve to the version provided by opencoarrays.

Not sure how this would play out with shared memory coarray.

-- 
Steve

Re: co-arrays and libgfortran

[Thomas Koenig]

Hi Steve,

> Well, one possible solution would have conv_intrinsic_random_init()
> convert random_init(repeatable, image_distinct) into
> 
> _gfortran_random_init(repeatable, image_distinct, hidden, sub)
> 
> where hidden is the value of this_image() as before, and sub is the
> name (or pointer?) to a subroutine.

I think it would be easier to do

   if (flag_coarray == GFC_FCOARRAY_LIB)
     tmp = build_call_expr_loc (input_location, gfor_fndecl_caf_ 
random_init, 3,
			     arg1, arg2, arg3);
   else if (flag_coarray == GFC_FCOARRAY_SHARED)
     tmp = build_call_expr_loc (input_location, gfor_fndecl_cas_ 
random_init, 3,
			     arg1, arg2, arg3);
   else
     tmp = build_call_expr_loc (input_location, gfor_fndecl_random_init, 3,
			     arg1, arg2, arg3);

This way, we do not have to change the normal libgfortran, and any
implementation can then do its own thing.

Best regards

	Thomas

Re: co-arrays and libgfortran

[Steve Kargl]

On Sun, Dec 20, 2020 at 11:52:59AM +0100, Thomas Koenig wrote:
> 
> Hi Steve,
> 
> > Well, one possible solution would have conv_intrinsic_random_init()
> > convert random_init(repeatable, image_distinct) into
> > 
> > _gfortran_random_init(repeatable, image_distinct, hidden, sub)
> > 
> > where hidden is the value of this_image() as before, and sub is the
> > name (or pointer?) to a subroutine.
> 
> I think it would be easier to do
> 
>   if (flag_coarray == GFC_FCOARRAY_LIB)
>     tmp = build_call_expr_loc (input_location, gfor_fndecl_caf_ random_init,
> 3,
> 			     arg1, arg2, arg3);
>   else if (flag_coarray == GFC_FCOARRAY_SHARED)
>     tmp = build_call_expr_loc (input_location, gfor_fndecl_cas_ random_init,
> 3,
> 			     arg1, arg2, arg3);
>   else
>     tmp = build_call_expr_loc (input_location, gfor_fndecl_random_init, 3,
> 			     arg1, arg2, arg3);
> 
> This way, we do not have to change the normal libgfortran, and any
> implementation can then do its own thing.
> 

Yes, that would work, too.  If arg3 is the 'hidden' argument that
gfortran generates for the image number, then it is likely unneeded
under your suggestion.  Both caf and cas can internally determine
this_image() and the 'none' and 'single' know that there is only 
one image.  Hmmm, now that I think about it.  I guess we cannot
drop the 3rd argument, because it would change the ABI for
libgfortran.

-- 
Steve

Re: co-arrays and libgfortran

[Steve Kargl]

On Sun, Dec 20, 2020 at 09:47:40AM -0800, Steve Kargl wrote:
> On Sun, Dec 20, 2020 at 11:52:59AM +0100, Thomas Koenig wrote:
> > 
> > > Well, one possible solution would have conv_intrinsic_random_init()
> > > convert random_init(repeatable, image_distinct) into
> > > 
> > > _gfortran_random_init(repeatable, image_distinct, hidden, sub)
> > > 
> > > where hidden is the value of this_image() as before, and sub is the
> > > name (or pointer?) to a subroutine.
> > 
> > I think it would be easier to do
> > 
> >   if (flag_coarray == GFC_FCOARRAY_LIB)
> >     tmp = build_call_expr_loc (input_location, gfor_fndecl_caf_ random_init,
> > 3,
> > 			     arg1, arg2, arg3);
> >   else if (flag_coarray == GFC_FCOARRAY_SHARED)
> >     tmp = build_call_expr_loc (input_location, gfor_fndecl_cas_ random_init,
> > 3,
> > 			     arg1, arg2, arg3);
> >   else
> >     tmp = build_call_expr_loc (input_location, gfor_fndecl_random_init, 3,
> > 			     arg1, arg2, arg3);
> > 
> > This way, we do not have to change the normal libgfortran, and any
> > implementation can then do its own thing.
> > 
> 
> Yes, that would work, too.  If arg3 is the 'hidden' argument that
> gfortran generates for the image number, then it is likely unneeded
> under your suggestion.  Both caf and cas can internally determine
> this_image() and the 'none' and 'single' know that there is only 
> one image.  Hmmm, now that I think about it.  I guess we cannot
> drop the 3rd argument, because it would change the ABI for
> libgfortran.

I have attached a patch to implements Thomas's idea to 
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98301

% cat d.f90
    program foo
      call random_init(.true.,  .true.)
      call random_init(.true.,  .false.)
      call random_init(.false., .true.)
      call random_init(.false., .false.)
    end program foo
% gfcx -c -fdump-tree-original d.f90
% grep random d.f90.004t.original
  _gfortran_random_init (1, 1, 0);
  _gfortran_random_init (1, 0, 0);
  _gfortran_random_init (0, 1, 0);
  _gfortran_random_init (0, 0, 0);
% gfcx -c -fdump-tree-original -fcoarray=lib d.f90
% grep random d.f90.004t.original
  _gfortran_caf_random_init (1, 1);
  _gfortran_caf_random_init (1, 0);
  _gfortran_caf_random_init (0, 1);
  _gfortran_caf_random_init (0, 0);

The patch includes an #if 0 ... #endif for generating a library
routine for -fcoarray=shared, _gfortran_cas_random_init().  Note,
that  _gfortran_random_init() preserves the current libgfortran
ABI.  Neither of the last two arguments are needed.  For 
_gfortran_caf_random_init() and _gfortran_cas_random_init(), if
someone wants to implement these functions (Hi, Damian!) for
OpenCoarray and shared memory coarray in Fortran, I suggest looking
at the code in the patch labeled "New diff with improvements" attached
to the PR.

Thanks for the idea, Thomas.

-- 
Steve

Re: co-arrays and libgfortran

[Damian Rouson]

On Sun, Dec 20, 2020 at 5:04 PM Steve Kargl via Fortran <fortran@gcc.gnu.org>
wrote:

>  For _gfortran_caf_random_init() and _gfortran_cas_random_init(), if
> someone wants to implement these functions (Hi, Damian!) for
> OpenCoarray and shared memory coarray in Fortran, I suggest looking
> at the code in the patch labeled "New diff with improvements" attached
> to the PR.
>
> Awesome!  Thanks, Steve and Thomas.  This sounds like a job for Jerry or
Andre if either is interested.  I'll check.

Damian

Re: co-arrays and libgfortran

[Thomas Koenig]

Hi Steve,

> One of these days, I'll figure a way to grab Nicolas and
> your work.  My svn gcc tree is almost a year out of date.

This can be done rather easily via github.

Go to https://github.com/gcc-mirror/gcc/tree/devel/coarray_native

and choose "Download Zip" or download via svn.

We would really appreciate if you could test this. So far, only
Linux is known to work, and anything that exposes some
Linux-centric assumptions in the code is welcome.

Best regards

	Thomas

Re: co-arrays and libgfortran

[Steve Kargl]

On Mon, Dec 21, 2020 at 02:15:30PM +0100, Thomas Koenig wrote:
> 
> Hi Steve,
> 
> > One of these days, I'll figure a way to grab Nicolas and
> > your work.  My svn gcc tree is almost a year out of date.
> 
> This can be done rather easily via github.
> 
> Go to https://github.com/gcc-mirror/gcc/tree/devel/coarray_native
> 
> and choose "Download Zip" or download via svn.
> 

disclaimer: I have zero, nada, nil, none, experience with git.

That URL appears to take me to the coarray_native branch.  If I click
on the 'clone' link, I see https://github.com/gcc-mirror/gcc.git.
Is that for a complete gcc repository or specific to your branch?
IOW, if I clone from that URL, do I need to do any git voodoo
magic incantation to get to the branch?

-- 
Steve

Re: co-arrays and libgfortran

[Thomas Koenig]

Hi Steve,

> That URL appears to take me to the coarray_native branch.  If I click
> on the 'clone' link, I seehttps://github.com/gcc-mirror/gcc.git.
> Is that for a complete gcc repository or specific to your branch?
> IOW, if I clone from that URL, do I need to do any git voodoo
> magic incantation to get to the branch?

Actually, I tried this via svn from github. Not the normal way of
accessing it, but if you're more familiar with svn, it chould work for
you.

What I did was

$ svn co --depth empty https://github.com/gcc-mirror/gcc.git Gcc
$ cd Gcc
$ svn up trunk
$ cd trunk
$ svn switch 
https://github.com/gcc-mirror/gcc.git/branches/devel/coarray_native

which then gave me a svn copy of the branch in the Gcc/trunk directory,
which looked correct.

(Alternatively, possibly easier in the longer run, you could use
the standard git commands to access the branch from the normal
gcc repository).

Hope this helps.

Best regards

	Thomas

Re: co-arrays and libgfortran

[Steve Kargl]

On Tue, Dec 22, 2020 at 12:44:29AM +0100, Thomas Koenig wrote:
> 
> Hi Steve,
> 
> > That URL appears to take me to the coarray_native branch.  If I click
> > on the 'clone' link, I seehttps://github.com/gcc-mirror/gcc.git.
> > Is that for a complete gcc repository or specific to your branch?
> > IOW, if I clone from that URL, do I need to do any git voodoo
> > magic incantation to get to the branch?
> 
> Actually, I tried this via svn from github. Not the normal way of
> accessing it, but if you're more familiar with svn, it chould work for
> you.
> 
> What I did was
> 
> $ svn co --depth empty https://github.com/gcc-mirror/gcc.git Gcc
> $ cd Gcc
> $ svn up trunk
> $ cd trunk
> $ svn switch
> https://github.com/gcc-mirror/gcc.git/branches/devel/coarray_native
> 

Thanks.  Downloading now.  It'll take a day to bootstrap on my
old i386-*-freebsd system.

-- 
Steve

Re: co-arrays and libgfortran

[Steve Kargl]

On Mon, Dec 21, 2020 at 04:18:43PM -0800, Steve Kargl wrote:
> On Tue, Dec 22, 2020 at 12:44:29AM +0100, Thomas Koenig wrote:
> > 
> > > That URL appears to take me to the coarray_native branch.  If I click
> > > on the 'clone' link, I seehttps://github.com/gcc-mirror/gcc.git.
> > > Is that for a complete gcc repository or specific to your branch?
> > > IOW, if I clone from that URL, do I need to do any git voodoo
> > > magic incantation to get to the branch?
> > 
> > Actually, I tried this via svn from github. Not the normal way of
> > accessing it, but if you're more familiar with svn, it chould work for
> > you.
> > 
> > What I did was
> > 
> > $ svn co --depth empty https://github.com/gcc-mirror/gcc.git Gcc
> > $ cd Gcc
> > $ svn up trunk
> > $ cd trunk
> > $ svn switch
> > https://github.com/gcc-mirror/gcc.git/branches/devel/coarray_native
> > 
> 
> Thanks.  Downloading now.  It'll take a day to bootstrap on my
> old i386-*-freebsd system.
> 

It appears the above worked.   Last questions.  Any special
--enable-XXX options you want me to test?  What's the 
checking target 'make check-shared'?

-- 
Steve

Re: co-arrays and libgfortran

[Thomas Koenig]

Hi Steve,

> It appears the above worked.

Great!

> Last questions.  Any special
> --enable-XXX options you want me to test?

That is currently not needed.

> What's the
> checking target 'make check-shared'?

Maybe you could check a simple "Hello world" program like
Dominique's first, see if it works at all.  You will need
to link using -lcaf_shared and possibly against -lrt (if
that is needed for shm_open).

If that works, you could then go into the gcc build
directory and run

make check-fortran RUNTESTFLAGS="cas.exp=*.f*"

Best regards

	Thomas

Re: co-arrays and libgfortran

[Steve Kargl]

On Tue, Dec 22, 2020 at 11:04:19AM +0100, Thomas Koenig wrote:
> 
> Hi Steve,
> 
> > It appears the above worked.
> 
> Great!
> 
> > Last questions.  Any special
> > --enable-XXX options you want me to test?
> 
> That is currently not needed.
> 
> > What's the
> > checking target 'make check-shared'?
> 
> Maybe you could check a simple "Hello world" program like
> Dominique's first, see if it works at all.  You will need
> to link using -lcaf_shared and possibly against -lrt (if
> that is needed for shm_open).
> 
> If that works, you could then go into the gcc build
> directory and run
> 
> make check-fortran RUNTESTFLAGS="cas.exp=*.f*"
> 

Thomas, Nicolas,

Congrats!  Bootstrap completed without any intervention.
At least a simple program appears to work on i386-*-freebsd!
I needed to change my wrapper gfccas so that gfortran could
find the caf_shared library (see below).
But, first:

% cat a.f90
  program hello
    print *, 'hello from ', this_image()
  end program hello
% gfccas -o z a.f90 -lcaf_shared
% ./z
 hello from            2
 hello from            1

I don't know how hard it would be to install libcaf_shared.so
in the same directory as libgfortran.so, but I think that
might save you from some unnecessary bug reports.  I had to
add the library paths to LD_LIBRARY_PATH and LD_RUN_PATH to
get the above to run.  I configured gcc to install into
/usr/home/kargl/work/cas/.  Without fixing gfccas and using
$prefix in the following:

% gfccas -o z a.f90 -lcaf_shared
% ./z
ld-elf.so.1: Shared object "libcaf_shared.so.0" not found, required by "z"
% ldd ./z
./z:
    libcaf_shared.so.0 => not found (0)
    libgfortran.so.5 => $prefix/lib/libgfortran.so.5 (0x28200000)
    libm.so.5 => /lib/libm.so.5 (0x28089000)
    libgcc_s.so.1 => $prefix/lib/libgcc_s.so.1 (0x280c0000)
    libquadmath.so.0 => $prefix/lib/libquadmath.so.0 (0x280db000)
    libc.so.7 => /lib/libc.so.7 (0x2847b000)
    libthr.so.3 => /lib/libthr.so.3 (0x2816c000)
    librt.so.1 => /usr/lib/librt.so.1 (0x28195000)

libthr.so.3 is FreeBSD thread library and librt is automatically
included.

% find ~/work/cas -name libcaf\*
$prefix/lib/gcc/i586-unknown-freebsd13.0/11.0.0/libcaf_single.la
$prefix/lib/gcc/i586-unknown-freebsd13.0/11.0.0/libcaf_single.a
$prefix/lib/gcc/i586-unknown-freebsd13.0/11.0.0/libcaf_shared.so.0
$prefix/lib/gcc/i586-unknown-freebsd13.0/11.0.0/libcaf_shared.so
$prefix/lib/gcc/i586-unknown-freebsd13.0/11.0.0/libcaf_shared.la
$prefix/lib/gcc/i586-unknown-freebsd13.0/11.0.0/libcaf_shared.a

Note most FreeBSD users of gcc will install it from FreeBSD's port
collection.  During installation, FreeBSD's packaging system runs
ldconfig, so the above library may automatically be found.

-- 
Steve

Re: co-arrays and libgfortran

[Steve Kargl]

On Tue, Dec 22, 2020 at 10:32:36AM -0800, Steve Kargl wrote:
> On Tue, Dec 22, 2020 at 11:04:19AM +0100, Thomas Koenig wrote:
> > 
> > > What's the
> > > checking target 'make check-shared'?
> > 
> > Maybe you could check a simple "Hello world" program like
> > Dominique's first, see if it works at all.  You will need
> > to link using -lcaf_shared and possibly against -lrt (if
> > that is needed for shm_open).
> > 
> > If that works, you could then go into the gcc build
> > directory and run
> > 
> > make check-fortran RUNTESTFLAGS="cas.exp=*.f*"
> > 

I started the make check command.  I'm see a few failures
of the form

WARNING: program timed out.
FAIL: gfortran.dg/caf-shared/dummy_1.f90 -pthread -fcoarray=shared \
  -O2  -lcaf_shared -lrt execution test

I'll send more info once the checking completes.

-- 
Steve

Re: co-arrays and libgfortran

[Steve Kargl]

On Tue, Dec 22, 2020 at 11:33:25AM -0800, Steve Kargl wrote:
> 
> I started the make check command.  I'm see a few failures
> of the form
> 
> WARNING: program timed out.
> FAIL: gfortran.dg/caf-shared/dummy_1.f90 -pthread -fcoarray=shared \
>   -O2  -lcaf_shared -lrt execution test
> 
> I'll send more info once the checking completes.
> 

Poking around.

% ps
 3491  3  I+   0:00.91 gmake check-fortran RUNTESTFLAGS=cas.exp=*.f*
 3576  3  I+   0:00.87 gmake TESTSUITEDIR=testsuite RUNTESTFLAGS=cas.exp=*.f* check_gfortran_pa
 3660  3  I+   0:00.00 /bin/sh -c (rootme=`${PWDCMD-pwd}`; export rootme; \\\nsrcdir=`cd ../../
 3670  3  I+   0:01.04 expect -- /usr/local/share/dejagnu/runtest.exp --tool gfortran cas.exp=*
 3726  3  I+   0:00.00 ./alloc_coarray_with_source_1.exe (alloc_coarray_with_)
 3727  3  I+   0:00.00 ./alloc_coarray_with_source_1.exe (alloc_coarray_with_)
 3728  3  I+   0:00.00 ./alloc_coarray_with_source_1.exe (alloc_coarray_with_)
 3751  3  I+   0:00.00 ./alloc_comp_1.exe
 3786  3  I+   0:00.00 ./arg_passing_1.exe
 3787  3  I+   0:00.00 ./arg_passing_1.exe
 3788  3  I+   0:00.00 ./arg_passing_1.exe
 3807  3  I+   0:00.00 ./assign_array_1.exe
 3826  3  I+   0:00.00 ./atomic_1.exe
 3827  3  I+   0:00.00 ./atomic_1.exe
 3828  3  I+   0:00.00 ./atomic_1.exe
 3884  3  I+   0:00.00 ./cobounds_torture_1.exe (cobounds_torture_1.)
 3912  3  I+   0:00.00 ./cobounds_torture_2.exe (cobounds_torture_2.)
 3931  3  I+   0:00.00 ./cobounds_torture_3.exe (cobounds_torture_3.)
 4014  3  I+   0:00.00 ./cosubscript_1.exe
 4015  3  I+   0:00.00 ./cosubscript_1.exe
 4016  3  I+   0:00.00 ./cosubscript_1.exe
 4037  3  I+   0:00.00 ./dummy_1.exe
 4069  3  I+   0:00.00 ./get_array.exe
 4070  3  I+   0:00.00 ./get_array.exe
 4071  3  I+   0:00.00 ./get_array.exe
 4144  3  I+   0:00.00 ./image_index_2.exe
 4145  3  I+   0:00.00 ./image_index_2.exe
 4146  3  I+   0:00.00 ./image_index_2.exe
 4176  3  I+   0:00.00 ./lower_cobound_1.exe
 4177  3  I+   0:00.00 ./lower_cobound_1.exe
 4178  3  I+   0:00.00 ./lower_cobound_1.exe
...

Looks like something is stuck in a deadlock.

-- 
Steve

Re: co-arrays and libgfortran

[Steve Kargl]

On Tue, Dec 22, 2020 at 12:12:02PM -0800, Steve Kargl wrote:
> On Tue, Dec 22, 2020 at 11:33:25AM -0800, Steve Kargl wrote:
> > 
> > I started the make check command.  I'm see a few failures
> > of the form
> > 
> > WARNING: program timed out.
> > FAIL: gfortran.dg/caf-shared/dummy_1.f90 -pthread -fcoarray=shared \
> >   -O2  -lcaf_shared -lrt execution test
> > 
> > I'll send more info once the checking completes.
> > 
> 
> Poking around.
> 
> % ps
>  3491  3  I+   0:00.91 gmake check-fortran RUNTESTFLAGS=cas.exp=*.f*
>  3576  3  I+   0:00.87 gmake TESTSUITEDIR=testsuite RUNTESTFLAGS=cas.exp=*.f* check_gfortran_pa
>  3660  3  I+   0:00.00 /bin/sh -c (rootme=`${PWDCMD-pwd}`; export rootme; \\\nsrcdir=`cd ../../
>  3670  3  I+   0:01.04 expect -- /usr/local/share/dejagnu/runtest.exp --tool gfortran cas.exp=*
>  3726  3  I+   0:00.00 ./alloc_coarray_with_source_1.exe (alloc_coarray_with_)
>  3727  3  I+   0:00.00 ./alloc_coarray_with_source_1.exe (alloc_coarray_with_)
>  3728  3  I+   0:00.00 ./alloc_coarray_with_source_1.exe (alloc_coarray_with_)

Apologies for the monologue.  After killing the idle processes, I see

                === gfortran Summary ===

# of expected passes            44
# of unexpected failures        21
# of unsupported tests          2
/usr/home/kargl/gcc/obj_cas/gcc/gfortran  version 11.0.0 20201025 (experimental) (GCC) 


-- 
Steve

Re: co-arrays and libgfortran

[Steve Kargl]

More monologuing.

I copied gcc/testsuite/gfortran.dg/caf-shared/dummy_1.f90 to tmp/dummy_1.f90.

% gfccas -o z dummy_1.f90 
% ./z
ERROR: Image 2(0x1748) failed
% ps | grep z
 5958  1  I    0:00.00 ./z
 5959  1  I    0:00.00 ./z
 5963  1  S+   0:00.01 grep z
% kill 5959
ERROR: Image 2(0x1747) failed

-- 
Steve

Re: co-arrays and libgfortran

[Steve Kargl]

On Tue, Dec 22, 2020 at 02:08:47PM -0800, Steve Kargl wrote:
> More monologuing.
> 
> I copied gcc/testsuite/gfortran.dg/caf-shared/dummy_1.f90 to tmp/dummy_1.f90.
> 
> % gfccas -o z dummy_1.f90 
> % ./z
> ERROR: Image 2(0x1748) failed
> % ps | grep z
>  5958  1  I    0:00.00 ./z
>  5959  1  I    0:00.00 ./z
>  5963  1  S+   0:00.01 grep z
> % kill 5959
> ERROR: Image 2(0x1747) failed
> 

Recompiled with -g and the image dropped core.

% gdb ./z z.core

Core was generated by `./z'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  scan_inside_lookahead (id=134523584, expected_off=<optimized out>, 
    hm=0x281c2198) at ../../../cas/libgfortran/caf_shared/hashmap.c:157
157       lookahead = data[expected_off].max_lookahead;
(gdb) bt
#0  scan_inside_lookahead (id=134523584, expected_off=<optimized out>, 
    hm=0x281c2198) at ../../../cas/libgfortran/caf_shared/hashmap.c:157
#1  hashmap_get (hm=0x281c2198, id=134523584)
    at ../../../cas/libgfortran/caf_shared/hashmap.c:196
#2  0x2808d002 in get_memory_by_id_internal (iface=0x281c2188, size=8, 
    id=134523584, zero_mem=false)
    at ../../../cas/libgfortran/caf_shared/alloc.c:55
#3  0x2808ed9f in _gfortran_cas_coarray_alloc (desc=0x804aac0 <b>, 
    elem_size=4, corank=1, alloc_type=1)
    at ../../../cas/libgfortran/caf_shared/wrapper.c:154
#4  0x08048dbe in MAIN__ () at dummy_1.f90:30
(gdb) up 1
#1  hashmap_get (hm=0x281c2198, id=134523584)
    at ../../../cas/libgfortran/caf_shared/hashmap.c:196
196       res = scan_inside_lookahead (hm, expected_offset, id);
(gdb) p *hm
$1 = {s = 0x281ef0bc, sm = 0x281c2184, a = 0x281c2190}
(gdb) p *hm->s
$2 = {data = {offset = 208}, size = 32, bitnum = 5}
(gdb) p *hm->sm
$3 = (shared_memory) 0x2808a020
(gdb) p **hm->sm
$4 = {meta = 0x281ef000, header = 0x0, last_seen_size = 4096, 
  num_local_allocs = 1, allocs = 0x2808a030}
(gdb) p *hm->a
$5 = {s = 0x281ef03c, shm = 0x281c2184}
(gdb) p *hm->a->s
$6 = {free_bucket_head = {{offset = -1} <repeats 32 times>}}
(gdb) p *hm->a->shm
$7 = (shared_memory) 0x2808a020
(gdb) p **hm->a->shm
$8 = {meta = 0x281ef000, header = 0x0, last_seen_size = 4096, 
  num_local_allocs = 1, allocs = 0x2808a030}

Any of this look familiar?

-- 
Steve

Re: co-arrays and libgfortran

[Steve Kargl]

On Mon, Dec 21, 2020 at 05:40:01PM -0800, Steve Kargl wrote:
> On Mon, Dec 21, 2020 at 04:18:43PM -0800, Steve Kargl wrote:
> > On Tue, Dec 22, 2020 at 12:44:29AM +0100, Thomas Koenig wrote:
> > > 
> > > > That URL appears to take me to the coarray_native branch.  If I click
> > > > on the 'clone' link, I seehttps://github.com/gcc-mirror/gcc.git.
> > > > Is that for a complete gcc repository or specific to your branch?
> > > > IOW, if I clone from that URL, do I need to do any git voodoo
> > > > magic incantation to get to the branch?
> > > 
> > > Actually, I tried this via svn from github. Not the normal way of
> > > accessing it, but if you're more familiar with svn, it chould work for
> > > you.
> > > 
> > > What I did was
> > > 
> > > $ svn co --depth empty https://github.com/gcc-mirror/gcc.git Gcc
> > > $ cd Gcc
> > > $ svn up trunk
> > > $ cd trunk
> > > $ svn switch
> > > https://github.com/gcc-mirror/gcc.git/branches/devel/coarray_native
> > > 
> > 
> > Thanks.  Downloading now.  It'll take a day to bootstrap on my
> > old i386-*-freebsd system.
> > 
> 
> It appears the above worked.   Last questions.  Any special
> --enable-XXX options you want me to test?  What's the 
> checking target 'make check-shared'?
> 

Just finished bootstrap on x86_64-*-freebsd.

% gmake -j7 check-fortran

                === gfortran Summary ===

# of expected passes            54886
# of unexpected failures        3
# of expected failures          193
# of unsupported tests          110


% gmake check-fortran RUNTESTFLAGS="cas.exp=\*.f\*"
                === gfortran Summary ===

# of expected passes            65
# of unsupported tests          2
objs/gcc/gfortran  version 11.0.0 20201025 (experimental) (GCC) 


So, the issue I've reported else thread is likely a
32-bit vs 64-bit OS issue.  Perhaps, a size_t or
ssize_t issue?

% cat a.c
#include <stdio.h>
int
main(void)
{
   printf("%d\n", sizeof(size_t));
   printf("%d\n", sizeof(ssize_t));
   return 0;
}

On i386-*-freebsd
% cc -o z a.c
% ./z
4
4

On x86_64-*-freebsd
%  cc -o z a.c
% ./z
8
8


-- 
Steve

Re: co-arrays and libgfortran

[Thomas Koenig]

Hi Steve,

> So, the issue I've reported else thread is likely a
> 32-bit vs 64-bit OS issue.  Perhaps, a size_t or
> ssize_t issue?

Thanks for testing!  This could well be the case.

We are also seeing some mysterious failures, probably due to
memory corruption - maybe some offset isn't yet quite right
(that wouldnt be the first case).

Let's see what we can find over the next couple of days.

Best regards

	Thomas