Re: co-arrays and libgfortran

[Steve Kargl <sgk@troutmask.apl.washington.edu>]

On Fri, Dec 18, 2020 at 01:26:37PM +0100, Thomas Koenig wrote:
> 
> Am 18.12.20 um 05:28 schrieb Steve Kargl via Fortran:
> > Dear All,
> > 
> > At the risk of causing consternation among the few
> > active gfortran developers, I'll offer the observation
> > that libgfortran likely need to be built with -fcoarray=lib
> > if one wants to use OpenCoarrays.
> 
> I hope there can be another solution :-)
> 

One of these days, I'll figure a way to grab Nicolas and
your work.  My svn gcc tree is almost a year out of date.


> > I'm not sure what
> > the status of shared memory coarrays is.
> 
> It is starting to be usable.  It works for Toon's "random weather"
> test program, and there is a growing number of test cases that work
> (see https://gcc.gnu.org/git/?p=gcc.git;a=tree;f=gcc/testsuite/gfortran.dg/caf-shared;hb=refs/heads/devel/coarray_native
> )
> but there are still quite a large number of things that don't.
> 
> > Why, you ask?
> 
> > J3 defined RANDOM_INIT() in a manner that requires it to be
> > aware of the backing coarray communication method.  That is,
> > RANDOM_INIT(repeatable=.false., image_distinct=.false.)
> > requires communication between image 1 and all other images
> > (if num_image > 1).  'repeatable=.false.' means that each
> > time a program is run, a different set of processor-dependent
> > seeds are used on image 1.  This is trivial to do with
> > Janne's reworking of random_seed() when he introduce gfortran
> > to xshiro++.  Now, if num_images() > 1, then all other images
> > are required to use the same set of seeds as image 1 to initial
> > their PRNG.  Therefore, the other images must ask image 1 for
> > its seeds.  I don't use co-arrays, so I'm unsure of the
> >   semantics; but, I have used MPI.  With MPI, one would have
> > image 1 broadcast the seeds and all other images would receive
> > the seeds.
> 
> The best way is probably to call different random_init routines
> depending on the value of flag_coarray.

That's what random_init() does.  The code for libgfortran/intrinsic/
random_init.f90 in the patch I submitted yesterday is cleaner to 
read and describes what I'm doing.

In particular, conv_intrinsic_random_init converts

  call random_init(repeatable, image_distinct)

to (could be some pass-by-value in here)
 
  _gfortran_random_init(repeatable, image_distinct, hidden)

where hidden is the value returned by this_image().  When I wrote
the function, this_image() maps to the library function in OpenCoarrays
via gfor_fndecl_caf_this_image.

  /* Create the hidden argument.  For non-coarray codes and -fcoarray=single,
     simply set this to 0.  For -fcoarray=lib, generate a call to
     THIS_IMAGE() without arguments.  */
  arg3 = build_int_cst (gfc_get_int_type (4), 0);
  if (flag_coarray == GFC_FCOARRAY_LIB)
    {
      arg3 = build_call_expr_loc (input_location, gfor_fndecl_caf_this_image,
      1, arg3);
      se.expr = fold_convert (gfc_get_int_type (4), arg3);
    }

If  gfor_fndecl_caf_this_image is rerouted to a shared memory implementation
for this_image(), then we're done; otherwise, we need to add an 

  else if (flag_coarray == GFC_FCOARRAY_SHARED)
    {
       /* map hidden to shared memory coarray's this_image() */
    }

> However, I am seeing some problems with synchronization with that
> approach. The standard does not say anything about needing SYNC ALL
> after random_init, or about implied synchronization.  That means
> copying could be problematic.
> 
> Hmm... I'll have to think about this somre more.

You're probably aware that the images are created async.

   5.3.4 Program execution

   Execution of a program consists of the asynchronous execution of
   a fixed number (which may be one) of its images.  Each image has
   its own execution state, floating-point status (17.7), and set of
   data objects, input/output units, and procedure pointers.

with MPI, the psuedocode would be

   if (this_image() == 1) then
      call random_seed()  ! processor-dependent seeds
      call mpi_send(seeds, ..., mpi_comm_world, ...)
   else
      call mpi_recv(seeds, ..., mpi_comm_world, ...)
   end if

I don't know if memory barriers are needed. As random_init.f90 is
Fortran, the psuedocode might be

   if (hidden < 2) then  
      ! get seeds on image 1
      call random_seed()
      call random_seed(get=seeds)
   else
      call co_broadcast(seeds,1)
   end if

This, however, would require libgfortran to be compiled with -fcoarry=lib
or shared.

-- 
Steve