coarrays using extended precision (80 bits) ?

coarrays using extended precision (80 bits) ?

[Jorge D'Elia]

[-- Attachment #1: Type: text/plain, Size: 711 bytes --]

Dear GFortran developers, 

One question: is there any chance of encoding with coarrays using 
extended precision (80 bits) at least inside a multicore computer? 
(as if to simplify a bit). 

To date, the possibility of using double precision (64 bits) or 
extended precision (80 bits) is an alternative in our production 
code, but sometimes we would like to do computations in 
80 bits and, in certain parts, there are coarrays. 
We have validated even in quadruple precision (128 bits), using 
ifort although, as is well known, the CPU times are largely 
excessive. 

Thanks in any case. 

Regards. 
Jorge. 
​​CIMEC (UNL-CONICET), [ http://www.cimec.org.ar/ | www.cimec.org.ar ] 
-- 

Re: coarrays using extended precision (80 bits) ?

[Steve Kargl]

On Fri, Apr 21, 2023 at 08:33:31AM -0300, Jorge D'Elia wrote:
> Dear GFortran developers, 
> 
> One question: is there any chance of encoding with coarrays using 
> extended precision (80 bits) at least inside a multicore computer? 
> (as if to simplify a bit). 
> 
> To date, the possibility of using double precision (64 bits) or 
> extended precision (80 bits) is an alternative in our production 
> code, but sometimes we would like to do computations in 
> 80 bits and, in certain parts, there are coarrays. 
> We have validated even in quadruple precision (128 bits), using 
> ifort although, as is well known, the CPU times are largely 
> excessive. 
> 
> Thanks in any case. 
> 

Well, I just installed OpenCoarray and downloaded a pi/4
monte carlo code that Thomas wrote using REAL.  I changed
everything to use REAL(10).  Compiled and executed without
a problem.  I also tested REAL(16), which worked although
it's painfully slow due to software floating point.  So,
I guess I don't understand what you're asking?

-- 
Steve

Re: coarrays using extended precision (80 bits) ?

[Jorge D'Elia]

Dear Steve,

----- Mensaje original -----
> De: "Gfortran List" org>
> Para: "Jorge D'Elia" conicet.gov.ar>
> CC: "Gfortran List" org>
> Enviado: Viernes, 21 de Abril 2023 19:53:26
> Asunto: Re: coarrays using extended precision (80 bits) ?
>
> On Fri, Apr 21, 2023 at 08:33:31AM -0300, Jorge D'Elia wrote:
>> Dear GFortran developers,
>> 
>> One question: is there any chance of encoding with coarrays using
>> extended precision (80 bits) at least inside a multicore computer?
>> (as if to simplify a bit).
>> 
>> To date, the possibility of using double precision (64 bits) or
>> extended precision (80 bits) is an alternative in our production
>> code, but sometimes we would like to do computations in
>> 80 bits and, in certain parts, there are coarrays.
>> We have validated even in quadruple precision (128 bits), using
>> ifort although, as is well known, the CPU times are largely
>> excessive.
>> 
>> Thanks in any case.
>> 
> 
> Well, I just installed OpenCoarray and downloaded a pi/4
> monte carlo code that Thomas wrote using REAL.  I changed
> everything to use REAL(10).  Compiled and executed without
> a problem.  I also tested REAL(16), which worked although
> it's painfully slow due to software floating point.  So,
> I guess I don't understand what you're asking?
> 
> --
> Steve

Thanks a lot for your answer. Now:

Since we were noticing numerical issues in certain cases in our code, 
we moved on to a toy model. The toy model is based on a standard LU 
factorization with a dense block-distributed system matrix. So:

1/2) When we use gfortran+opencoarrays:
The verification computation of the numerical solution of the system 
of equations is OK if we use precision either (single, double, extended, 
quadruple) when the number Z of images is equal to 1. It is also OK if 
we use precision either (single, double) when Z>1. But it fails if we 
use precision either (extended, quadruple) when Z>1.

2/2) When we use ifort:
The verification computation of the numerical solution of the system 
of equations is OK if we use precision either (single, double, quadruple) 
either when Z=1 or when Z>1. We cannot check it in extended precision 
because ifort does not support the use of extended precision.

As a first attempt to explain the discrepancy, we assume that those 
verification failures in the solution could be attributed to 
gfortran+opencoarrays not quite correctly transmitting numbers in 
extended precision, because opencoarrays relies on some standard MPI 
for single and double precision (it would be like this?).

Best regards.
Jorge.
--
CIMEC (UNL-CONICET), cimec.conicet.gov.ar, www.cimec.org.ar

Re: coarrays using extended precision (80 bits) ?

[Steve Kargl]

On Sat, Apr 22, 2023 at 07:46:12AM -0300, Jorge D'Elia wrote:
> > On Fri, Apr 21, 2023 at 08:33:31AM -0300, Jorge D'Elia wrote:
> >> 
> >> One question: is there any chance of encoding with coarrays using
> >> extended precision (80 bits) at least inside a multicore computer?
> >> (as if to simplify a bit).
> >> 
> >> To date, the possibility of using double precision (64 bits) or
> >> extended precision (80 bits) is an alternative in our production
> >> code, but sometimes we would like to do computations in
> >> 80 bits and, in certain parts, there are coarrays.
> >> We have validated even in quadruple precision (128 bits), using
> >> ifort although, as is well known, the CPU times are largely
> >> excessive.
> >> 
> > Well, I just installed OpenCoarray and downloaded a pi/4
> > monte carlo code that Thomas wrote using REAL.  I changed
> > everything to use REAL(10).  Compiled and executed without
> > a problem.  I also tested REAL(16), which worked although
> > it's painfully slow due to software floating point.  So,
> > I guess I don't understand what you're asking?
> > 
> 
> Thanks a lot for your answer. Now:
> 
> Since we were noticing numerical issues in certain cases in our code, 
> we moved on to a toy model. The toy model is based on a standard LU 
> factorization with a dense block-distributed system matrix. So:
> 
> 1/2) When we use gfortran+opencoarrays:
> The verification computation of the numerical solution of the system 
> of equations is OK if we use precision either (single, double, extended, 
> quadruple) when the number Z of images is equal to 1. It is also OK if 
> we use precision either (single, double) when Z>1. But it fails if we 
> use precision either (extended, quadruple) when Z>1.
> 
> 2/2) When we use ifort:
> The verification computation of the numerical solution of the system 
> of equations is OK if we use precision either (single, double, quadruple) 
> either when Z=1 or when Z>1. We cannot check it in extended precision 
> because ifort does not support the use of extended precision.
> 
> As a first attempt to explain the discrepancy, we assume that those 
> verification failures in the solution could be attributed to 
> gfortran+opencoarrays not quite correctly transmitting numbers in 
> extended precision, because opencoarrays relies on some standard MPI 
> for single and double precision (it would be like this?).

This might be a bug in OC or gfortran or both.  It is unclear if
there is any further work being done on OC.  If the LU toy code
is short enough, you might try compiling it with -fcoarray=lib
-fdump-tree-original to see if there are any obvious function
argument mismatches in the underlying code.

I looked more closely at Thomas's code.  It was passing
integer arrays between images while the images internally
used REAL(10).  If it's an argument passing issues with
REAL(10), his code would not expose it.

-- 
Steve

Re: coarrays using extended precision (80 bits) ?

[Jorge D'Elia]

[-- Attachment #1: Type: text/plain, Size: 4120 bytes --]

----- Mensaje original -----
> De: "sgk" <sgk@troutmask.apl.washington.edu>
> Para: "Jorge D'Elia" 
> CC: "Gfortran List" 
> Enviado: Sábado, 22 de Abril 2023 10:07:33
> Asunto: Re: coarrays using extended precision (80 bits) ?
>
> On Sat, Apr 22, 2023 at 07:46:12AM -0300, Jorge D'Elia wrote:
>> > On Fri, Apr 21, 2023 at 08:33:31AM -0300, Jorge D'Elia wrote:
>> >> 
>> >> One question: is there any chance of encoding with coarrays using
>> >> extended precision (80 bits) at least inside a multicore computer?
>> >> (as if to simplify a bit).
>> >> 
>> >> To date, the possibility of using double precision (64 bits) or
>> >> extended precision (80 bits) is an alternative in our production
>> >> code, but sometimes we would like to do computations in
>> >> 80 bits and, in certain parts, there are coarrays.
>> >> We have validated even in quadruple precision (128 bits), using
>> >> ifort although, as is well known, the CPU times are largely
>> >> excessive.
>> >> 
>> > Well, I just installed OpenCoarray and downloaded a pi/4
>> > monte carlo code that Thomas wrote using REAL.  I changed
>> > everything to use REAL(10).  Compiled and executed without
>> > a problem.  I also tested REAL(16), which worked although
>> > it's painfully slow due to software floating point.  So,
>> > I guess I don't understand what you're asking?
>> > 
>> 
>> Thanks a lot for your answer. Now:
>> 
>> Since we were noticing numerical issues in certain cases in our code,
>> we moved on to a toy model. The toy model is based on a standard LU
>> factorization with a dense block-distributed system matrix. So:
>> 
>> 1/2) When we use gfortran+opencoarrays:
>> The verification computation of the numerical solution of the system
>> of equations is OK if we use precision either (single, double, extended,
>> quadruple) when the number Z of images is equal to 1. It is also OK if
>> we use precision either (single, double) when Z>1. But it fails if we
>> use precision either (extended, quadruple) when Z>1.
>> 
>> 2/2) When we use ifort:
>> The verification computation of the numerical solution of the system
>> of equations is OK if we use precision either (single, double, quadruple)
>> either when Z=1 or when Z>1. We cannot check it in extended precision
>> because ifort does not support the use of extended precision.
>> 
>> As a first attempt to explain the discrepancy, we assume that those
>> verification failures in the solution could be attributed to
>> gfortran+opencoarrays not quite correctly transmitting numbers in
>> extended precision, because opencoarrays relies on some standard MPI
>> for single and double precision (it would be like this?).
> 
> This might be a bug in OC or gfortran or both.  
> It is unclear if there is any further work being done on OC.  

Ok. We suspect (without any foundation) that the problem would 
be in the communication libraries (opencoarrays, gasnet, or 
openmpi).

> If the LU toy code is short enough, you might try compiling it 
> with -fcoarray=lib -fdump-tree-original to see if there are any 
> obvious function argument mismatches in the underlying code.

Ok. We made a shortened version of lu-toy-d.f90, added 
the -fdump-tree-original flag to the Makefile, and got 
the file lu-toy-block-d.f90.005t.original, included in 
the *.tgz file, although we don't quite know where to 
check more carefully...

> I looked more closely at Thomas's code.  It was passing
> integer arrays between images while the images internally
> used REAL(10).  If it's an argument passing issues with
> REAL(10), his code would not expose it.
> --
> Steve


Ok, thanks in advance. In addition, just in case it would 
help to look at the LU toy code too, a tgz file is attached 
that includes the source files, and the outputs obtained 
with (i) gfortran-opencoarrays through openmpi-gasnet; 
(ii) and with ifort. In the Makefile you have to predefine 
if it is gfortran or ifort, and include in the line of the 
Make command the value of the IRP integer (4, 8, 10, or 16).


Regards,
Jorge.
--

[-- Attachment #2: lu-toy.tgz --]
[-- Type: application/x-compressed-tar, Size: 147515 bytes --]