* gfortran/OpenMP/declare target link issue
@ 2020-09-03 18:36 Ioannis E. Venetis
2020-09-05 14:35 ` Ioannis E. Venetis
0 siblings, 1 reply; 3+ messages in thread
From: Ioannis E. Venetis @ 2020-09-03 18:36 UTC (permalink / raw)
To: gcc-help
[-- Attachment #1: Type: text/plain, Size: 3955 bytes --]
Hello everyone,
I have a larger program that I am trying to convert so that the
computationally intensive part will run on an NVidia GPU using OpenMP.
However, I am running into trouble when compiling the program. The part
of the program to run on the GPU contains calls to subroutines, where
variables declared in a separate module are used. This seems to be
creating issues. I have reduced the problem to the attached files.
I compile as follows:
gfortran test_link.f90 common_vars.f90 parameters.f90 -O0 -fopenmp -Wall
-Wextra -o test_link
With the file test_link.f90 as attached, the program compiles and runs
without a problem.
If I remove the comments for the subroutine TEST() and comment out line
31 in test_link.f90 (the line "I = NR") the compilation gives the
following error:
ptxas /tmp/ccw3FqJD.o, line 52; error : Illegal operand type to
instruction 'ld'
ptxas /tmp/ccw3FqJD.o, line 52; error : Unknown symbol
'__common_vars_MOD_nr$linkptr'
ptxas fatal : Ptx assembly aborted due to errors
nvptx-as: ptxas returned 255 exit status
mkoffload: fatal error:
/home/myself/apps/gcc-10.2.0/bin/x86_64-pc-linux-gnu-accel-nvptx-none-gcc
returned 1 exit status
compilation terminated.
lto-wrapper: fatal error:
/home/myself/apps/gcc-10.2.0/libexec/gcc/x86_64-pc-linux-gnu/10.2.0//accel/nvptx-none/mkoffload
returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
Is this a bug in gfortran or have I misunderstood how DECLARE TARGET
LINK works in combination with using subroutines?
And a last point: If I use -O3 during compilation the program compiles
and runs fine in both cases. I assume that TEST() is inlined in this
case and the error disappears?
Any help to overcome this issue is more than welcome.
Best regards,
Ioannis E. Venetis
PS1: The problem happens with gcc 10.2 that I compiled myself:
$ ~/apps/gcc-10.2.0/bin/gfortran -v
Using built-in specs.
COLLECT_GCC=/home/myself/apps/gcc-10.2.0/bin/gfortran
COLLECT_LTO_WRAPPER=/home/myself/apps/gcc-10.2.0/libexec/gcc/x86_64-pc-linux-gnu/10.2.0/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
Target: x86_64-pc-linux-gnu
Configured with: ../gcc-10.2.0/configure
--enable-offload-targets=nvptx-none
--with-cuda-driver-include=/usr/local/cuda/include
--with-cuda-driver-lib=/usr/local/cuda/lib64 --disable-bootstrap
--disable-multilib --enable-languages=c,c++,fortran,lto
--prefix=/home/myself/apps/gcc-10.2.0
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10.2.0 (GCC)
PS2: Same problem also happens with gcc 9.3 as installed on Ubuntu 16.04
from apt:
$ gfortran -v
Using built-in specs.
COLLECT_GCC=gfortran
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu
9.3.0-10ubuntu2~16.04'
--with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs
--enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2
--prefix=/usr --with-gcc-major-version-only --program-suffix=-9
--program-prefix=x86_64-linux-gnu- --enable-shared
--enable-linker-build-id --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix --libdir=/usr/lib
--enable-nls --enable-clocale=gnu --enable-libstdcxx-debug
--enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new
--enable-gnu-unique-object --disable-vtable-verify --enable-plugin
--with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto
--enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64
--with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic
--enable-offload-targets=nvptx-none,hsa --without-cuda-driver
--enable-checking=release --build=x86_64-linux-gnu
--host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2~16.04)
[-- Attachment #2: common_vars.f90 --]
[-- Type: text/plain, Size: 126 bytes --]
MODULE COMMON_VARS
USE PARAMETERS
IMPLICIT NONE
!$OMP DECLARE TARGET LINK(NR)
INTEGER :: NR
END MODULE COMMON_VARS
[-- Attachment #3: parameters.f90 --]
[-- Type: text/plain, Size: 120 bytes --]
MODULE PARAMETERS
IMPLICIT NONE
INTEGER, PARAMETER :: MAX_SOURCE_POSITIONS = 100
END MODULE PARAMETERS
[-- Attachment #4: test_link.f90 --]
[-- Type: text/plain, Size: 1530 bytes --]
PROGRAM TEST_LINK
USE COMMON_VARS
USE PARAMETERS
IMPLICIT NONE
REAL :: XMO(MAX_SOURCE_POSITIONS), DCP(MAX_SOURCE_POSITIONS)
INTEGER :: IS1(MAX_SOURCE_POSITIONS), IS2(MAX_SOURCE_POSITIONS)
INTEGER :: X, Y, Z, MAX_X, MAX_Y, MAX_Z, ISOUR, I
MAX_X = 3
MAX_Y = 4
MAX_Z = 5
NR = 6
!$OMP TARGET MAP(TOFROM:IS1,IS2,DCP,XMO) MAP(TO:NR)
!$OMP TEAMS DISTRIBUTE PARALLEL DO COLLAPSE(3)
DO X = 1, MAX_X
DO Y = 1, MAX_Y
DO Z = 1, MAX_Z
ISOUR = (X - 1)*MAX_Y*MAX_Z + (Y - 1)*MAX_Z + Z
XMO(ISOUR) = 1.0 * NR
DCP(ISOUR) = 2.0 * NR
IS1(ISOUR) = 3 * NR
IS2(ISOUR) = 4 * NR
I = NR
! CALL TEST()
ENDDO ! End of z loop
ENDDO ! End of y loop
ENDDO ! End of x loop
!$OMP END TEAMS DISTRIBUTE PARALLEL DO
!$OMP END TARGET
DO X = 1, MAX_X
DO Y = 1, MAX_Y
DO Z = 1, MAX_Z
ISOUR = (X - 1)*MAX_Y*MAX_Z + (Y - 1)*MAX_Z + Z
WRITE(*, *) 'ISOUR = ', ISOUR, 'XMO = ', XMO(ISOUR), 'DCP = ', DCP(ISOUR), 'IS1 = ', IS1(ISOUR), 'IS2 = ', IS2(ISOUR)
ENDDO ! End of z loop
ENDDO ! End of y loop
ENDDO ! End of x loop
END PROGRAM TEST_LINK
! SUBROUTINE TEST()
! !$OMP DECLARE TARGET
! USE COMMON_VARS
! IMPLICIT NONE
! INTEGER I
! I = NR
! END SUBROUTINE TEST
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: gfortran/OpenMP/declare target link issue
2020-09-03 18:36 gfortran/OpenMP/declare target link issue Ioannis E. Venetis
@ 2020-09-05 14:35 ` Ioannis E. Venetis
2020-09-05 14:52 ` Alin Marin Elena
0 siblings, 1 reply; 3+ messages in thread
From: Ioannis E. Venetis @ 2020-09-05 14:35 UTC (permalink / raw)
To: gcc-help
Any comment on this? Are global variables defined in a module and used
in subroutines in the target region supported through the declare target
link mechanism currently on gcc? If so, am I doing something wrong on
how to use that mechanism? If not, is there any other approach I could
follow without too many changes in the code? In the real code I have
about 20 such variables and the call tree in the target region goes up
to 3 subroutines deep.
Ioannis E. Venetis
On 3/9/2020 9:36 μ.μ., Ioannis E. Venetis wrote:
> Hello everyone,
>
> I have a larger program that I am trying to convert so that the
> computationally intensive part will run on an NVidia GPU using OpenMP.
> However, I am running into trouble when compiling the program. The
> part of the program to run on the GPU contains calls to subroutines,
> where variables declared in a separate module are used. This seems to
> be creating issues. I have reduced the problem to the attached files.
>
> I compile as follows:
>
> gfortran test_link.f90 common_vars.f90 parameters.f90 -O0 -fopenmp
> -Wall -Wextra -o test_link
>
> With the file test_link.f90 as attached, the program compiles and runs
> without a problem.
>
> If I remove the comments for the subroutine TEST() and comment out
> line 31 in test_link.f90 (the line "I = NR") the compilation gives the
> following error:
>
> ptxas /tmp/ccw3FqJD.o, line 52; error : Illegal operand type to
> instruction 'ld'
> ptxas /tmp/ccw3FqJD.o, line 52; error : Unknown symbol
> '__common_vars_MOD_nr$linkptr'
> ptxas fatal : Ptx assembly aborted due to errors
> nvptx-as: ptxas returned 255 exit status
> mkoffload: fatal error:
> /home/myself/apps/gcc-10.2.0/bin/x86_64-pc-linux-gnu-accel-nvptx-none-gcc
> returned 1 exit status
> compilation terminated.
> lto-wrapper: fatal error:
> /home/myself/apps/gcc-10.2.0/libexec/gcc/x86_64-pc-linux-gnu/10.2.0//accel/nvptx-none/mkoffload
> returned 1 exit status
> compilation terminated.
> /usr/bin/ld: error: lto-wrapper failed
> collect2: error: ld returned 1 exit status
>
> Is this a bug in gfortran or have I misunderstood how DECLARE TARGET
> LINK works in combination with using subroutines?
>
> And a last point: If I use -O3 during compilation the program compiles
> and runs fine in both cases. I assume that TEST() is inlined in this
> case and the error disappears?
>
> Any help to overcome this issue is more than welcome.
>
> Best regards,
>
> Ioannis E. Venetis
>
> PS1: The problem happens with gcc 10.2 that I compiled myself:
>
> $ ~/apps/gcc-10.2.0/bin/gfortran -v
> Using built-in specs.
> COLLECT_GCC=/home/myself/apps/gcc-10.2.0/bin/gfortran
> COLLECT_LTO_WRAPPER=/home/myself/apps/gcc-10.2.0/libexec/gcc/x86_64-pc-linux-gnu/10.2.0/lto-wrapper
>
> OFFLOAD_TARGET_NAMES=nvptx-none
> Target: x86_64-pc-linux-gnu
> Configured with: ../gcc-10.2.0/configure
> --enable-offload-targets=nvptx-none
> --with-cuda-driver-include=/usr/local/cuda/include
> --with-cuda-driver-lib=/usr/local/cuda/lib64 --disable-bootstrap
> --disable-multilib --enable-languages=c,c++,fortran,lto
> --prefix=/home/myself/apps/gcc-10.2.0
> Thread model: posix
> Supported LTO compression algorithms: zlib
> gcc version 10.2.0 (GCC)
>
>
> PS2: Same problem also happens with gcc 9.3 as installed on Ubuntu
> 16.04 from apt:
>
> $ gfortran -v
> Using built-in specs.
> COLLECT_GCC=gfortran
> COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper
> OFFLOAD_TARGET_NAMES=nvptx-none:hsa
> OFFLOAD_TARGET_DEFAULT=1
> Target: x86_64-linux-gnu
> Configured with: ../src/configure -v --with-pkgversion='Ubuntu
> 9.3.0-10ubuntu2~16.04'
> --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs
> --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2
> --prefix=/usr --with-gcc-major-version-only --program-suffix=-9
> --program-prefix=x86_64-linux-gnu- --enable-shared
> --enable-linker-build-id --libexecdir=/usr/lib
> --without-included-gettext --enable-threads=posix --libdir=/usr/lib
> --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug
> --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new
> --enable-gnu-unique-object --disable-vtable-verify --enable-plugin
> --with-system-zlib --with-target-system-zlib=auto
> --enable-objc-gc=auto --enable-multiarch --disable-werror
> --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32
> --enable-multilib --with-tune=generic
> --enable-offload-targets=nvptx-none,hsa --without-cuda-driver
> --enable-checking=release --build=x86_64-linux-gnu
> --host=x86_64-linux-gnu --target=x86_64-linux-gnu
> Thread model: posix
> gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2~16.04)
>
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: gfortran/OpenMP/declare target link issue
2020-09-05 14:35 ` Ioannis E. Venetis
@ 2020-09-05 14:52 ` Alin Marin Elena
0 siblings, 0 replies; 3+ messages in thread
From: Alin Marin Elena @ 2020-09-05 14:52 UTC (permalink / raw)
To: Ioannis E. Venetis; +Cc: gcc-help
Dear Ioannis,
this is my experience with offloading using omp4... it is slightly out
of date but i suspect syntax did not change.
in the module that contains the data (i suspect in reality is a
glorified common block)
you need things like this
```
!$omp declare target(xxx,yyy,zzz,vxx,vyy,vzz,fxx,fyy,fzz)
```
then of course you will need to sync the data to device before the
call of the routine
```
!$omp target map(to: imcon) &
!$omp map(to: xxx,yyy,zzz)
```
Regards,
Alin
Without Questions there are no Answers!
______________________________________________________________________
Dr. Alin Marin ELENA
http://alin.elena.space/
______________________________________________________________________
On Sat, 5 Sep 2020 at 15:35, Ioannis E. Venetis <venetis@ceid.upatras.gr> wrote:
>
> Any comment on this? Are global variables defined in a module and used
> in subroutines in the target region supported through the declare target
> link mechanism currently on gcc? If so, am I doing something wrong on
> how to use that mechanism? If not, is there any other approach I could
> follow without too many changes in the code? In the real code I have
> about 20 such variables and the call tree in the target region goes up
> to 3 subroutines deep.
>
> Ioannis E. Venetis
>
> On 3/9/2020 9:36 μ.μ., Ioannis E. Venetis wrote:
> > Hello everyone,
> >
> > I have a larger program that I am trying to convert so that the
> > computationally intensive part will run on an NVidia GPU using OpenMP.
> > However, I am running into trouble when compiling the program. The
> > part of the program to run on the GPU contains calls to subroutines,
> > where variables declared in a separate module are used. This seems to
> > be creating issues. I have reduced the problem to the attached files.
> >
> > I compile as follows:
> >
> > gfortran test_link.f90 common_vars.f90 parameters.f90 -O0 -fopenmp
> > -Wall -Wextra -o test_link
> >
> > With the file test_link.f90 as attached, the program compiles and runs
> > without a problem.
> >
> > If I remove the comments for the subroutine TEST() and comment out
> > line 31 in test_link.f90 (the line "I = NR") the compilation gives the
> > following error:
> >
> > ptxas /tmp/ccw3FqJD.o, line 52; error : Illegal operand type to
> > instruction 'ld'
> > ptxas /tmp/ccw3FqJD.o, line 52; error : Unknown symbol
> > '__common_vars_MOD_nr$linkptr'
> > ptxas fatal : Ptx assembly aborted due to errors
> > nvptx-as: ptxas returned 255 exit status
> > mkoffload: fatal error:
> > /home/myself/apps/gcc-10.2.0/bin/x86_64-pc-linux-gnu-accel-nvptx-none-gcc
> > returned 1 exit status
> > compilation terminated.
> > lto-wrapper: fatal error:
> > /home/myself/apps/gcc-10.2.0/libexec/gcc/x86_64-pc-linux-gnu/10.2.0//accel/nvptx-none/mkoffload
> > returned 1 exit status
> > compilation terminated.
> > /usr/bin/ld: error: lto-wrapper failed
> > collect2: error: ld returned 1 exit status
> >
> > Is this a bug in gfortran or have I misunderstood how DECLARE TARGET
> > LINK works in combination with using subroutines?
> >
> > And a last point: If I use -O3 during compilation the program compiles
> > and runs fine in both cases. I assume that TEST() is inlined in this
> > case and the error disappears?
> >
> > Any help to overcome this issue is more than welcome.
> >
> > Best regards,
> >
> > Ioannis E. Venetis
> >
> > PS1: The problem happens with gcc 10.2 that I compiled myself:
> >
> > $ ~/apps/gcc-10.2.0/bin/gfortran -v
> > Using built-in specs.
> > COLLECT_GCC=/home/myself/apps/gcc-10.2.0/bin/gfortran
> > COLLECT_LTO_WRAPPER=/home/myself/apps/gcc-10.2.0/libexec/gcc/x86_64-pc-linux-gnu/10.2.0/lto-wrapper
> >
> > OFFLOAD_TARGET_NAMES=nvptx-none
> > Target: x86_64-pc-linux-gnu
> > Configured with: ../gcc-10.2.0/configure
> > --enable-offload-targets=nvptx-none
> > --with-cuda-driver-include=/usr/local/cuda/include
> > --with-cuda-driver-lib=/usr/local/cuda/lib64 --disable-bootstrap
> > --disable-multilib --enable-languages=c,c++,fortran,lto
> > --prefix=/home/myself/apps/gcc-10.2.0
> > Thread model: posix
> > Supported LTO compression algorithms: zlib
> > gcc version 10.2.0 (GCC)
> >
> >
> > PS2: Same problem also happens with gcc 9.3 as installed on Ubuntu
> > 16.04 from apt:
> >
> > $ gfortran -v
> > Using built-in specs.
> > COLLECT_GCC=gfortran
> > COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper
> > OFFLOAD_TARGET_NAMES=nvptx-none:hsa
> > OFFLOAD_TARGET_DEFAULT=1
> > Target: x86_64-linux-gnu
> > Configured with: ../src/configure -v --with-pkgversion='Ubuntu
> > 9.3.0-10ubuntu2~16.04'
> > --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs
> > --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2
> > --prefix=/usr --with-gcc-major-version-only --program-suffix=-9
> > --program-prefix=x86_64-linux-gnu- --enable-shared
> > --enable-linker-build-id --libexecdir=/usr/lib
> > --without-included-gettext --enable-threads=posix --libdir=/usr/lib
> > --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug
> > --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new
> > --enable-gnu-unique-object --disable-vtable-verify --enable-plugin
> > --with-system-zlib --with-target-system-zlib=auto
> > --enable-objc-gc=auto --enable-multiarch --disable-werror
> > --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32
> > --enable-multilib --with-tune=generic
> > --enable-offload-targets=nvptx-none,hsa --without-cuda-driver
> > --enable-checking=release --build=x86_64-linux-gnu
> > --host=x86_64-linux-gnu --target=x86_64-linux-gnu
> > Thread model: posix
> > gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2~16.04)
> >
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-09-05 14:52 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-03 18:36 gfortran/OpenMP/declare target link issue Ioannis E. Venetis
2020-09-05 14:35 ` Ioannis E. Venetis
2020-09-05 14:52 ` Alin Marin Elena
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).