[Bug fortran/94129] New: Using almost any openacc !$acc directive causes ICE "compressed stream: data error"

[Bug fortran/94129] New: Using almost any openacc !$acc directive causes ICE "compressed stream: data error"

[andrew at blamsoft dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94129

            Bug ID: 94129
           Summary: Using almost any openacc !$acc directive causes ICE
                    "compressed stream: data error"
           Product: gcc
           Version: 10.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: fortran
          Assignee: unassigned at gcc dot gnu.org
          Reporter: andrew at blamsoft dot com
  Target Milestone: ---

I am using the Ubuntu gcc/gfortran 10 packages with nvptx offloading.
Specifically, the packages are gfortran-10 and gcc-10-offload-nvptx. The
specific versions are:

GNU Fortran (Ubuntu 10-20200304-1ubuntu1) 10.0.1 20200304 (experimental)
[master revision
0b0908c1f27:cb0a7e0ca53:94f7d7ec6ebef49a50da777fd71db3d03ee03aa0]

gcc (Ubuntu 10-20200304-1ubuntu1) 10.0.1 20200304 (experimental) [master
revision 0b0908c1f27:cb0a7e0ca53:94f7d7ec6ebef49a50da777fd71db3d03ee03aa0]

When trying to use openacc I get the following ICE.

0x8bc90a lto_uncompression_zlib
        ../../src-nvptx/gcc/lto-compress.c:393
0x8bc90a lto_end_uncompression(lto_compression_stream*, lto_compression)
        ../../src-nvptx/gcc/lto-compress.c:415
0x8bac1e lto_get_section_data(lto_file_decl_data*, lto_section_type, char
const*, int, unsigned long*, bool)
        ../../src-nvptx/gcc/lto-section-in.c:166
0x8ac51b lto_input_mode_table(lto_file_decl_data*)
        ../../src-nvptx/gcc/lto-streamer-in.c:1603
0x5c776d lto_file_finalize
        ../../src-nvptx/gcc/lto/lto-common.c:2200
0x5c776d lto_create_files_from_ids
        ../../src-nvptx/gcc/lto/lto-common.c:2237
0x5c776d lto_file_read
        ../../src-nvptx/gcc/lto/lto-common.c:2292
0x5c776d read_cgraph_and_symbols(unsigned int, char const**)
        ../../src-nvptx/gcc/lto/lto-common.c:2744
0x5b6466 lto_main()
        ../../src-nvptx/gcc/lto/lto.c:630
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.
mkoffload: fatal error: x86_64-linux-gnu-accel-nvptx-none-gcc-10 returned 1
exit status
compilation terminated.
lto-wrapper: fatal error:
/usr/lib/gcc/x86_64-linux-gnu/10//accel/nvptx-none/mkoffload returned 1 exit
status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status

I have found that using "data" and "host_data" work. Any other directive I try
such as "parallel" or "kernels" causes a crash. Here is a tutorial program I
found that causes the crash. I could not find any simple program online that
didn't crash.

program main

    ! Size of vectors
    integer :: n = 100000000

    ! Input vectors
    real(8),dimension(:),allocatable :: a
    real(8),dimension(:),allocatable :: b 
    ! Output vector
    real(8),dimension(:),allocatable :: c

    integer :: i
    real(8) :: sum

    ! Allocate memory for each vector
    allocate(a(n))
    allocate(b(n))
    allocate(c(n))

    ! Initialize content of input vectors, vector a[i] = sin(i)^2 vector b[i] =
cos(i)^2
    do i=1,n
        a(i) = sin(i*1D0)*sin(i*1D0)
        b(i) = cos(i*1D0)*cos(i*1D0) 
    enddo

    ! Sum component wise and save result into vector c

    !$acc kernels copyin(a(1:n),b(1:n)), copyout(c(1:n))
    do i=1,n
        c(i) = a(i) + b(i)
    enddo
    !$acc end kernels

    ! Sum up vector c and print result divided by n, this should equal 1 within
error
    do i=1,n
        sum = sum +  c(i)
    enddo
    sum = sum/n
    print *, 'final result: ', sum

    ! Release memory
    deallocate(a)
    deallocate(b)
    deallocate(c)

end program

Compile with:
gfortran -fopenacc program.f90

[Bug fortran/94129] Using almost any openacc !$acc directive causes ICE "compressed stream: data error"

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94129

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |openacc
                 CC|                            |doko at gcc dot gnu.org,
                   |                            |marxin at gcc dot gnu.org

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> ---
You should report this to Ubuntu, I suspect a build problem there, eventually
enabling zstd compression for gfortran-10 but zlib compression for
gcc-10-offload-nvptx (I'm not sure we inter-operate here - Martin?)

[Bug fortran/94129] Using almost any openacc !$acc directive causes ICE "compressed stream: data error"

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94129

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2020-03-11
             Status|UNCONFIRMED                 |NEW

--- Comment #2 from Martin Liška <marxin at gcc dot gnu.org> ---
(In reply to Richard Biener from comment #1)
> You should report this to Ubuntu, I suspect a build problem there,
> eventually enabling zstd compression for gfortran-10 but zlib compression
> for gcc-10-offload-nvptx (I'm not sure we inter-operate here - Martin?)

Yes, I bet one of the compilers does not have enabled ZSTD compression.
You can see it with gcc -v:

$ gcc -v
...
Supported LTO compression algorithms: zlib zstd
gcc version 10.0.1 20200310 (experimental) (GCC) 

Right now we should catch different compression algorithm:

void
lto_end_uncompression (struct lto_compression_stream *stream,
                       lto_compression compression)
{
#ifdef HAVE_ZSTD_H
  if (compression == ZSTD)
    {
      lto_uncompression_zstd (stream);
      return;
    }
#endif
  if (compression == ZSTD)
    internal_error ("compiler does not support ZSTD LTO compression");

  lto_uncompression_zlib (stream);
}

[Bug fortran/94129] Using almost any openacc !$acc directive causes ICE "compressed stream: data error"

[doko at debian dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94129

Matthias Klose <doko at debian dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |doko at debian dot org

--- Comment #3 from Matthias Klose <doko at debian dot org> ---
$ gfortran-10 -v -fopenacc program.f90 2>&1 |grep zstd
Supported LTO compression algorithms: zlib zstd
Supported LTO compression algorithms: zlib zstd

afaics both builds are correctly configured.

[Bug fortran/94129] Using almost any openacc !$acc directive causes ICE "compressed stream: data error"

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94129

--- Comment #4 from Richard Biener <rguenth at gcc dot gnu.org> ---
So I tried it with the SUSE GCC 10 packages and it works fine (I've
double-checked nvptx is offloaded).  But my packages are only configured for
zlib ...
(I'm testing on Leap 15.1 which doesn't have zstd I think).

[Bug fortran/94129] Using almost any openacc !$acc directive causes ICE "compressed stream: data error"

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94129

--- Comment #5 from Richard Biener <rguenth at gcc dot gnu.org> ---
Btw, your backtrace ends up in lto_uncompression_zlib but Matthias shows the
Ubuntu packages have zstd enabled.  I'd have expected only zstd compressed
sections there.  Matthias, can you reproduce the issue?

[Bug fortran/94129] Using almost any openacc !$acc directive causes ICE "compressed stream: data error"

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94129

--- Comment #6 from Martin Liška <marxin at gcc dot gnu.org> ---
(In reply to Matthias Klose from comment #3)
> $ gfortran-10 -v -fopenacc program.f90 2>&1 |grep zstd
> Supported LTO compression algorithms: zlib zstd
> Supported LTO compression algorithms: zlib zstd
> 
> afaics both builds are correctly configured.

Then you can dump lto header in a .o file:

$ gcc -flto -c bss.c
$ readelf -SW bss.o  | grep lto_.lto
  [ 6] .gnu.lto_.lto.2cfeb5fb4c8095f PROGBITS        0000000000000000 000062
000008 00   E  0   0  1

$ objdump -s -j .gnu.lto_.lto.2cfeb5fb4c8095f bss.o

bss.o:     file format elf64-x86-64

Contents of section .gnu.lto_.lto.2cfeb5fb4c8095f:
 0000 09000000 01000100                    ........        

that ending 0100 is zstd == 1.

[Bug fortran/94129] Using almost any openacc !$acc directive causes ICE "compressed stream: data error"

[doko at debian dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94129

--- Comment #7 from Matthias Klose <doko at debian dot org> ---
$ gfortran-10 -flto -c program.f90

$ readelf -SW program.o  | grep lto_.lto
  [ 6] .gnu.lto_.lto.c7eb6f75a94ea29a PROGBITS        0000000000000000 0000cc
000008 00   E  0   0  1

$ objdump -s -j .gnu.lto_.lto.c7eb6f75a94ea29a program.o 

program.o:     file format elf64-x86-64

Contents of section .gnu.lto_.lto.c7eb6f75a94ea29a:
 0000 09000000 01000100                    ........        

Yes, I get the very same stacktrace ending in lto_uncompression_zlib.

[Bug fortran/94129] Using almost any openacc !$acc directive causes ICE "compressed stream: data error"

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94129

--- Comment #8 from Martin Liška <marxin at gcc dot gnu.org> ---
@Richi: Can you please enable zstd for our nvptx cross compiler:

$ x86_64-suse-linux-accel-nvptx-none-gcc-10 -v
...
Supported LTO compression algorithms: zlib

We'll need to add
BuildRequires: libzstd-devel

into cross spec file.

[Bug fortran/94129] Using almost any openacc !$acc directive causes ICE "compressed stream: data error"

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94129

--- Comment #9 from Martin Liška <marxin at gcc dot gnu.org> ---
(In reply to Martin Liška from comment #8)
> @Richi: Can you please enable zstd for our nvptx cross compiler:
> 
> $ x86_64-suse-linux-accel-nvptx-none-gcc-10 -v
> ...
> Supported LTO compression algorithms: zlib
> 
> We'll need to add
> BuildRequires: libzstd-devel
> 
> into cross spec file.

@Richi: PING

[Bug fortran/94129] Using almost any openacc !$acc directive causes ICE "compressed stream: data error"

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94129

--- Comment #10 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Martin Liška from comment #9)
> (In reply to Martin Liška from comment #8)
> > @Richi: Can you please enable zstd for our nvptx cross compiler:
> > 
> > $ x86_64-suse-linux-accel-nvptx-none-gcc-10 -v
> > ...
> > Supported LTO compression algorithms: zlib
> > 
> > We'll need to add
> > BuildRequires: libzstd-devel
> > 
> > into cross spec file.
> 
> @Richi: PING

done

[Bug fortran/94129] Using almost any openacc !$acc directive causes ICE "compressed stream: data error"

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94129

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |DUPLICATE

--- Comment #11 from Martin Liška <marxin at gcc dot gnu.org> ---
Dup.

*** This bug has been marked as a duplicate of bug 94612 ***