[Bug fortran/101514] New: ICE: out of memory allocating 18446744073709551600 bytes

[Bug fortran/101514] New: ICE: out of memory allocating 18446744073709551600 bytes

[gscfq@t-online.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101514

            Bug ID: 101514
           Summary: ICE: out of memory allocating 18446744073709551600
                    bytes
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: fortran
          Assignee: unassigned at gcc dot gnu.org
          Reporter: gscfq@t-online.de
  Target Milestone: ---

Affects versions down to at least r5 :


$ cat z1.f90
program p
   type t
      integer :: a(..)
   end type
   type(t) :: x
   print *, transfer(1, x)
end


$ cat z2.f90
program p
   type t
      integer :: a(..)
   end type
   type(t) :: x
   b = transfer(1, x)
end


$ gfortran-12-20210718 -c z1.f90
z1.f90:3:22:

    3 |       integer :: a(..)
      |                      1
Error: Array component of structure at (1) must have an explicit shape

f951: out of memory allocating 18446744073709551600 bytes after a total of
618496 bytes

[Bug fortran/101514] ICE: out of memory allocating 18446744073709551600 bytes

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101514

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2021-07-20
     Ever confirmed|0                           |1

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> ---
gfortran.h:#define gfc_get_shape(rank) (XCNEWVEC (mpz_t, (rank)))

#4  0x0000000000ba3717 in gfc_interpret_derived (buffer=0x7fffffffd730 "\001", 
    buffer_size=4, result=0x392fe40)
    at /home/rguenther/src/gcc2/gcc/fortran/target-memory.c:540
540               e->shape = gfc_get_shape (e->rank);
(gdb) l
535               int n;
536
537               e->expr_type = EXPR_ARRAY;
538               e->rank = cmp->as->rank;
539
540               e->shape = gfc_get_shape (e->rank);
541               for (n = 0; n < e->rank; n++)
542                  {
543                    mpz_init_set_ui (e->shape[n], 1);
544                    mpz_add (e->shape[n], e->shape[n],
(gdb) p e->rank
$1 = -1

[Bug fortran/101514] ICE: out of memory allocating 18446744073709551600 bytes

[anlauf at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101514

anlauf at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |anlauf at gcc dot gnu.org

--- Comment #2 from anlauf at gcc dot gnu.org ---
Something like the following fixes the problem:

diff --git a/gcc/fortran/target-memory.c b/gcc/fortran/target-memory.c
index cfa8402dd3f..4be36bef856 100644
--- a/gcc/fortran/target-memory.c
+++ b/gcc/fortran/target-memory.c
@@ -534,6 +534,9 @@ gfc_interpret_derived (unsigned char *buffer, size_t
buffer_size, gfc_expr *resu
        {
          int n;

+         if (cmp->as->type == AS_ASSUMED_RANK || cmp->as->rank < 0)
+           return 0;
+
          e->expr_type = EXPR_ARRAY;
          e->rank = cmp->as->rank;

[Bug fortran/101514] ICE: out of memory allocating 18446744073709551600 bytes

[anlauf at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101514

anlauf at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |anlauf at gcc dot gnu.org

--- Comment #3 from anlauf at gcc dot gnu.org ---
Improved patch with enhanced testcase:

https://gcc.gnu.org/pipermail/fortran/2021-July/056253.html

[Bug fortran/101514] ICE: out of memory allocating 18446744073709551600 bytes

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101514

--- Comment #4 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Harald Anlauf <anlauf@gcc.gnu.org>:

https://gcc.gnu.org/g:c2b15fe27e6a0e42b108111d51acce69628593b4

commit r12-2439-gc2b15fe27e6a0e42b108111d51acce69628593b4
Author: Harald Anlauf <anlauf@gmx.de>
Date:   Wed Jul 21 18:54:00 2021 +0200

    Fortran: ICE, OOM while calculating sizes of derived type array components

    gcc/fortran/ChangeLog:

            PR fortran/101514
            * target-memory.c (gfc_interpret_derived): Size of array component
            of derived type can only be computed here for explicit shape.
            * trans-types.c (gfc_get_nodesc_array_type): Do not dereference
            NULL pointers.

    gcc/testsuite/ChangeLog:

            PR fortran/101514
            * gfortran.dg/pr101514.f90: New test.

[Bug fortran/101514] ICE: out of memory allocating 18446744073709551600 bytes

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101514

--- Comment #5 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Harald Anlauf
<anlauf@gcc.gnu.org>:

https://gcc.gnu.org/g:1645998adcba19d69634dad3baaf54a9c0a258ed

commit r11-8812-g1645998adcba19d69634dad3baaf54a9c0a258ed
Author: Harald Anlauf <anlauf@gmx.de>
Date:   Wed Jul 21 18:54:00 2021 +0200

    Fortran: ICE, OOM while calculating sizes of derived type array components

    gcc/fortran/ChangeLog:

            PR fortran/101514
            * target-memory.c (gfc_interpret_derived): Size of array component
            of derived type can only be computed here for explicit shape.
            * trans-types.c (gfc_get_nodesc_array_type): Do not dereference
            NULL pointers.

    gcc/testsuite/ChangeLog:

            PR fortran/101514
            * gfortran.dg/pr101514.f90: New test.

    (cherry picked from commit c2b15fe27e6a0e42b108111d51acce69628593b4)

[Bug fortran/101514] ICE: out of memory allocating 18446744073709551600 bytes

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101514

--- Comment #6 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-10 branch has been updated by Harald Anlauf
<anlauf@gcc.gnu.org>:

https://gcc.gnu.org/g:fa9729e7d2ee69fa0b18c925aeb3c2750874a05b

commit r10-10015-gfa9729e7d2ee69fa0b18c925aeb3c2750874a05b
Author: Harald Anlauf <anlauf@gmx.de>
Date:   Wed Jul 21 18:54:00 2021 +0200

    Fortran: ICE, OOM while calculating sizes of derived type array components

    gcc/fortran/ChangeLog:

            PR fortran/101514
            * target-memory.c (gfc_interpret_derived): Size of array component
            of derived type can only be computed here for explicit shape.
            * trans-types.c (gfc_get_nodesc_array_type): Do not dereference
            NULL pointers.

    gcc/testsuite/ChangeLog:

            PR fortran/101514
            * gfortran.dg/pr101514.f90: New test.

    (cherry picked from commit c2b15fe27e6a0e42b108111d51acce69628593b4)

[Bug fortran/101514] ICE: out of memory allocating 18446744073709551600 bytes

[anlauf at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101514

anlauf at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED
   Target Milestone|---                         |10.4

--- Comment #7 from anlauf at gcc dot gnu.org ---
Fixed on mainline for gcc-12, and on 11- and 10-branch.  Not a regression,
so not backporting further.  Closing.

Thanks for the report!