[Bug fortran/99711] New: Crash when reading an allocated character array in namelist

[Bug fortran/99711] New: Crash when reading an allocated character array in namelist

[phil1691 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

            Bug ID: 99711
           Summary: Crash when reading an allocated character array in
                    namelist
           Product: gcc
           Version: 10.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: fortran
          Assignee: unassigned at gcc dot gnu.org
          Reporter: phil1691 at gmail dot com
  Target Milestone: ---

The following code crashes with a segmentation fault (gcc 10.2.0):
> cat alloc_nml.f90 
program allocnml
  implicit none

  character(len=:), dimension(:), allocatable :: cbulist_ru
  integer :: iluseg

  namelist /nam_bu_ru/ cbulist_ru

  allocate( character(len=10) :: cbulist_ru(5) )

  open(newunit=iluseg, file='list.nml', status='old', &
       action='read', form='formatted', access='sequential')

  read(unit=iluseg, nml=nam_bu_ru)

  print *, cbulist_ru

  close(unit=iluseg)
end program allocnml

With the following namelist file:
> cat list.nml
&NAM_BU_RU
 CBULIST_RU(1) = 'PRES'
 CBULIST_RU(2) = 'NEST'
 CBULIST_RU(3) = 'HTURB'
 CBULIST_RU(4) = 'VTURB'
/

Compilation:
> gfortran -g alloc_nml.f90

Execution:
> ./a.out 

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0  0x146bdba3349f in ???
#1  0x146bdbb57d7a in ???
#2  0x146bdc79df4b in nml_read_obj
        at ../../../gcc-10.2.0-srcdir/libgfortran/io/list_read.c:3142
#3  0x146bdc79ee8a in nml_get_obj_data
        at ../../../gcc-10.2.0-srcdir/libgfortran/io/list_read.c:3512
#4  0x146bdc7a78b7 in finalize_transfer
        at ../../../gcc-10.2.0-srcdir/libgfortran/io/transfer.c:4128
#5  0x400d48 in allocnml
        at
/home/waup/F90/bug_gcc/character_allocatable_namelist/alloc_nml.f90:14
#6  0x400e20 in main
        at
/home/waup/F90/bug_gcc/character_allocatable_namelist/alloc_nml.f90:19
Erreur de segmentation (core dumped)

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jvdelisle at gcc dot gnu.org
         Resolution|---                         |INVALID
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #1 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
F2018 standard section 13.10.3 List-directed Input

There is a note NOTE 13.29 at the end of the first sub-section 13.10.3.1"

"An allocatable, deferred-length character effective item does not have its
allocation status or allocated length changed as a result of list-directed
input."

This implies that if the strings of the array are not already allocated to a
resonable length, for example a string of blanks, then the read will attempt to
transfer the file contents into unallocated strings.

If the status is unallocated the list READ cannot do any allocation to change
its status to ALLOCATED.

Thus all bets are off if it crashes. The read routines have no idea of the size
to allocate the length to until the read is completed.  There may be other
places in the standard to clarify this, but looks like invalid fortran.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[kargl at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

kargl at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kargl at gcc dot gnu.org

--- Comment #2 from kargl at gcc dot gnu.org ---
(In reply to Jerry DeLisle from comment #1)
> F2018 standard section 13.10.3 List-directed Input
> 
> There is a note NOTE 13.29 at the end of the first sub-section 13.10.3.1"
> 
> "An allocatable, deferred-length character effective item does not have its
> allocation status or allocated length changed as a result of list-directed
> input."
> 
> This implies that if the strings of the array are not already allocated to a
> resonable length, for example a string of blanks, then the read will attempt
> to transfer the file contents into unallocated strings.

Doesn't the line

 allocate( character(len=10) :: cbulist_ru(5) )

that is the first executable statement in the program
allocate an array of 5 strings with length 10?

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|INVALID                     |---
   Last reconfirmed|                            |2021-03-23
             Status|RESOLVED                    |NEW
     Ever confirmed|0                           |1

--- Comment #3 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
(In reply to kargl from comment #2)
> (In reply to Jerry DeLisle from comment #1)
> > F2018 standard section 13.10.3 List-directed Input
> > 
> > There is a note NOTE 13.29 at the end of the first sub-section 13.10.3.1"
> > 
> > "An allocatable, deferred-length character effective item does not have its
> > allocation status or allocated length changed as a result of list-directed
> > input."
> > 
> > This implies that if the strings of the array are not already allocated to a
> > resonable length, for example a string of blanks, then the read will attempt
> > to transfer the file contents into unallocated strings.
> 
> Doesn't the line
> 
>  allocate( character(len=10) :: cbulist_ru(5) )
> 
> that is the first executable statement in the program
> allocate an array of 5 strings with length 10?

You are right Steve, tired eyes were "seeing" only the declaration.  In that
case this should be OK and I do see the segfault.  It is likely a front end
issue.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #4 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Setting the character length in the type spec enables this to run correctly, so
the string length is not getting set correctly in the allocate statement.

program allocnml
  implicit none

  character(len=10), dimension(:), allocatable :: cbulist_ru ! set here
  integer :: iluseg

  namelist /nam_bu_ru/ cbulist_ru

  allocate( character(len=10) :: cbulist_ru(5) )

  open(newunit=iluseg, file='list.nml', status='old', &
       action='read', form='formatted', access='sequential')

  read(unit=iluseg, nml=nam_bu_ru)

  print *, cbulist_ru

  close(unit=iluseg)
end program allocnml

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[sgk at troutmask dot apl.washington.edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #5 from Steve Kargl <sgk at troutmask dot apl.washington.edu> ---
On Tue, Mar 23, 2021 at 02:41:27AM +0000, jvdelisle at gcc dot gnu.org wrote:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711
> 
> --- Comment #4 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
> Setting the character length in the type spec enables this to run correctly, so
> the string length is not getting set correctly in the allocate statement.
> 
> program allocnml
>   implicit none
> 
>   character(len=10), dimension(:), allocatable :: cbulist_ru ! set here
>   integer :: iluseg
> 
>   namelist /nam_bu_ru/ cbulist_ru
> 
>   allocate( character(len=10) :: cbulist_ru(5) )
> 
>   open(newunit=iluseg, file='list.nml', status='old', &
>        action='read', form='formatted', access='sequential')
> 
>   read(unit=iluseg, nml=nam_bu_ru)
> 
>   print *, cbulist_ru
> 

I don't use namelist, so will defer to you on the details.
I input file only defines cbulist_ru(1) through cbulist_ru(4),
so the namelist initializes only an array section.  If that
is conforming, then technically the print statement is invalid,
because cbulist_ru(5) is uninitialized.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[philippe.wautelet at aero dot obs-mip.fr]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #6 from philippe.wautelet at aero dot obs-mip.fr ---
Hello,

The segfault occurs on the read line.

If I allocate the array with the right number of lines (allocate(
character(len=10) :: cbulist_ru(4)), I get exactly the same behavior
(segfault).

However, if I initialize the array just after its allocation:
cbulist_ru(:)=''
and runs the application, it prints:
> ./a.out 
 VTURB                                   
which is the last line of the variable in the namelist.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #7 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Digging further within gfc_resolve_dt which is resolving the READ statement,
one can find:

(gdb) p *e->symtree.n.sym.ts.u.cl
$31 = {length = 0x0, next = 0x0, length_from_typespec = false, backend_decl =
0x0, 
  passed_length = 0x0, resolved = 1}

e is the gfc_expr for cbulist_ru in the test case

I have checked both the gfc_expr itself and it's symtree and in all cases the
character length is NULL, not being intialized.  I suspect it is getting missed
in the resolution of the allocate statement.  At this point in gfc_resolve_dt
it is marked as having been resolved.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #8 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
I see nowhere in resolve.c (resolve_allocate_expr) any attempt to resolve the
chraracter length. I think it has been missed.  I have cc'ed Paul to see if he
has any thoughts.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[sgk at troutmask dot apl.washington.edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #9 from Steve Kargl <sgk at troutmask dot apl.washington.edu> ---
On Wed, Mar 24, 2021 at 02:41:08AM +0000, jvdelisle at gcc dot gnu.org wrote:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711
> 
> --- Comment #7 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
> Digging further within gfc_resolve_dt which is resolving the READ statement,
> one can find:
> 
> (gdb) p *e->symtree.n.sym.ts.u.cl
> $31 = {length = 0x0, next = 0x0, length_from_typespec = false, backend_decl =
> 0x0, 
>   passed_length = 0x0, resolved = 1}
> 
> e is the gfc_expr for cbulist_ru in the test case
> 
> I have checked both the gfc_expr itself and it's symtree and in all cases the
> character length is NULL, not being intialized.  I suspect it is getting missed
> in the resolution of the allocate statement.  At this point in gfc_resolve_dt
> it is marked as having been resolved.
> 

What does -ftree-dump-original show?  I wonder if gfortran is doing a
re-allocation on assignment when it shouldn't.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[sgk at troutmask dot apl.washington.edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #10 from Steve Kargl <sgk at troutmask dot apl.washington.edu> ---
On Wed, Mar 24, 2021 at 05:35:31AM +0000, sgk at troutmask dot
apl.washington.edu wrote:
>
> What does -ftree-dump-original show?  I wonder if gfortran is doing a
> re-allocation on assignment when it shouldn't.
> 

Nope.  It looks like a counting problem.  Here's a modified
testcase.

   program allocnml
   implicit none
   character(len=:), dimension(:), allocatable :: cbulist_ru
   integer :: iluseg
   namelist /nam_bu_ru/ cbulist_ru
   allocate( character(len=10) :: cbulist_ru(4) )
   cbulist_ru = 'abc'
   print *, size(cbulist_ru), len(cbulist_ru(1))
   open(newunit=iluseg, file='list.nml', status='old')
   read(unit=iluseg, nml=nam_bu_ru)
   print *, size(cbulist_ru), len(cbulist_ru(1))
   print *, cbulist_ru
   print *, '>'//cbulist_ru(1)//'<'
   print *, '>'//cbulist_ru(2)//'<'
   print *, '>'//cbulist_ru(3)//'<'
   print *, '>'//cbulist_ru(4)//'<'
   close(unit=iluseg)
   end program allocnml

I get

% gfcx -o z -O a.f90 && ./z
           4          10
           4          10
 VTURB     abc       abc       abc       
 >VTURB     <
 >abc       <
 >abc       <
 >abc       <

which suggests that during the READ, the individual array
elements aren't assigned.  That is, the array index is 
never incremented from 1 to 4.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #11 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Steve, I agree with your comment. Now in the READ at runtime there is a loop
specification that is initialized to allow the read process to loop through
these elements of an array. This works fine elsewhere, so I suspect, based on
what you see here, that this loopspec is not getting initialized correctly
meaning the array descriptor is uninitialized or otherwize corrupted.  It is a
bit late for me to look at this tonight, but as I get time I will examine where
we set the loop spec and where it comes from.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #12 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
This is interesting, compiling with the -g option for debugging.

Running a test case that is working with:

  character(len=10), dimension(:), allocatable :: cbulist_ru ! explicit char
len

Breakpoint 1, nml_read_obj (dtp=dtp@entry=0x7fffffffd260, 
    nl=nl@entry=0x5184c0, offset=offset@entry=0, 
    pprev_nl=pprev_nl@entry=0x7fffffffd068, 
    nml_err_msg=nml_err_msg@entry=0x7fffffffd100 "Internal namelist read
error", clow=1, chigh=0, nml_err_msg_size=200)
    at ../../../trunk/libgfortran/io/list_read.c:2886
2886      if (dtp->u.p.nml_read_error || !nl->touched)
(gdb) p nl
$1 = (namelist_info *) 0x5184c0
(gdb) p *nl
$2 = {type = BT_CHARACTER, var_name = 0x518530 "cbulist_ru", 
  mem_pos = 0x515e80, dtio_sub = 0x0, vtable = 0x0, touched = 1, len = 1, 
  var_rank = 1, size = 10, string_length = 10, dim = 0x518550, ls = 0x518570, 
  next = 0x0}

string_length as expected.

Running the failing test case with:

  character(len=:), dimension(:), allocatable :: cbulist_ru ! deferred len

Breakpoint 1, nml_read_obj (dtp=dtp@entry=0x7fffffffd240, 
    nl=nl@entry=0x5184c0, offset=offset@entry=0, 
    pprev_nl=pprev_nl@entry=0x7fffffffd048, 
    nml_err_msg=nml_err_msg@entry=0x7fffffffd0e0 "Internal namelist read
error", clow=1, chigh=0, nml_err_msg_size=200)
    at ../../../trunk/libgfortran/io/list_read.c:2886
2886      if (dtp->u.p.nml_read_error || !nl->touched)
(gdb) p nl->string_length 
$13 = 10
(gdb) p *nl
$14 = {type = BT_CHARACTER, var_name = 0x518530 "cbulist_ru", 
  mem_pos = 0x515e80, dtio_sub = 0x0, vtable = 0x0, touched = 1, len = 1, 
  var_rank = 1, size = 967676983855022080, string_length = 10, dim = 0x518550, 
  ls = 0x518570, next = 0x0}
(gdb) p *nl->dim
$15 = {_stride = 1, lower_bound = 1, _ubound = 5}


size is messed up badly.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #13 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Looking at the -fdump-tree-originals in the two cases:

The working case:

{.elem_len=10, .rank=1, .type=6});


The failing case:

D.3962 = (sizetype) NON_LVALUE_EXPR <.cbulist_ru>;

.....

{.elem_len=(unsigned long) SAVE_EXPR <D.3962>, .rank=1, .type=6});

Also one sees this:

Breakpoint 1, _gfortran_st_set_nml_var (dtp=0x7fffffffd2a0, var_addr=0x408910, 
    var_name=0x402053 "cbulist_ru", len=1, string_length=10, dtype=...)
    at ../../../trunk/libgfortran/io/transfer.c:4597
4597    {
(gdb) p *dtype
Structure has no component named operator*.
(gdb) p dtype
$1 = {elem_len = 14971996835529359360, version = 0, rank = 1 '\001', 
  type = 6 '\006', attribute = 0}

The call to _gfortran_st_set_nml_var is created by the frontend as shown in the
dump, the elem_len is bogus.

As far as I can tell then, the front-end is failing in the allocation
initializing the size correctly.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #14 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Created attachment 50473
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=50473&action=edit
-fdump-tree-original for failing case

Attached the dump file of the failing case.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[sgk at troutmask dot apl.washington.edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #15 from Steve Kargl <sgk at troutmask dot apl.washington.edu> ---
On Fri, Mar 26, 2021 at 01:47:28AM +0000, jvdelisle at gcc dot gnu.org wrote:
> --- Comment #12 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
> This is interesting, compiling with the -g option for debugging.
> 
> Running a test case that is working with:
> 
>   character(len=10), dimension(:), allocatable :: cbulist_ru ! explicit char
> len
> 
> Breakpoint 1, nml_read_obj (dtp=dtp@entry=0x7fffffffd260, 
>     nl=nl@entry=0x5184c0, offset=offset@entry=0, 
>     pprev_nl=pprev_nl@entry=0x7fffffffd068, 
>     nml_err_msg=nml_err_msg@entry=0x7fffffffd100 "Internal namelist read
> error", clow=1, chigh=0, nml_err_msg_size=200)
>     at ../../../trunk/libgfortran/io/list_read.c:2886
> 2886      if (dtp->u.p.nml_read_error || !nl->touched)
> (gdb) p nl
> $1 = (namelist_info *) 0x5184c0
> (gdb) p *nl
> $2 = {type = BT_CHARACTER, var_name = 0x518530 "cbulist_ru", 
>   mem_pos = 0x515e80, dtio_sub = 0x0, vtable = 0x0, touched = 1, len = 1, 
>   var_rank = 1, size = 10, string_length = 10, dim = 0x518550, ls = 0x518570, 
>   next = 0x0}
> 
> string_length as expected.
> 

Hmmm.  Another place that string length info is held?  gfortran
has ts->u.cl.length,  expr->value->character->length, and now 
nl->string_length.  Yep, I would say that the deferred length 
isn't being propagated into the nl struct.  Probably need to 
check for expr->ts.type == BT_CHARACTER and expr->ts.deferred
to grab the string length from the type spec.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #16 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
FWIW it also segfaults on:

  write(unit=6, nml=nam_bu_ru)

I have been digging around with gdb in trans-io.c and I can see the cl is 0x0
in the sym->ts.u.cl and I can find nothing else accessible at this point. I
think I am going to have to go back to the "trans" for the allocate which
should have the length resolved and stuff it into the sym.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #17 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
(In reply to Jerry DeLisle from comment #16)
> FWIW it also segfaults on:
> 
>   write(unit=6, nml=nam_bu_ru)
> 
> I have been digging around with gdb in trans-io.c and I can see the cl is
> 0x0 in the sym->ts.u.cl and I can find nothing else accessible at this
> point. I think I am going to have to go back to the "trans" for the allocate
> which should have the length resolved and stuff it into the sym.

In trans-stmt.c (gfc_trans_allocate (gfc_code * code))

I find:

(gdb) p *code->ext->alloc.ts.u.cl.length.value.integer._mp_d
$41 = 10

So it is there and there is code to handle it in that function. However, I do
not yet understand the inner workings of frontend to see what the problem is.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[philippe.wautelet at aero dot obs-mip.fr]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #18 from philippe.wautelet at aero dot obs-mip.fr ---
Hello,

What is the status of this bug?

It seems that it is not present any more in the 12.x and 13.x GCC versions.

11.4 does not crash but gives incorrect results.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[sgk at troutmask dot apl.washington.edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

--- Comment #19 from Steve Kargl <sgk at troutmask dot apl.washington.edu> ---
On Thu, Aug 31, 2023 at 12:58:15PM +0000, philippe.wautelet at aero dot
obs-mip.fr wrote:
> 
> What is the status of this bug?
> 
> It seems that it is not present any more in the 12.x and 13.x GCC versions.
> 

Looks like it is fixed by the patch for some other bug report.
A scan of gcc/fortran/ChangeLog shows a number patches that
tinker with the character length for deferred length strings.
I don't which one is the responsiblei, and since I don't use 
git (other than for trivial 'git diff' and 'git pull') doing a
bisection is beyond my skillset.

Someone should likely convert your testcase into something
that the testsuite can digest, and commit it.

[Bug fortran/99711] Crash when reading an allocated character array in namelist

[kargl at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99711

kargl at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Known to work|                            |12.2.0, 14.0
           Priority|P3                          |P4