[Bug fortran/66578] New: [F2008] Invalid free on allocate(...,source=a(:)) in block

[Bug fortran/66578] New: [F2008] Invalid free on allocate(...,source=a(:)) in block

[tkoenig at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

            Bug ID: 66578
           Summary: [F2008] Invalid free on allocate(...,source=a(:)) in
                    block
           Product: gcc
           Version: 6.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: fortran
          Assignee: unassigned at gcc dot gnu.org
          Reporter: tkoenig at gcc dot gnu.org
  Target Milestone: ---

$ cat a4.f90
program main
  integer :: n
  n = 42
  block
    real, dimension(0:n) :: a
    real, dimension(:), allocatable :: c
    call random_number(a)
    allocate(c,source=a(:))
  end block
end program main
$ gfortran -g a4.f90 
$ ./a.out
*** Error in `./a.out': free(): invalid pointer: 0x00000000020c2110 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7364f)[0x7fa5b086664f]
/lib64/libc.so.6(+0x78eae)[0x7fa5b086beae]
./a.out[0x400b0d]
./a.out[0x400b6b]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7fa5b0814be5]
./a.out[0x4007c9]
======= Memory map: ========
00400000-00401000 r-xp 00000000 08:04 4731063                           
/home/ig25/Krempel/Andre/a.out
00601000-00602000 r--p 00001000 08:04 4731063                           
/home/ig25/Krempel/Andre/a.out
00602000-00603000 rw-p 00002000 08:04 4731063                           
/home/ig25/Krempel/Andre/a.out
020bf000-020e0000 rw-p 00000000 00:00 0                                  [heap]

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[tkoenig at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

Thomas Koenig <tkoenig at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |wrong-code
                 CC|                            |vehre at gcc dot gnu.org

--- Comment #1 from Thomas Koenig <tkoenig at gcc dot gnu.org> ---
Well, here's one of the corner cases :-)

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[dominiq at lps dot ens.fr]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

Dominique d'Humieres <dominiq at lps dot ens.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2015-06-17
     Ever confirmed|0                           |1

--- Comment #2 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
I don't see the invalid free on x86_64-apple-darwin14, but if I compile the
code with -fsanitize=address, I get at run time

==49974==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x60f00000effc at pc 0x00010b20fa1d bp 0x7fff549f11f0 sp 0x7fff549f11e8
READ of size 4 at 0x60f00000effc thread T0
    #0 0x10b20fa1c 
(/Users/dominiq/Documents/Fortran/g95bench/win/f90/bug/a.out+0x100001a1c)
    #1 0x10b20fb67 
(/Users/dominiq/Documents/Fortran/g95bench/win/f90/bug/a.out+0x100001b67)
    #2 0x7fff8ecb55c8  (/usr/lib/system/libdyld.dylib+0x35c8)
    #3 0x0  (<unknown module>)

0x60f00000effc is located 0 bytes to the right of 172-byte region
[0x60f00000ef50,0x60f00000effc)
allocated by thread T0 here:
    #0 0x10b24755a  (/opt/gcc/gcc6w/lib/libasan.2.dylib+0x3255a)
    #1 0x10b20f761 
(/Users/dominiq/Documents/Fortran/g95bench/win/f90/bug/a.out+0x100001761)
    #2 0x10b20fb67 
(/Users/dominiq/Documents/Fortran/g95bench/win/f90/bug/a.out+0x100001b67)
    #3 0x7fff8ecb55c8  (/usr/lib/system/libdyld.dylib+0x35c8)
    #4 0x0  (<unknown module>)

SUMMARY: AddressSanitizer: heap-buffer-overflow ??:0 ??
Shadow bytes around the buggy address:
  0x1c1e00001da0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c1e00001db0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c1e00001dc0: fa fa fa fa fa fa fa fa fa fa fa fa 00 00 00 00
  0x1c1e00001dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1c1e00001de0: 00 04 fa fa fa fa fa fa fa fa 00 00 00 00 00 00
=>0x1c1e00001df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00[04]
  0x1c1e00001e00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c1e00001e10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c1e00001e20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c1e00001e30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c1e00001e40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
==49974==ABORTING

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[vehre at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

vehre at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |vehre at gcc dot gnu.org

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[vehre at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

--- Comment #4 from vehre at gcc dot gnu.org ---
Further analysis showed that while the offset of source's temporary descriptor
parm.3 is not as expected:

// allocate(c, source=a(:))
//       lb, ub,    , offset, data
parm.3 = {1, ub(a)+1,      0, &a[0]} // offset should be -1, when lb is set to
1.

c is initialized like this:

c = { parm.3.lb, parm.3.ub, -parm.3.lb, malloc( ((parm.3.ub - parm.3.lb) + 1) *
sizeof(int)) }

the loop to copy the data from "a", aka "parm.3.data", is this (simplified to
show the mistake more clearly):

for (i= 0; i <= c.ub; ++i)
  c.data[i + c.lb + c.offset] = parm.3.data[i + parm.3.offset]

The issue is that the loop is iterating over c.ub + 1 elements. It should
either be starting at 1 or stop at i < c.ub. Unfortunately is this computed by
the scalarizer, which I just don't understand. I assume, that having the offset
of parm.3 set correctly and the from setting of the loop to be 1 should resolve
the issue reliably. 

Any thoughts?

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[tkoenig at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

--- Comment #5 from Thomas Koenig <tkoenig at gcc dot gnu.org> ---
What I currently do not understand is why

    allocate(c,source=a(:))

fails and

    c = a(:)

works.

And yes, the scalarizer is pretty incomprehensible.

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[vehre at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

--- Comment #6 from vehre at gcc dot gnu.org ---
c= a(:) works because there is no additional array descriptor inbetween. 

The (new) allocate gets its own temporary array descriptor for the source=
expression, which in turn has incorrect bounds in it. The arrayspec is an
AS_DEFERRED which is then used to generate the temporary array descriptor and
defaults to a lower bound of zero, where all the miserable things begin with.

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[mikael at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

Mikael Morin <mikael at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mikael at gcc dot gnu.org

--- Comment #7 from Mikael Morin <mikael at gcc dot gnu.org> ---
(In reply to vehre from comment #4)
> Any thoughts?

Don't care about the scalarizer; generate a correct descriptor to begin with,
and the scalarizer will follow (hopefully).

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[tkoenig at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

--- Comment #9 from Thomas Koenig <tkoenig at gcc dot gnu.org> ---
(In reply to Mikael Morin from comment #8)
> diff --git a/gcc/fortran/trans-array.c b/gcc/fortran/trans-array.c
> index fece3ab..0b96de1 100644
> --- a/gcc/fortran/trans-array.c
> +++ b/gcc/fortran/trans-array.c
> @@ -7079,7 +7077,7 @@ gfc_conv_expr_descriptor (gfc_se *se, gfc_expr *expr)
>  	    {
>  	      tmp = gfc_conv_array_lbound (desc, n);
>  	      tmp = fold_build2_loc (input_location, MINUS_EXPR,
> -				     TREE_TYPE (base), tmp, loop.from[dim]);
> +				     TREE_TYPE (base), tmp, from);
>  	      tmp = fold_build2_loc (input_location, MULT_EXPR,
>  				     TREE_TYPE (base), tmp,
>  				     gfc_conv_array_stride (desc, n));

Unfortunately, this does not appear to fix the bug (at least not completely).
I still get an invalid free.
>From gcc-bugs-return-489367-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Fri Jun 19 06:42:12 2015
Return-Path: <gcc-bugs-return-489367-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 18121 invoked by alias); 19 Jun 2015 06:42:11 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 18071 invoked by uid 48); 19 Jun 2015 06:42:06 -0000
From: "glisse at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug tree-optimization/64130] vrp: handle non zero constant divided by range cannot be zero.
Date: Fri, 19 Jun 2015 06:42:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: tree-optimization
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords: missed-optimization
X-Bugzilla-Severity: enhancement
X-Bugzilla-Who: glisse at gcc dot gnu.org
X-Bugzilla-Status: NEW
X-Bugzilla-Resolution:
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-64130-4-pfCGlugtx0@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-64130-4@http.gcc.gnu.org/bugzilla/>
References: <bug-64130-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-06/txt/msg01699.txt.bz2
Content-length: 839

https://gcc.gnu.org/bugzilla/show_bug.cgi?idd130

--- Comment #6 from Marc Glisse <glisse at gcc dot gnu.org> ---
(In reply to kugan from comment #5)
> I think it should be in from front-end?

?

> Tried fixing it in VRP like:

You don't seem to use ranges at all. This might be the right place to implement
the suggestion from comment #2 (though if it does not use ranges, match.pd
would be better), but for the original optimization, what you want to improve
is the computation of the range of a division. When a has range [0, 4294967295]
we compute for 2305843009213693951 / a the range [0, 2305843009213693951] which
is not optimal, the left bound should be 536870912 not 0. If the good interval
is computed, VRP will automatically fold == 0 to false without extra code. We
already get this right when a has range [1, 4294967295].

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[mikael at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

--- Comment #11 from Mikael Morin <mikael at gcc dot gnu.org> ---
In gfc_conv_expr_descriptor, the bounds used to initialize the descriptor are
different from the ones passed to gfc_get_array_type_bounds.  That is the heart
of the problem, I think.
What I don't understand is how this could remain unnoticed for so long.

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[vehre at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

--- Comment #12 from vehre at gcc dot gnu.org ---
Created attachment 35806
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=35806&action=edit
Incomplete patch

The attached patch addresses some of the issues, but unfortunately does it open
some regressions in allocate_with_source_7 and allocate_with_source_8. I also
feel it is too much tailored to problem at hand, but may be it rings a bell...

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[mikael at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

--- Comment #13 from Mikael Morin <mikael at gcc dot gnu.org> ---
Created attachment 35823
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=35823&action=edit
next attempt

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[tkoenig at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

--- Comment #14 from Thomas Koenig <tkoenig at gcc dot gnu.org> ---
(In reply to Mikael Morin from comment #13)
> Created attachment 35823 [details]
> next attempt

Looks very good (fixes the test case and the variants I have here).

Regression-testing next.

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[vehre at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

--- Comment #16 from vehre at gcc dot gnu.org ---
I am getting a regression in char_length_5.f90. Anyone else?

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[vehre at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

vehre at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #35806|0                           |1
        is obsolete|                            |

--- Comment #17 from vehre at gcc dot gnu.org ---
Created attachment 35887
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=35887&action=edit
Extended version of Mikael's patch

I have encountered some other issues with the patch for 44672 that are not
fixed by Mikael's patch. Using Mikael's patch as a base I could resolve these
issues:

- when the rank in an assignment was decreased (i.e., the rank of the object(!)
on the rhs was greater then the one needed for the assignment, e.q. selecting a
row from a matrix, testcases are allocate_with_source_7/8), 
- twice indirect element addressing (e.q. using a section of a vector of
indices to address elements in a second array, as in the new testcase
allocate_with_source_9.f08 provided by the patch), and
- fixing char_length_8.

In most cases the issues occurred because the offset was not set correctly when
rebasing to one for the lower bound. 

Please have look whether the things to accomplish can be done easier. This
patch bootstraps and regtests fine x86_64-linux-gnu/f21.

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[vehre at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

--- Comment #18 from vehre at gcc dot gnu.org ---
Author: vehre
Date: Tue Jul  7 11:10:12 2015
New Revision: 225507

URL: https://gcc.gnu.org/viewcvs?rev=225507&root=gcc&view=rev
Log:
gcc/testsuite/ChangeLog:

2015-07-07  Andre Vehreschild  <vehre@gcc.gnu.org>

        PR fortran/66578
        * gfortran.dg/allocate_with_source_9.f08: New test.


gcc/fortran/ChangeLog:

2015-07-07  Mikael Morin  <mikael@gcc.gnu.org>
            Andre Vehreschild  <vehre@gcc.gnu.org>

        PR fortran/66578
        * trans-array.c (gfc_conv_expr_descriptor): Ensure array descriptor
        is one-based for non-full array refs. Correct the offset when a
        rank_remap occurs.


Added:
    trunk/gcc/testsuite/gfortran.dg/allocate_with_source_9.f08
Modified:
    trunk/gcc/fortran/ChangeLog
    trunk/gcc/fortran/trans-array.c
    trunk/gcc/testsuite/ChangeLog

[Bug fortran/66578] [F2008] Invalid free on allocate(...,source=a(:)) in block

[vehre at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66578

vehre at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #19 from vehre at gcc dot gnu.org ---
No objections so far. Closing as fixed with r225507.