causes for an invalid memory reference?

causes for an invalid memory reference?

[Damian Rouson]

Gfortran developers,

Other than array indices out of bounds, could someone tell me what else
could cause the error "Segmentation fault - invalid memory reference" in
the following line?

  derivU(:,n+1) = sum( &
    [(PascalEntry(psn+k)*(derivU(:,n-k) .cross. derivB(:,k)),k=0,km)]&
  )

where derivU, derivB, and PascalEntry are non-allocatable arrays declared
with either module scope or submodule scope, none of the involved arrays,
has the pointer attribute, and the .cross. operator refers to a pure function
that computes the cross product from vector calculus.  A bit more context is
below.  Just before the problematic line, there is code (not shown below)
that verifies that the values used in all array indices are within bounds.

Damian

module particle

  real :: derivU(1:3, 0:5)

  interface operator(.cross.) !! cross-product operator
    module procedure cross
  end interface

  interface
     module subroutine evalDUn(np1, maxDB)
       implicit none
       integer, intent(in) :: np1, maxDB
     end subroutine

     pure module function cross(a, b) result(axb)
       implicit none
       real, intent(in) :: a(3), b(3)
       real axb(3)
     end function cross
  end interface

end module

submodule(particle) particle_impl

  real :: derivB(1:3, 0:4)

contains

module procedure evalDUn
  use taylor, only : PascalEntry
  integer k, km, n, psn

  !... use dummy arguments to set n, psn, km
  !... verify that all array indices below will be in bounds

  derivU(:,n+1) = sum( &
    [(PascalEntry(psn+k)*(derivU(:,n-k) .cross. derivB(:,k)),k=0,km)]&
  )
end procedure evalDUn

module procedure cross
  !... compute 3-vector cross product
end procedure

end submodule

Re: causes for an invalid memory reference?

[Steve Kargl]

On Sun, Feb 24, 2019 at 12:57:10PM -0800, Damian Rouson wrote:
> Gfortran developers,
> 
> Other than array indices out of bounds, could someone tell me what else
> could cause the error "Segmentation fault - invalid memory reference" in
> the following line?
> 
>   derivU(:,n+1) = sum( &
>     [(PascalEntry(psn+k)*(derivU(:,n-k) .cross. derivB(:,k)),k=0,km)]&
>   )
> 
> where derivU, derivB, and PascalEntry are non-allocatable arrays declared
> with either module scope or submodule scope, none of the involved arrays,
> has the pointer attribute, and the .cross. operator refers to a pure function
> that computes the cross product from vector calculus.  A bit more context is
> below.  Just before the problematic line, there is code (not shown below)
> that verifies that the values used in all array indices are within bounds.
> 

Clever uses of an implied-do-loop can cause very difficult to
fix bugs (either in one's code or a compiler).  I suspect that
you're getting one (or more) temporary array(s) with issues
with the bounds.  Using -fdump-tree-original might shed some
light on how the implied-do-loop and the user defined operator
are interacting.

-- 
Steve

Re: causes for an invalid memory reference?

[Damian Rouson]

On Sun, Feb 24, 2019 at 4:33 PM Steve Kargl
<sgk@troutmask.apl.washington.edu> wrote:
>
> Clever uses of an implied-do-loop can cause very difficult to
> fix bugs (either in one's code or a compiler).  I suspect that
> you're getting one (or more) temporary array(s) with issues
> with the bounds.  Using -fdump-tree-original might shed some
> light on how the implied-do-loop and the user defined operator
> are interacting.

Thanks, Steve.  I probably should have mentioned that the
problem was first identified with a regular do loop of the form

     derivU(:,np1) = 0.
     do k = 0, km
       uxb(:) = derivU(:,n-k) .cross. derivB(:,k)
       derivU(:,np1) = derivU(:,np1) + PascalEntry(psn + k) * uxb(:)
     end do

Part of the reason I switched to the implied do loop was to take
advantage of the SUM intrinsic function.  The other part was
because I try to make code as compact as possible before
submitting the code to compiler developers.  In this case, the
same behavior happens with the DO loop and the implied DO loop.

Damian

Re: causes for an invalid memory reference?

[Thomas König]

Damian,

> The other part was
> because I try to make code as compact as possible before
> submitting the code to compiler developers.  In this case, the
> same behavior happens with the DO loop and the implied DO loop.

Implied DO loops have their own code paths through the compiler, which are often more complex than for normal DO loops.

If you file a bug report with self-contained examples, please include variants like that, or we might miss the bug next door, so to speak 😉

Re: causes for an invalid memory reference?

[Steve Kargl]

On Sun, Feb 24, 2019 at 10:10:54PM -0800, Damian Rouson wrote:
> On Sun, Feb 24, 2019 at 4:33 PM Steve Kargl
> <sgk@troutmask.apl.washington.edu> wrote:
> >
> > Clever uses of an implied-do-loop can cause very difficult to
> > fix bugs (either in one's code or a compiler).  I suspect that
> > you're getting one (or more) temporary array(s) with issues
> > with the bounds.  Using -fdump-tree-original might shed some
> > light on how the implied-do-loop and the user defined operator
> > are interacting.
> 
> Thanks, Steve.  I probably should have mentioned that the
> problem was first identified with a regular do loop of the form
> 
>      derivU(:,np1) = 0.
>      do k = 0, km
>        uxb(:) = derivU(:,n-k) .cross. derivB(:,k)
>        derivU(:,np1) = derivU(:,np1) + PascalEntry(psn + k) * uxb(:)
>      end do
> 
> Part of the reason I switched to the implied do loop was to take
> advantage of the SUM intrinsic function.  The other part was
> because I try to make code as compact as possible before
> submitting the code to compiler developers.  In this case, the
> same behavior happens with the DO loop and the implied DO loop.
> 

This is valuable info, because I implied-do-loop can have odd issues
with scope.  I may have missed it.  What are the declared dimensions
of devivu and derivb.  In particular, is deriv[B,U] defined for 
k = 0 and/or n-k <= 0.

-- 
Steve

Re: causes for an invalid memory reference?

[Dan T. Abell]

> On 25 Feb 2019, at 00:19, Steve Kargl <sgk@troutmask.apl.washington.edu> wrote:
> 
> On Sun, Feb 24, 2019 at 10:10:54PM -0800, Damian Rouson wrote:
>> On Sun, Feb 24, 2019 at 4:33 PM Steve Kargl
>> <sgk@troutmask.apl.washington.edu> wrote:
>>> 
>>> Clever uses of an implied-do-loop can cause very difficult to
>>> fix bugs (either in one's code or a compiler).  I suspect that
>>> you're getting one (or more) temporary array(s) with issues
>>> with the bounds.  Using -fdump-tree-original might shed some
>>> light on how the implied-do-loop and the user defined operator
>>> are interacting.
>> 
>> Thanks, Steve.  I probably should have mentioned that the
>> problem was first identified with a regular do loop of the form
>> 
>>     derivU(:,np1) = 0.
>>     do k = 0, km
>>       uxb(:) = derivU(:,n-k) .cross. derivB(:,k)
>>       derivU(:,np1) = derivU(:,np1) + PascalEntry(psn + k) * uxb(:)
>>     end do
>> 
>> Part of the reason I switched to the implied do loop was to take
>> advantage of the SUM intrinsic function.  The other part was
>> because I try to make code as compact as possible before
>> submitting the code to compiler developers.  In this case, the
>> same behavior happens with the DO loop and the implied DO loop.
>> 
> 
> This is valuable info, because I implied-do-loop can have odd issues
> with scope.  I may have missed it.  What are the declared dimensions
> of devivu and derivb.  In particular, is deriv[B,U] defined for 
> k = 0 and/or n-k <= 0.

We found the bug, which was my fault.

Both deriv[B,U](:,k) are defined for k=0, k=n, and values in between. But there was another array that was not properly allocated. It should have been of size (3,34), but was instead (3,0)—mea maxima culpa. That array is fully populated (by a different subroutine) prior to the loop described above, but the traceback pointed instead to this loop. Any insight about this would be much appreciated.

Best Regards,
Dan

Re: causes for an invalid memory reference?

[Steve Kargl]

On Tue, Feb 26, 2019 at 02:11:09PM -0700, Dan T. Abell wrote:
> 
> > On 25 Feb 2019, at 00:19, Steve Kargl <sgk@troutmask.apl.washington.edu> wrote:
> > On Sun, Feb 24, 2019 at 10:10:54PM -0800, Damian Rouson wrote:
> >> On Sun, Feb 24, 2019 at 4:33 PM Steve Kargl
> >> <sgk@troutmask.apl.washington.edu> wrote:
> >>> 
> >>> Clever uses of an implied-do-loop can cause very difficult to
> >>> fix bugs (either in one's code or a compiler).  I suspect that
> >>> you're getting one (or more) temporary array(s) with issues
> >>> with the bounds.  Using -fdump-tree-original might shed some
> >>> light on how the implied-do-loop and the user defined operator
> >>> are interacting.
> >> 
> >> Thanks, Steve.  I probably should have mentioned that the
> >> problem was first identified with a regular do loop of the form
> >> 
> >>     derivU(:,np1) = 0.
> >>     do k = 0, km
> >>       uxb(:) = derivU(:,n-k) .cross. derivB(:,k)
> >>       derivU(:,np1) = derivU(:,np1) + PascalEntry(psn + k) * uxb(:)
> >>     end do
> >> 
> >> Part of the reason I switched to the implied do loop was to take
> >> advantage of the SUM intrinsic function.  The other part was
> >> because I try to make code as compact as possible before
> >> submitting the code to compiler developers.  In this case, the
> >> same behavior happens with the DO loop and the implied DO loop.
> > 
> > This is valuable info, because I implied-do-loop can have odd issues
> > with scope.  I may have missed it.  What are the declared dimensions
> > of devivu and derivb.  In particular, is deriv[B,U] defined for 
> > k = 0 and/or n-k <= 0.
> 
> We found the bug, which was my fault.
> 
> Both deriv[B,U](:,k) are defined for k=0, k=n, and values in
> between. But there was another array that was not properly
> allocated. It should have been of size (3,34), but was instead
> (3,0)—mea maxima culpa. That array is fully populated (by a different
> subroutine) prior to the loop described above, but the traceback
> pointed instead to this loop. Any insight about this would be
> much appreciated.
> 

Dan, thanks for the follow-up.  It's nice to see a bug that isn't
gfortran's fault.  :-)

If you were compiling with -O2 or higher optimization, the
backtrace can get messed up (for lack of better term).  As Damian
reported the issue, I assume the code uses coarrays (or perhaps
openmp), this might also cause some interesting issues.   Finally,
there is always the popular "Why does adding a PRINT statement
cause my code to segfault?".  When the array was incorrectly 
allocated and filled in the subroutine, it may have been in 
a place in memory were writing beyond the end of an array
just writes into unused random memory.  However, when you arrive
the line pointed to as the problem, something has found it
has been altered.

-- 
Steve
20170425 https://www.youtube.com/watch?v=VWUpyCsUKR4
20161221 https://www.youtube.com/watch?v=IbCHE-hONow