seemingly out of place errors?

seemingly out of place errors?

[Nicholas Murphy]

I'm getting errors like the following:

ERROR: Array overflow, check MAXMAPENTRIES near identifier '$filename'

...for a line that is essentially aliasing a string variable:

filename = $filename

...and similarly:

ERROR: Array overflow, check MAXMAPENTRIES near identifier '$fd'

...for:

if ($fd == 0 && track_utility && !(pid() in redirected_stdin)) {

Admittedly in the latter case there is a check against an array, but it seems to be flagging "$fd" not the array check.

Is some array simply concurrently overflowing elsewhere?  Why am I getting these particular errors?

Thanks,
Nick

Re: seemingly out of place errors?

[Josh Stone]

On 09/28/2012 09:46 AM, Nicholas Murphy wrote:
> I'm getting errors like the following:
> 
> ERROR: Array overflow, check MAXMAPENTRIES near identifier '$filename'
> 
> ...for a line that is essentially aliasing a string variable:
> 
> filename = $filename
> 
> ...and similarly:
> 
> ERROR: Array overflow, check MAXMAPENTRIES near identifier '$fd'
> 
> ...for:
> 
> if ($fd == 0 && track_utility && !(pid() in redirected_stdin)) {
> 
> Admittedly in the latter case there is a check against an array, but
> it seems to be flagging "$fd" not the array check.
> 
> Is some array simply concurrently overflowing elsewhere?  Why am I
> getting these particular errors?

Are these in .return probes?  That's my best guess; ignore the rest of
this reply if not...

When you read any $var (other than $return) in a .return probe, we
actually save the value on entry.  There are a few ways we do this
depending on context, but one way involves an array/map.  Since that map
doesn't exist in your source, we can only tie the error message to the
$var access which generated that code.

The map entries are supposed to remain balanced - added on entry,
removed on return.  But there are at least a few ways this can fail:

1- If this is a deeply recursive function, deeper than MAXMAPENTRIES,
then the number of saved entries will outgrow the map.  Increasing
MAXMAPENTRIES may help, if you can identify some upper bound.

2- If many instances of that function are simultaneously active for any
other reason, like having MANY threads, the same issue/solution applies.

3- If a thread terminates within that function for any reason, without
going through the return path, then those saved entries will be leaked.
 Increasing MAXMAPENTRIES here will only help if you can limit how often
this occurs.  To solve this generally, we might have to also synthesize
a thread-exit probe to delete these stale entries.


Josh

Re: seemingly out of place errors?

[Nicholas Murphy]

Hmm...interesting.  Yes, one is in a read.return and the other is in an open.return probe.  Your second theory seems the most likely.  Is there any way to verify that's what's going on, and is there any fix other than to try increasing MAXMAPENTRIES?

Thanks,
Nick

On Sep 28, 2012, at 1:27 PM, Josh Stone <jistone@redhat.com> wrote:

> On 09/28/2012 09:46 AM, Nicholas Murphy wrote:
>> I'm getting errors like the following:
>> 
>> ERROR: Array overflow, check MAXMAPENTRIES near identifier '$filename'
>> 
>> ...for a line that is essentially aliasing a string variable:
>> 
>> filename = $filename
>> 
>> ...and similarly:
>> 
>> ERROR: Array overflow, check MAXMAPENTRIES near identifier '$fd'
>> 
>> ...for:
>> 
>> if ($fd == 0 && track_utility && !(pid() in redirected_stdin)) {
>> 
>> Admittedly in the latter case there is a check against an array, but
>> it seems to be flagging "$fd" not the array check.
>> 
>> Is some array simply concurrently overflowing elsewhere?  Why am I
>> getting these particular errors?
> 
> Are these in .return probes?  That's my best guess; ignore the rest of
> this reply if not...
> 
> When you read any $var (other than $return) in a .return probe, we
> actually save the value on entry.  There are a few ways we do this
> depending on context, but one way involves an array/map.  Since that map
> doesn't exist in your source, we can only tie the error message to the
> $var access which generated that code.
> 
> The map entries are supposed to remain balanced - added on entry,
> removed on return.  But there are at least a few ways this can fail:
> 
> 1- If this is a deeply recursive function, deeper than MAXMAPENTRIES,
> then the number of saved entries will outgrow the map.  Increasing
> MAXMAPENTRIES may help, if you can identify some upper bound.
> 
> 2- If many instances of that function are simultaneously active for any
> other reason, like having MANY threads, the same issue/solution applies.
> 
> 3- If a thread terminates within that function for any reason, without
> going through the return path, then those saved entries will be leaked.
> Increasing MAXMAPENTRIES here will only help if you can limit how often
> this occurs.  To solve this generally, we might have to also synthesize
> a thread-exit probe to delete these stale entries.
> 
> 
> Josh

Re: seemingly out of place errors?

[Josh Stone]

On 09/28/2012 10:51 AM, Nicholas Murphy wrote:
> Hmm...interesting.  Yes, one is in a read.return and the other is in
> an open.return probe.  Your second theory seems the most likely.  Is
> there any way to verify that's what's going on, and is there any fix
> other than to try increasing MAXMAPENTRIES?

Well, you could track it in parallel yourself to confirm that there are
too many calls without returns, something like:

global count
probe YOURPROBE.call { ++count }
probe YOURPROBE.return { --count }
probe error { printf("script error with %d active calls\n", count) }

Once confirmed, there are a lot of possibilities to figure out why this
is happening.  Maybe start by adding printfs with pid(),tid(),etc. to
the call/return probes above to trace where these are happening.

Josh