Issues with probing multiple modules in script

Issues with probing multiple modules in script

[Klein, Matt]

Hi,

I'm having some strange issues trying to probe multiple modules in the same script. For starters:

[~/AmazonData/Source/stap]$ stap -V
SystemTap translator/driver (version 1.3/0.152 non-git sources)

If I try to probe a single module, everything works as expected. However if I probe multiple modules, it seems like probes in the first module found works fine, but any probes on subsequent modules don't actually work. I also see the following error for all modules other than the first:

WARNING: missing unwind/symbol data for module ...

Here is how I am invoking stap:
stap -p4 -m test -r /home/mklein/AmazonData/Source/linux test.stp

I have looked through the -vvvvv output, and it looks to me like all of the module probe points are being identified correctly, but then I see the WARNING message and it doesn't look like the probes ever actually get invoked.

I know that that the other module probes are correct, because if I probe them in isolation, everything works as expected.

Is there anything obvious I am doing wrong (like probing multiple modules from the same script doesn't work)? If not, what data can I provide to help get to the bottom of this?

Thanks,
Matt

--
Matt Klein
Senior Software Engineer
Amazon Web Services
mklein@amazon.com / 206-266-3347

Re: Issues with probing multiple modules in script

[Frank Ch. Eigler]

Hi -

mklein wrote:

> SystemTap translator/driver (version 1.3/0.152 non-git sources)

That's a little old, but probably not a factor.

> If I try to probe a single module, everything works as
> expected. However if I probe multiple modules, it seems like probes
> in the first module found works fine, but any probes on subsequent
> modules don't actually work. I also see the following error for all
> modules other than the first:
> WARNING: missing unwind/symbol data for module ...

This message indicates only that operations such as symname() /
backtrace() might not be able to cover those other modules.  (This
information is extracted during pass 3, in translate.cxx
dump_unwindsyms().)  There may be a bug here, but I couldn't reproduce
the problem here.  Could you share the "stap --vp 009 -p4 ..." output?

The probes' basic activity should be otherwise unaffected, meaning
that you should be able to place probes whereever.

> I have looked through the -vvvvv output, and it looks to me like all
> of the module probe points are being identified correctly, but then
> I see the WARNING message and it doesn't look like the probes ever
> actually get invoked.

Why do you think they're not getting invoked?  You can try "stap -t
..."  to get hit counts/times for individual probe points.  You could
try "stap -DDEBUG_SYMBOLS ..." to see some of the runtime
symbol-lookup-related activity.

> Is there anything obvious I am doing wrong (like probing multiple
> modules from the same script doesn't work)? [...]

It should all work smoothly.  We just need more information to figure
out what's going wrong.

- FChE