Re: Handling pgoff in perf elf mmap/mmap2 elf info

[Milian Wolff <mail@milianw.de>]

[-- Attachment #1: Type: text/plain, Size: 4450 bytes --]

On Montag, 15. Oktober 2018 22:38:53 CEST Milian Wolff wrote:
> On Donnerstag, 11. Oktober 2018 20:14:43 CEST Milian Wolff wrote:
> > On Donnerstag, 11. Oktober 2018 19:37:07 CEST Mark Wielaard wrote:
> > > Hi,
> > > 
> > > My apologies for not having looked deeper at this.
> > > It is a bit tricky and I just didnt have enough time to
> > > really sit down and think it all through yet.
> > > 
> > > On Thu, Oct 11, 2018 at 05:02:18PM +0000, Ulf Hermann wrote:
> > > > is there any pattern in how the loader maps the ELF sections into
> > > > memory? What sections does it actually map and which of those do we
> > > > need
> > > > for unwinding?
> > > 
> > > Yes, it would be helpful to have some examples of mmap events plus
> > > the associated segment header (eu-readelf -l) of the ELF file.
> > > 
> > > Note that the kernel and dynamic loader will use the (PT_LOAD) segments,
> > > not the sections, to map things into memory. Each segment might contain
> > > multiple sections.
> > > 
> > > libdwfl then tries to associate the correct sections (and address bias)
> > > with how the ELF file was mapped into memory.
> > > 
> > > > I hope that only one of those MMAPs per ELF is actually meaningful and
> > > > we can simply add that one's pgoff as an extra member to Dwfl_Module
> > > > and
> > > > use it whenever we poke the underlying file.
> > > 
> > > One "trick" might be to just substract the pgoff from the load address.
> > > And so report as if the ELF file was being mapped from the start. This
> > > isn't really correct, but it might be interesting to see if that makes
> > > libdwfl able to just associate the whole ELF file with the correct
> > > address map.
> > 
> > I'll try to come up with some minimal code examples we can use to test all
> > of this. But from what I remember, neither of the above suggestions will
> > be
> > sufficient as we can still run into overlapping module errors from
> > elfutils
> > when we always load everything. I.e. I believe we've seen mappings that
> > eventually become partially obsoleted by a future mmap event. At that
> > point, we somehow need to be able to only map parts of a file, not all of
> > it. So just subtracting or honoring pgoff is not enough, I believe we also
> > need to be able to explicitly say how much of a file to map.
> > 
> > But to make this discussion easier to follow for others, I'll create some
> > standalone cpp code that takes a `perf script --show-mmap-events  | grep
> > PERF_RECORD_MMAP` input file and then runs this through elfutils API to
> > reproduce the issues we are facing.
> > 
> > I'll get back to you all once this is done.
> 
> Hey all,
> 
> here's one example of mmap events recorded by perf:
> 
> 0x7fac5ec0b000 to 0x7fac5ed9a000, len =   0x18f000, offset =               
> 0 r--p    /usr/lib/libstdc++.so.6.0.25
> 0x7fac5ec94000 to 0x7fac5ed8a000, len =    0xf6000, offset =         
> 0x89000 ---p    /usr/lib/libstdc++.so.6.0.25
> 0x7fac5ec94000 to 0x7fac5ed4c000, len =    0xb8000, offset =         
> 0x89000 r-xp    /usr/lib/libstdc++.so.6.0.25
> 0x7fac5ed4c000 to 0x7fac5ed89000, len =    0x3d000, offset =        
> 0x141000 r--p    /usr/lib/libstdc++.so.6.0.25
> 0x7fac5ed8a000 to 0x7fac5ed97000, len =     0xd000, offset =        
> 0x17e000 rw-p    /usr/lib/libstdc++.so.6.0.25

Spending more time on this issue, I've come up with a seemingly viable 
approach to workaround the libdwfl API limitations. Most notably, one must not 
take the raw mmap events and try to report them. Instead, we now associate the 
"base mmap", i.e. the first one with pgoff = 0, with the following mmaps for 
this file. Then, when we'd otherwise try to report one of the following mmaps, 
we lookup the base mmap addr and use that in our interaction with libdwfl.

Now I'm not seeing any "overlapping address" errors anymore, and unwinding 
seems to work fine again for perf files with mmap events like in the above.

There are still quite a few broken backtraces, but so far I can't say what the 
reason for this is.

So, from my POV, I'm fine - I have a viable workaround. But from the POV of 
future users of libdwfl, I still believe it would be useful to have more 
control over what gets reported and where. I.e. instead of having libdwfl 
analyze the PT_LOAD sections, offer an API that would allow us to feed the 
mmaped regions directly with pgoff etc.

Thanks for the input everyone

-- 
Milian Wolff
mail@milianw.de
http://milianw.de

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]