dwfl_thread_getframes: no matching address range

dwfl_thread_getframes: no matching address range

[Kurt Roeckx]

[-- Attachment #1: Type: text/plain, Size: 326 bytes --]

Hi,

I finally uploaded the 0.164 version to Debian, but it's failing
in the test suite on most arches with an error message like:
dwfl_thread_getframes: no matching address range

You can see a full log here:
https://buildd.debian.org/status/fetch.php?pkg=elfutils&arch=powerpc&ver=0.164-1&stamp=1451167794


Kurt

Re: dwfl_thread_getframes: no matching address range

[Kurt Roeckx]

[-- Attachment #1: Type: text/plain, Size: 907 bytes --]

On Sun, Dec 27, 2015 at 09:38:19PM +0100, Mark Wielaard wrote:
> (Replied from my other email address, since it seems the list
>  isn't accepting my home email address for some reason.)
> 
> On Sun, Dec 27, 2015 at 07:29:03PM +0100, Kurt Roeckx wrote:
> > So it seems that something on the system changed causing the
> > breakage.
> 
> Found it. It is two things.
> 
> 1) For some architectures (the ones that fail) debian glibc seems to
>    have not been compiled with -fasynchronous-unwind-tables. This really
>    should be the default on all arches since otherwise various programs
>    will be unable to unwind/backtrace through all libraries. I believe
>    there was already a bug filed against gcc for this, but I cannot find
>    it now.

I already filed 2 bugs for this:
https://bugs.debian.org/746426 (gcc, you replied to it)
https://bugs.debian.org/746516 (glibc)


Kurt

Re: dwfl_thread_getframes: no matching address range

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 2413 bytes --]

(Replied from my other email address, since it seems the list
 isn't accepting my home email address for some reason.)

On Sun, Dec 27, 2015 at 07:29:03PM +0100, Kurt Roeckx wrote:
> So it seems that something on the system changed causing the
> breakage.

Found it. It is two things.

1) For some architectures (the ones that fail) debian glibc seems to
   have not been compiled with -fasynchronous-unwind-tables. This really
   should be the default on all arches since otherwise various programs
   will be unable to unwind/backtrace through all libraries. I believe
   there was already a bug filed against gcc for this, but I cannot find
   it now.

   For that reason you also need libc6-dbg installed to get proper
   backtraces. That contains the missing unwind info (in .debug_frame).
   You do have that installed as build dependency, which should have made
   the test succeed. But that doesn't work with the latest libc6-dbg
   because of...

2) The debug ELF file uses compressed ELF sections. This is IMHO not a
   very good idea for various reasons since it makes lots of things a lot
   slower since various programs, like elfutils, gdb, etc. cannot just
   lazily load the info they need, or use indexes, like gdb_index, to get
   directly at the information they need because they will have to read
   in whole sections of compressed data, that will need to be uncompressed
   first. Causing lots of disk/cpu load for something simple like a quick
   backtrace. So I would suggest to file a bug against glibc to not do
   this.

   This wouldn't actually be a problem if the compressed was the .zdebug
   style GNU compressed ELF sections. Which elfutils actually supports.
   But the sections are compressed in the new ELF gabi style, which as
   far as I know no release of binutils nor elfutils supports yet.
   (I assume debian sid uses a special patched or prerelease binutils
    to generate these gabi style compressed sections.)

   The good news is that I already have a working patch for this. I'll
   post a cleaned up version tomorrow. The bad news is that it fairly
   big. So I don't think it will be easy to backport. We really need to
   do a new release once this lands. If things work out we could do a
   0.165 release early January if needed. Ideally IMHO debian should fix
   bug 1 and/or 2 before that though.

Cheers,

Mark

Re: dwfl_thread_getframes: no matching address range

[Kurt Roeckx]

[-- Attachment #1: Type: text/plain, Size: 820 bytes --]

On Sun, Dec 27, 2015 at 05:47:50PM +0100, Mark Wielaard wrote:
> On Sun, Dec 27, 2015 at 01:26:59PM +0100, Kurt Roeckx wrote:
> > > Does the same test with 0.163 succeed on the same setup/arch?
> > > If not, did glibc change between the last successful run and
> > > the current failure on that architecture?
> > 
> > I didn't see that failure with 0.163.  The last 0.163 upload was
> > using glibc 2.19-19, the current version is 2.21-6.
> 
> Does the test in 0.164 fail against glibc 2.19-19 on ppc?

Building 0.164 in jessie (stable) works.  That's with libc6
2.19-18+deb8u1 and gcc-4.9 4.9.2-10.

> Does the test in 0.163 succeed against glibc 2.21-6 on ppc?

Building 0.163 in unstable also fails with the error.

So it seems that something on the system changed causing the
breakage.


Kurt

Re: dwfl_thread_getframes: no matching address range

[Kurt Roeckx]

[-- Attachment #1: Type: text/plain, Size: 2255 bytes --]

On Sun, Dec 27, 2015 at 05:47:50PM +0100, Mark Wielaard wrote:
> On Sun, Dec 27, 2015 at 01:26:59PM +0100, Kurt Roeckx wrote:
> > > Does the same test with 0.163 succeed on the same setup/arch?
> > > If not, did glibc change between the last successful run and
> > > the current failure on that architecture?
> > 
> > I didn't see that failure with 0.163.  The last 0.163 upload was
> > using glibc 2.19-19, the current version is 2.21-6.
> 
> Does the test in 0.164 fail against glibc 2.19-19 on ppc?
> Does the test in 0.163 succeed against glibc 2.21-6 on ppc?

I will try this soon, but I'm probably going to test it against
2.19-18+deb8u1 which should be close enough to the -19 version I
think.  (Not sure I can install the 2.19-19 version myself.)

> > > You probably need access to a failing setup to debug a bit.
> > > Most interesting would be to see whether or not the glibc.so
> > > has CFI for raise and which addresses that covers.
> > > $ eu-readelf --debug-dump=frame /lib/libc.so.6 | grep -4 '<raise>'
> > > should give you a hint.
> > 
> > That just returns:
> > eu-readelf: cannot get debug context descriptor: No DWARF information found
> >
> > (I get that message on working systems too.)
> 
> That is just a warning because eu-readelf will try to print both .eh_frame
> and .debug_frame information. It cannot find .debug_frame, but that
> shouldn't be necessary in this case.
>  
> > Without the grep it does return all kind of information, just
> > nothing about raise.  Both with a 2.19 and 2.21 libc6.so.
> 
> That is surprising. I found libc6_2.21-6_powerpc.deb at
> http://ftp.nl.debian.org/debian/pool/main/g/glibc/
> And indeed it doesn't seem to provide unwind info for "raise".
> 
> But I cannot find the 2.19-19_powerpc one there. Do you know where I
> could find it, to compare?

If you really want to test 2.19-19, it can be found at:
http://snapshot.debian.org/archive/debian/20150710T034220Z/pool/main/g/glibc/

> And if possible where to find the build logs of both and/or the
> debuginfo packages?

The libc6-dbg should be in the same directory in the pool.

Build logs can be found at:
https://buildd.debian.org/status/logs.php?pkg=elfutils&arch=powerpc


Kurt

Re: dwfl_thread_getframes: no matching address range

[Kurt Roeckx]

[-- Attachment #1: Type: text/plain, Size: 1595 bytes --]

On Sun, Dec 27, 2015 at 03:16:48AM +0100, Mark Wielaard wrote:
> On Sat, Dec 26, 2015 at 11:46:50PM +0100, Kurt Roeckx wrote:
> > I finally uploaded the 0.164 version to Debian, but it's failing
> > in the test suite on most arches with an error message like:
> > dwfl_thread_getframes: no matching address range
> > 
> > You can see a full log here:
> > https://buildd.debian.org/status/fetch.php?pkg=elfutils&arch=powerpc&ver=0.164-1&stamp=1451167794
> 
> That is strange indeed. Especially since a somewhat similar test
> run-backtrace-native.sh does seem to succeed. The output seems to
> indicate that we cannot find the unwind (.eh_frame CFI) for raise
> in this case. But there is not enough info to know for sure.
> 
> Does the same test with 0.163 succeed on the same setup/arch?
> If not, did glibc change between the last successful run and
> the current failure on that architecture?

I didn't see that failure with 0.163.  The last 0.163 upload was
using glibc 2.19-19, the current version is 2.21-6.

> You probably need access to a failing setup to debug a bit.
> Most interesting would be to see whether or not the glibc.so
> has CFI for raise and which addresses that covers.
> $ eu-readelf --debug-dump=frame /lib/libc.so.6 | grep -4 '<raise>'
> should give you a hint.

That just returns:
eu-readelf: cannot get debug context descriptor: No DWARF information found

(I get that message on working systems too.)

Without the grep it does return all kind of information, just
nothing about raise.  Both with a 2.19 and 2.21 libc6.so.


Kurt