* [Bug general/29176] run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy
2022-05-24 21:14 [Bug general/29176] New: run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy evvers at ya dot ru
@ 2022-05-27 15:31 ` mark at klomp dot org
2022-05-27 16:04 ` evvers at ya dot ru
` (12 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: mark at klomp dot org @ 2022-05-27 15:31 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
Mark Wielaard <mark at klomp dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |mark at klomp dot org
--- Comment #1 from Mark Wielaard <mark at klomp dot org> ---
(In reply to Evgeny Vereshchagin from comment #0)
> I tried to switch to Ubuntu Jammy in
> https://github.com/evverx/elfutils/pull/83 and the test started failing
> there with
> ```
> FAIL: run-backtrace-native-biarch.sh
> ====================================
>
> case 0: expected symname 'raise' got '(null)'
> ./test-subr.sh: line 84: 23451 Aborted (core dumped)
> LD_LIBRARY_PATH="${built_library_path}${LD_LIBRARY_PATH:+:}$LD_LIBRARY_PATH"
> $VALGRIND_CMD "$@"
> backtrace-child-biarch: no main
> FAIL run-backtrace-native-biarch.sh (exit status: 1)
>
> ```
> It still passes on Ubuntu Focal.
>
> FWIW switching to Ubuntu Jammy somehow "fixed"
> run-debuginfod-fd-prefetch-caches.sh (which appears to be flaky on Ubuntu
> Focal and fails more or less consistently when elfutils is built with
> --enable-gcov: https://github.com/evverx/elfutils/runs/6577995202)
Note that github makes log non-public by default, so it is hard to see what is
going on.
Do you have any more information on what changed between "Focal" and "Jammy",
glibc upgrade? some system settings, gcc upgrade? That might explain what you
are seeing?
Basically the testcase says it cannot find the name associated with the frame.
It is NULL while it is expecting the symbol name "raise".
This is a somewhat gnarly test. Best might be to add some extra printfs to
tests/backtrace.c (callback_verify) printing the frameno and framename found to
see what is going on.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [Bug general/29176] run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy
2022-05-24 21:14 [Bug general/29176] New: run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy evvers at ya dot ru
2022-05-27 15:31 ` [Bug general/29176] " mark at klomp dot org
@ 2022-05-27 16:04 ` evvers at ya dot ru
2022-05-27 17:02 ` evvers at ya dot ru
` (11 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: evvers at ya dot ru @ 2022-05-27 16:04 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
--- Comment #2 from Evgeny Vereshchagin <evvers at ya dot ru> ---
> Do you have any more information on what changed between "Focal" and "Jammy", glibc upgrade? some system settings, gcc upgrade? That might explain what you are seeing?
I think everything was upgraded there. As far as I can see gcc-9.4.0 was
replaced with gcc-11.2.0 and glibc was upgraded from 2.31-0ubuntu9.7 to
2.35-0ubuntu3.
> Best might be to add some extra printfs to tests/backtrace.c (callback_verify) printing the frameno and framename found to see what is going on.
I'll try to do that.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [Bug general/29176] run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy
2022-05-24 21:14 [Bug general/29176] New: run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy evvers at ya dot ru
2022-05-27 15:31 ` [Bug general/29176] " mark at klomp dot org
2022-05-27 16:04 ` evvers at ya dot ru
@ 2022-05-27 17:02 ` evvers at ya dot ru
2022-05-27 20:58 ` evvers at ya dot ru
` (10 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: evvers at ya dot ru @ 2022-05-27 17:02 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
--- Comment #3 from Evgeny Vereshchagin <evvers at ya dot ru> ---
I added printf and here's what it printed on Ubuntu Jammy:
```
FRAMENO: '0', SYMNAME: '__kernel_vsyscall'
FRAMENO: '1', SYMNAME: ''
FRAMENO: '2', SYMNAME: 'raise'
FRAMENO: '3', SYMNAME: 'main'
FRAMENO: '4', SYMNAME: ''
FRAMENO: '5', SYMNAME: '__libc_start_main'
FRAMENO: '6', SYMNAME: '_start'
FRAMENO: '0', SYMNAME: '__kernel_vsyscall'
FRAMENO: '1', SYMNAME: ''
case 0: expected symname 'raise' got '(null)'
```
On Fedora 35 (where the test passes) I got
```
FRAMENO: '0', SYMNAME: '__kernel_vsyscall'
FRAMENO: '1', SYMNAME: '__pthread_kill_implementation'
FRAMENO: '2', SYMNAME: 'raise'
FRAMENO: '3', SYMNAME: 'main'
FRAMENO: '0', SYMNAME: '__kernel_vsyscall'
FRAMENO: '1', SYMNAME: '__pthread_kill_implementation'
FRAMENO: '2', SYMNAME: 'raise'
FRAMENO: '3', SYMNAME: 'sigusr2'
FRAMENO: '4', SYMNAME: 'stdarg'
FRAMENO: '5', SYMNAME: 'backtracegen'
FRAMENO: '6', SYMNAME: 'start'
FRAMENO: '7', SYMNAME: 'start_thread'
FRAMENO: '8', SYMNAME: '__clone3'
FRAMENO: '0', SYMNAME: ''
```
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [Bug general/29176] run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy
2022-05-24 21:14 [Bug general/29176] New: run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy evvers at ya dot ru
` (2 preceding siblings ...)
2022-05-27 17:02 ` evvers at ya dot ru
@ 2022-05-27 20:58 ` evvers at ya dot ru
2022-05-28 0:01 ` mark at klomp dot org
` (9 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: evvers at ya dot ru @ 2022-05-27 20:58 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
Evgeny Vereshchagin <evvers at ya dot ru> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |RESOLVED
Resolution|--- |INVALID
--- Comment #4 from Evgeny Vereshchagin <evvers at ya dot ru> ---
Looks like it's possible to make the test pass there by installing
libc6-i386-dbgsym (though I'm not sure why the test passes without that package
on Focal). Anyway it doesn't seem to be an elfutils issue. I'll go ahead and
close it. Thanks!
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [Bug general/29176] run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy
2022-05-24 21:14 [Bug general/29176] New: run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy evvers at ya dot ru
` (3 preceding siblings ...)
2022-05-27 20:58 ` evvers at ya dot ru
@ 2022-05-28 0:01 ` mark at klomp dot org
2022-05-28 0:22 ` evvers at ya dot ru
` (8 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: mark at klomp dot org @ 2022-05-28 0:01 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
Mark Wielaard <mark at klomp dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|INVALID |---
Ever confirmed|0 |1
Last reconfirmed| |2022-05-28
Status|RESOLVED |REOPENED
--- Comment #5 from Mark Wielaard <mark at klomp dot org> ---
I hope you don't mind me reopening the bug. I would like the testcase to work
even without the dbgsym installed.
Is the dbgsym package for the main (x86_64) libc6 package also installed?
The problem is that the testcase requires all addresses to map to a know symbol
name. But what we are really interested in is that we are unwinding through the
main or the start_thread symbols. We probably should just skip any unknown/NULL
symbols.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [Bug general/29176] run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy
2022-05-24 21:14 [Bug general/29176] New: run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy evvers at ya dot ru
` (4 preceding siblings ...)
2022-05-28 0:01 ` mark at klomp dot org
@ 2022-05-28 0:22 ` evvers at ya dot ru
2022-05-31 14:14 ` fantasquex at gmail dot com
` (7 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: evvers at ya dot ru @ 2022-05-28 0:22 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
--- Comment #6 from Evgeny Vereshchagin <evvers at ya dot ru> ---
> Is the dbgsym package for the main (x86_64) libc6 package also installed?
As far as I can see libc6-dbg is installed there but even without it when code
is compiled without -m32 and aborts backtraces don't contain NULL/unknown
symbols:
```
#0 0x00007ffff7e25a7c in pthread_kill () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007ffff7dd1476 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007ffff7db77f3 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x0000555555555161 in main ()
```
With -m32 and without the "x32" debug symbols backtraces look like
```
#0 0xf7fc4559 in __kernel_vsyscall ()
#1 0xf7e10e37 in ?? () from /lib32/libc.so.6
#2 0xf7dc04c5 in raise () from /lib32/libc.so.6
#3 0xf7da93ac in abort () from /lib32/libc.so.6
#4 0x565561b5 in main ()
```
> We probably should just skip any unknown/NULL symbols.
As far as I understand it should make the test pass even without the debug
symbols.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [Bug general/29176] run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy
2022-05-24 21:14 [Bug general/29176] New: run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy evvers at ya dot ru
` (5 preceding siblings ...)
2022-05-28 0:22 ` evvers at ya dot ru
@ 2022-05-31 14:14 ` fantasquex at gmail dot com
2022-09-13 10:06 ` jbglaw@lug-owl.de
` (6 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: fantasquex at gmail dot com @ 2022-05-31 14:14 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
--- Comment #7 from Letu Ren <fantasquex at gmail dot com> ---
*** Bug 29206 has been marked as a duplicate of this bug. ***
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [Bug general/29176] run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy
2022-05-24 21:14 [Bug general/29176] New: run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy evvers at ya dot ru
` (6 preceding siblings ...)
2022-05-31 14:14 ` fantasquex at gmail dot com
@ 2022-09-13 10:06 ` jbglaw@lug-owl.de
2023-03-04 11:24 ` jbglaw@lug-owl.de
` (5 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: jbglaw@lug-owl.de @ 2022-09-13 10:06 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
--- Comment #8 from Jan-Benedict Glaw <jbglaw@lug-owl.de> ---
I see this on my autobuilder as well (for run-backtrace-native-biarch.sh and
run-backtrace-native-core-biarch.sh), so keeping an eye on this.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [Bug general/29176] run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy
2022-05-24 21:14 [Bug general/29176] New: run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy evvers at ya dot ru
` (7 preceding siblings ...)
2022-09-13 10:06 ` jbglaw@lug-owl.de
@ 2023-03-04 11:24 ` jbglaw@lug-owl.de
2023-03-04 20:35 ` mark at klomp dot org
` (4 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: jbglaw@lug-owl.de @ 2023-03-04 11:24 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
--- Comment #9 from Jan-Benedict Glaw <jbglaw@lug-owl.de> ---
Is there already a decision on whether or not the tests should pass when
there's no dbgsym package installed for libc?
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [Bug general/29176] run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy
2022-05-24 21:14 [Bug general/29176] New: run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy evvers at ya dot ru
` (8 preceding siblings ...)
2023-03-04 11:24 ` jbglaw@lug-owl.de
@ 2023-03-04 20:35 ` mark at klomp dot org
2023-03-04 21:40 ` mark at klomp dot org
` (3 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: mark at klomp dot org @ 2023-03-04 20:35 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
--- Comment #10 from Mark Wielaard <mark at klomp dot org> ---
(In reply to Jan-Benedict Glaw from comment #9)
> Is there already a decision on whether or not the tests should pass when
> there's no dbgsym package installed for libc?
The test should pass even without the dbgsym package. The unwinder should work
without any extra debuginfo installed. It (now) also fails on debian-testing in
the buildbot:
https://builder.sourceware.org/buildbot/#/builders/elfutils-debian-testing-x86_64
PASS: run-backtrace-native.sh
SKIP: run-backtrace-native-core.sh
FAIL: run-backtrace-native-biarch.sh
SKIP: run-backtrace-native-core-biarch.sh
The skips there are because there are no core files created.
The issue seems to be a testcase issue, where it expects "real" symbol names,
even if it doesn't really matter.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [Bug general/29176] run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy
2022-05-24 21:14 [Bug general/29176] New: run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy evvers at ya dot ru
` (9 preceding siblings ...)
2023-03-04 20:35 ` mark at klomp dot org
@ 2023-03-04 21:40 ` mark at klomp dot org
2023-03-05 9:34 ` jbglaw@lug-owl.de
` (2 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: mark at klomp dot org @ 2023-03-04 21:40 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
--- Comment #11 from Mark Wielaard <mark at klomp dot org> ---
Proposed patch:
https://patchwork.sourceware.org/project/elfutils/patch/20230304213534.1448550-1-mark@klomp.org/
Did pass on the elfutils-debian-amd64 builder:
https://builder.sourceware.org/buildbot/#/builders/205/builds/58
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [Bug general/29176] run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy
2022-05-24 21:14 [Bug general/29176] New: run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy evvers at ya dot ru
` (10 preceding siblings ...)
2023-03-04 21:40 ` mark at klomp dot org
@ 2023-03-05 9:34 ` jbglaw@lug-owl.de
2023-03-05 12:16 ` mark at klomp dot org
2023-11-14 11:56 ` bschnzl at cotse dot net
13 siblings, 0 replies; 15+ messages in thread
From: jbglaw@lug-owl.de @ 2023-03-05 9:34 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
--- Comment #12 from Jan-Benedict Glaw <jbglaw@lug-owl.de> ---
Pulling in that patch works for me as well.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [Bug general/29176] run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy
2022-05-24 21:14 [Bug general/29176] New: run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy evvers at ya dot ru
` (11 preceding siblings ...)
2023-03-05 9:34 ` jbglaw@lug-owl.de
@ 2023-03-05 12:16 ` mark at klomp dot org
2023-11-14 11:56 ` bschnzl at cotse dot net
13 siblings, 0 replies; 15+ messages in thread
From: mark at klomp dot org @ 2023-03-05 12:16 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
Mark Wielaard <mark at klomp dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |FIXED
Status|REOPENED |RESOLVED
--- Comment #13 from Mark Wielaard <mark at klomp dot org> ---
(In reply to Jan-Benedict Glaw from comment #12)
> Pulling in that patch works for me as well.
Great. Pushed as:
commit a7f65495258933eaf361e82eb325c9d826b455d5 (HEAD -> master)
Author: Mark Wielaard <mark@klomp.org>
Date: Sat Mar 4 21:55:56 2023 +0100
tests: skip '(null)' symname frames in backtrace tests
Some setups might have some frames for unknown (null) functions
in the thread backtrace. Skip these frames instead of failing
immediately.
* tests/backtrace.c (callback_verify): Check and skip nulls_seen.
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
Signed-off-by: Mark Wielaard <mark@klomp.org>
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [Bug general/29176] run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy
2022-05-24 21:14 [Bug general/29176] New: run-backtrace-native-biarch.sh seems to fail on Ubuntu Jammy evvers at ya dot ru
` (12 preceding siblings ...)
2023-03-05 12:16 ` mark at klomp dot org
@ 2023-11-14 11:56 ` bschnzl at cotse dot net
13 siblings, 0 replies; 15+ messages in thread
From: bschnzl at cotse dot net @ 2023-11-14 11:56 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29176
Bill Scherr <bschnzl at cotse dot net> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |bschnzl at cotse dot net
--- Comment #14 from Bill Scherr <bschnzl at cotse dot net> ---
elfutils-0.190 solves
FAIL: run-backtrace-native-biarch.sh
and
FAIL: run-backtrace-native.sh
on Gentoo. Verified Patch included!
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 15+ messages in thread