From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 48) id 357C63858D28; Tue, 31 Jan 2023 05:45:07 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 357C63858D28 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1675143907; bh=fx8x99BYuWTRjHFSDlz1Zk9MtXrwNvXdCfSpCR1RKNc=; h=From:To:Subject:Date:From; b=ez2ybl8zF6sm9k08UvvYcnBj4QxYBmq5BpiPYt0m2apG5zF5mEifMMXG6eTIN31hy lBQHIOsjKdmoWPnD0TaBGjc3I9AE2GkIFKznt66bb3+4/mrTql/DhzeIx1TxlsK6pf BPZpzI4A7/X3okzs6hHM3GChPQ6vEyfMVKD+XDF8= From: "Qi.Chen at windriver dot com" To: glibc-bugs@sourceware.org Subject: [Bug dynamic-link/30062] New: glibc 2.36 loader has intermittent segfault error on ppc64le Date: Tue, 31 Jan 2023 05:45:06 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: glibc X-Bugzilla-Component: dynamic-link X-Bugzilla-Version: 2.36 X-Bugzilla-Keywords: X-Bugzilla-Severity: normal X-Bugzilla-Who: Qi.Chen at windriver dot com X-Bugzilla-Status: UNCONFIRMED X-Bugzilla-Resolution: X-Bugzilla-Priority: P2 X-Bugzilla-Assigned-To: unassigned at sourceware dot org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version bug_status bug_severity priority component assigned_to reporter target_milestone Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://sourceware.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 List-Id: https://sourceware.org/bugzilla/show_bug.cgi?id=3D30062 Bug ID: 30062 Summary: glibc 2.36 loader has intermittent segfault error on ppc64le Product: glibc Version: 2.36 Status: UNCONFIRMED Severity: normal Priority: P2 Component: dynamic-link Assignee: unassigned at sourceware dot org Reporter: Qi.Chen at windriver dot com Target Milestone: --- Summary: syslog-ng loads and then unloads a few shared libraries at runtime, and on ppc64le, we have intermittent segfault error. I enabled LD_DEBUG and the la= st few line in log is: calling fini: /usr/lib64/syslog-ng/librate-limit-filter= .so [0]. More detailed bug report are as below. glibc version: 2.36 (the latest release/2.36/master has the same issue) build host: x86-64; target: ppc64le options given to configure (except a few cross-compilation specific options= ):=20 --enable-kernel=3D3.10.0 \ --disable-profile \ --disable-debug \ --without-gd \ --enable-clocale=3Dgnu \ --without-selinux \ --enable-tunables \ --enable-bind-now \ --enable-stack-protector=3Dstrong \ --disable-crypt \ --with-default-link \ --disable-werror \ --with-cpu=3Dpower9 \ --disable-static \ --enable-memory-tagging \ --enable-nscd kernel: 6.1.5 gcc version: 12.2.0 More detailed description: Running syslog-ng on qemuppc64 gives us intermittent segfault error. The easiest way to reproduce the bug is running `syslog-ng -V' a few times. I checked the source codes of syslog-ng, what it does is doing loading & unloading of shared libraries under /usr/lib64/syslog-ng to get some basic information of the libraries. What it really invokes is 'g_module_open' and 'g_module_close'. Below are the error messages: [ 41.117221] syslog-ng[187]: segfault (11) at 7fffcb0cd9a0 nip 7fffcb0cd9a0= lr 7fffcb0cd9a0 code 2 [ 41.118589] syslog-ng[187]: code: cb0cd9a0 00007fff 24d0cca0 00000001 b41b= 3200 00007fff b539b770 00007fff [ 41.118734] syslog-ng[187]: code: 00000001 00000000 24d0bd10 00000001 0000= 7fff 24d24370 00000001 Segmentation fault (core dumped) backtrace (note that it could segfault on any .so file, the libkvformat.so = is just one example): Core was generated by `/usr/sbin/syslog-ng -F --enable-core --cfgfile /etc/syslog-ng/syslog-ng.conf --'. Program terminated with signal SIGSEGV, Segmentation fault. --Type for more, q to quit, c to continue without paging-- #0 0x00007fff9ace34c8 in __do_global_dtors_aux () from /usr/lib64/syslog-ng/libkvformat.so (gdb) bt #0 0x00007fff9ace34c8 in __do_global_dtors_aux () from /usr/lib64/syslog-ng/libkvformat.so #1 0x00007fff9ace34b8 in __do_global_dtors_aux () from /usr/lib64/syslog-ng/libkvformat.so #2 0x00007fff9ba401d4 in __GI__dl_catch_exception (exception=3D0x7fff9acff4= 58, operate=3D0x0, args=3D0x11cc50ff0) at dl-error-skeleton.c:182 #3 0x00007fff9be12d3c in _dl_close_worker (force=3Dfalse, map=3D) at dl-close.c:292 #4 _dl_close (_map=3D) at dl-close.c:818 #5 0x00007fff9ba40174 in __GI__dl_catch_exception ( exception=3Dexception@entry=3D0x7ffff25d7230, operate=3D0x7fff9be12930 <_dl_close>, args=3D0x11cc50ff0) at dl-error-skeleton.c:208 #6 0x00007fff9ba4025c in __GI__dl_catch_error (objname=3D0x7ffff25d72b8, errstring=3D0x7ffff25d72b0, mallocedp=3D0x7ffff25d72c0, operate=3D, args=3D) at dl-error-skeleton.c:227 #7 0x00007fff9be41888 in _rtld_catch_error (objname=3D, errstring=3D, mallocedp=3D, operate=3D, args=3D) at dl-error-skeleton.c:260 #8 0x00007fff9b928378 in _dlerror_run (operate=3D, args=3D) at dlerror.c:138 #9 0x00007fff9b927f3c in __dlclose (handle=3D) at dlclose.c:31 #10 0x00007fff9b4f18f8 in _g_module_close (handle=3D) --Type for more, q to quit, c to continue without paging-- at /usr/src/debug/glib-2.0/1_2.74.4-r0/gmodule/gmodule-dl.c:190 #11 g_module_close (module=3D0x11cc4e8a0) at /usr/src/debug/glib-2.0/1_2.74.4-r0/gmodule/gmodule.c:753 #12 0x00007fff9bcf6350 in plugin_discover_candidate_modules ( context=3D0x11cc46630) at /usr/src/debug/syslog-ng/3.38.1-r0/lib/plugin.c:5= 32 #13 0x00007fff9bcc6828 in cfg_discover_candidate_modules (self=3D) at /usr/src/debug/syslog-ng/3.38.1-r0/lib/cfg.c:208 #14 cfg_discover_candidate_modules (self=3D) at /usr/src/debug/syslog-ng/3.38.1-r0/lib/cfg.c:203 #15 0x00007fff9bcc787c in cfg_read_config (self=3D, fname=3D, preprocess_into=3D) at /usr/src/debug/syslog-ng/3.38.1-r0/lib/cfg.c:655 #16 0x00007fff9bceb2b0 in main_loop_read_and_init_config ( self=3D0x7fff9bdc35b0 ) at /usr/src/debug/syslog-ng/3.38.1-r0/lib/mainloop.c:618 #17 0x000000011cbe21e0 in main (argc=3D, argv=3D) at /usr/src/debug/syslog-ng/3.38.1-r0/syslog-ng/main.c:284 The last few lines of the LD_DEBUG_OUTPUT when segfault happens after LD_DE= BUG is enabled (as said above, it could segfault on any .so file, this time it's librate-limit-filter.so: 306: symbol=3Dg_module_check_init; lookup in file=3D/usr/lib64/libssl.so.3 = [0] 306: symbol=3Dg_module_check_init; lookup in file=3D/usr/lib64/libcrypto.so= .3 [0] 306: symbol=3Dg_module_check_init; lookup in file=3D/lib64/libc.so.6 [0] 306: symbol=3Dg_module_check_init; lookup in file=3D/lib64/ld64.so.2 [0] 306: symbol=3Dg_module_check_init; lookup in file=3D/lib64/libcap.so.2 [0] 306: symbol=3Dg_module_check_init; lookup in file=3D/usr/lib64/libzstd.so.1= [0] 306: symbol=3Dg_module_check_init; lookup in file=3D/usr/lib64/libpcre2-8.s= o.0 [0] 306: /usr/lib64/syslog-ng/librate-limit-filter.so: error: symbol lookup err= or: undefined symbol: g_module_check_init (fatal) 306: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/syslog-ng/librate-limit-filter.so [0] 306: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libsyslog-ng-3.3= 8.so.0 [0] 306: symbol=3Dg_module_unload; lookup in file=3D/lib64/libm.so.6 [0] 306: symbol=3Dg_module_unload; lookup in file=3D/lib64/librt.so.1 [0] 306: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libevtlog-3.38.s= o.0 [0] 306: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libpcre.so.1 [0] 306: symbol=3Dg_module_unload; lookup in file=3D/lib64/libsystemd.so.0 [0] 306: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libsecret-storag= e.so.0 [0] 306: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libgmodule-2.0.s= o.0 [0] 306: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libgthread-2.0.s= o.0 [0] 306: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libglib-2.0.so.0= [0] 306: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libssl.so.3 [0] 306: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libcrypto.so.3 [= 0] 306: symbol=3Dg_module_unload; lookup in file=3D/lib64/libc.so.6 [0] 306: symbol=3Dg_module_unload; lookup in file=3D/lib64/ld64.so.2 [0] 306: symbol=3Dg_module_unload; lookup in file=3D/lib64/libcap.so.2 [0] 306: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libzstd.so.1 [0] 306: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libpcre2-8.so.0 = [0] 306: /usr/lib64/syslog-ng/librate-limit-filter.so: error: symbol lookup err= or: undefined symbol: g_module_unload (fatal) 306: symbol=3Dmodule_info; lookup in file=3D/usr/lib64/syslog-ng/librate-limit-filter.so [0] 306: 306: calling fini: /usr/lib64/syslog-ng/librate-limit-filter.so [0] 306: As a comparison, here are a few lines when no segfault happens: 305: symbol=3Dg_module_check_init; lookup in file=3D/usr/lib64/libglib-2.0.= so.0 [0] 305: symbol=3Dg_module_check_init; lookup in file=3D/usr/lib64/libssl.so.3 = [0] 305: symbol=3Dg_module_check_init; lookup in file=3D/usr/lib64/libcrypto.so= .3 [0] 305: symbol=3Dg_module_check_init; lookup in file=3D/lib64/libc.so.6 [0] 305: symbol=3Dg_module_check_init; lookup in file=3D/lib64/ld64.so.2 [0] 305: symbol=3Dg_module_check_init; lookup in file=3D/lib64/libcap.so.2 [0] 305: symbol=3Dg_module_check_init; lookup in file=3D/usr/lib64/libzstd.so.1= [0] 305: symbol=3Dg_module_check_init; lookup in file=3D/usr/lib64/libpcre2-8.s= o.0 [0] 305: /usr/lib64/syslog-ng/libappmodel.so: error: symbol lookup error: undef= ined symbol: g_module_check_init (fatal) 305: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/syslog-ng/libapp= model.so [0] 305: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libsyslog-ng-3.3= 8.so.0 [0] 305: symbol=3Dg_module_unload; lookup in file=3D/lib64/libm.so.6 [0] 305: symbol=3Dg_module_unload; lookup in file=3D/lib64/librt.so.1 [0] 305: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libevtlog-3.38.s= o.0 [0] 305: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libpcre.so.1 [0] 305: symbol=3Dg_module_unload; lookup in file=3D/lib64/libsystemd.so.0 [0] 305: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libsecret-storag= e.so.0 [0] 305: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libgmodule-2.0.s= o.0 [0] 305: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libgthread-2.0.s= o.0 [0] 305: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libglib-2.0.so.0= [0] 305: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libssl.so.3 [0] 305: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libcrypto.so.3 [= 0] 305: symbol=3Dg_module_unload; lookup in file=3D/lib64/libc.so.6 [0] 305: symbol=3Dg_module_unload; lookup in file=3D/lib64/ld64.so.2 [0] 305: symbol=3Dg_module_unload; lookup in file=3D/lib64/libcap.so.2 [0] 305: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libzstd.so.1 [0] 305: symbol=3Dg_module_unload; lookup in file=3D/usr/lib64/libpcre2-8.so.0 = [0] 305: /usr/lib64/syslog-ng/libappmodel.so: error: symbol lookup error: undef= ined symbol: g_module_unload (fatal) 305: symbol=3Dmodule_info; lookup in file=3D/usr/lib64/syslog-ng/libappmode= l.so [0] 305: 305: calling fini: /usr/lib64/syslog-ng/libappmodel.so [0] 305: 305: 305: file=3D/usr/lib64/syslog-ng/libappmodel.so [0]; destroying link map 305: 305: 305: file=3D/usr/lib64/syslog-ng/libappmodel.so [0]; destroying link map 305: 305: calling fini: syslog-ng [0] 305: 305: 305: calling fini: /usr/lib64/libsyslog-ng-3.38.so.0 [0] 305: 305: 305: calling fini: /lib64/libm.so.6 [0] 305: 305: 305: calling fini: /lib64/librt.so.1 [0] 305: 305: 305: calling fini: /usr/lib64/libevtlog-3.38.so.0 [0] 305: 305: 305: calling fini: /usr/lib64/libpcre.so.1 [0] 305: 305: 305: calling fini: /lib64/libsystemd.so.0 [0] 305: 305: 305: calling fini: /usr/lib64/libsecret-storage.so.0 [0] 305: 305: 305: calling fini: /usr/lib64/libgmodule-2.0.so.0 [0] 305: 305: 305: calling fini: /usr/lib64/libgthread-2.0.so.0 [0] 305: 305: 305: calling fini: /usr/lib64/libglib-2.0.so.0 [0] 305: 305: 305: calling fini: /usr/lib64/libssl.so.3 [0] 305: 305: 305: calling fini: /usr/lib64/libcrypto.so.3 [0] 305: 305: 305: calling fini: /usr/lib64/libpcre2-8.so.0 [0] 305: 305: 305: calling fini: /lib64/libcap.so.2 [0] 305: 305: 305: calling fini: /usr/lib64/libzstd.so.1 [0] 305: The .fini contents: /usr/lib64/syslog-ng/librate-limit-filter.so: file format elf64-powerpc= le Disassembly of section .fini: 0000000000004a4c <.fini>: 4a4c: 02 00 4c 3c addis r2,r12,2 4a50: b4 34 42 38 addi r2,r2,13492 4a54: a6 02 08 7c mflr r0 4a58: 10 00 01 f8 std r0,16(r1) 4a5c: a1 ff 21 f8 stdu r1,-96(r1) 4a60: 60 00 21 38 addi r1,r1,96 4a64: 10 00 01 e8 ld r0,16(r1) 4a68: a6 03 08 7c mtlr r0 4a6c: 20 00 80 4e blr root@qemuppc64:~# Reproduce steps: step 1: install necessary packages Use ubuntu as an example: sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool For other hosts, refer to https://docs.yoctoproject.org/ref-manual/system-requirements.html#required-= packages-for-the-build-host step 2: clone repo, build image, start image via qemu mkdir workdir && cd workdir git clone git://git.yoctoproject.org/poky git clone git://git.openembedded.org/meta-openembedded . poky/oe-init-build-env build bitbake-layers add-layer ../meta-openembedded/meta-* cat >> conf/local.conf <