From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mscreen.etri.re.kr (mscreen.etri.re.kr [129.254.9.16]) by sourceware.org (Postfix) with ESMTPS id 65CFF385781D for ; Tue, 25 May 2021 12:17:19 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 65CFF385781D Received: from unknown (HELO send001-relay.gov-dooray.com) (211.180.235.152) by 129.254.9.16 with ESMTP; 25 May 2021 21:17:17 +0900 X-Original-SENDERIP: 211.180.235.152 X-Original-MAILFROM: ckim@etri.re.kr X-Original-RCPTTO: libc-help@sourceware.org Received: from [10.162.225.106] (HELO smtp001-imp.gov-dooray.com) ([10.162.225.106]) by send001-relay.gov-dooray.com with SMTP id e440690e60aceacc; Tue, 25 May 2021 21:17:16 +0900 Received: from [129.254.132.39] (HELO CHANKIMPC) ([129.254.132.39]) by smtp001-imp.gov-dooray.com with SMTP id a52f080f60aceacc; Tue, 25 May 2021 21:17:16 +0900 From: To: "'Florian Weimer'" , "'Chan Kim via Libc-help'" References: <025a01d75121$117aa560$346ff020$@etri.re.kr> <87y2c31d08.fsf@oldenburg.str.redhat.com> In-Reply-To: <87y2c31d08.fsf@oldenburg.str.redhat.com> Subject: RE: undefined reference to symbol 'dlsym@@GLIBC_2.2.5' during qemu build Date: Tue, 25 May 2021 21:17:15 +0900 Message-ID: <027f01d7515f$e65ee280$b31ca780$@etri.re.kr> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQG65mMxl0iRNBw2Iy35u9pAzvmVvwG9UJQHqx75UjA= Content-Language: ko X-Spam-Status: No, score=-0.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, SPF_HELO_NONE, SPF_PASS, TXREP, URIBL_BLACK autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-help@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-help mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2021 12:17:22 -0000 Hello Florian, Thank you for the reply. I examined if the -ldl option is really used. I added V=1 during the qemu make to see the detail. This is the compile command for axpu_if.c which calls dlsym function. cc -iquote /home/ckim/prj/abdsn/ab21sim/ab21tsim/QEMU/qemu-5.1.0/build/hw/misc -iquote hw/misc -iquote /home/ckim/prj/abdsn/ab21sim/ab21tsim/QEMU/qemu-5.1.0/tcg/i386 -isystem /home/ckim/prj/abdsn/ab21sim/ab21tsim/QEMU/qemu-5.1.0/linux-headers -isystem /home/ckim/prj/abdsn/ab21sim/ab21tsim/QEMU/qemu-5.1.0/build/linux-headers -iquote . -iquote /home/ckim/prj/abdsn/ab21sim/ab21tsim/QEMU/qemu-5.1.0 -iquote /home/ckim/prj/abdsn/ab21sim/ab21tsim/QEMU/qemu-5.1.0/accel/tcg -iquote /home/ckim/prj/abdsn/ab21sim/ab21tsim/QEMU/qemu-5.1.0/include -iquote /home/ckim/prj/abdsn/ab21sim/ab21tsim/QEMU/qemu-5.1.0/disas/libvixl -I/usr/include/pixman-1 -pthread -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -std=gnu99 -DBAREMETAL_USE_AXPUSIM -ldl -Wold-style-declaration -Wold-style-definition -Wtype-limits -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wempty-body -Wnested-externs -Wendif-labels -Wexpansion-to-defined -Wno-missing-include-dirs -Wno-shift-negative-value -Wno-psabi -fstack-protector-strong -I/usr/include/p11-kit-1 -DSTRUCT_IOVEC_DEFINED -I/usr/include/libpng16 -I/usr/include/spice-server -I/usr/include/spice-1 -I/home/ckim/prj/abdsn/ab21sim/ab21tsim/QEMU/qemu-5.1.0/capstone/include -I/home/ckim/prj/abdsn/ab21sim/ab21tsim/QEMU/qemu-5.1.0/tests -I/home/ckim/prj/abdsn/ab21sim/ab21tsim/QEMU/qemu-5.1.0/tests/qtest -MMD -MP -MT hw/misc/axpu_if.o -MF hw/misc/axpu_if.d -g -c -o hw/misc/axpu_if.o /home/ckim/prj/abdsn/ab21sim/ab21tsim/QEMU/qemu-5.1.0/hw/misc/axpu_if.c Here cc is used to make axpu_if.o. and This is the last link command to build the final executable qemu-system-aarch64. c++ -g -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -DBAREMETAL_USE_AXPUSIM -ldl -fstack-protector-strong -o qemu-system-aarch64 exec.o exec-vary.o tcg/tcg.o tcg/tcg-op.o tcg/tcg-op-vec.o tcg/tcg-op-gvec.o tcg/tcg-common.o tcg/optimize.o fpu/softfloat.o disas.o gdbstub-xml.o gdbstub.o migration/ram.o accel/qtest.o accel/stubs/hax-stub.o accel/stubs/hvf-stub.o accel/stubs/whpx-stub.o accel/stubs/kvm-stub.o accel/stubs/xen-stub.o accel/tcg/tcg-all.o accel/tcg/cputlb.o accel/tcg/tcg-runtime.o accel/tcg/tcg-runtime-gvec.o accel/tcg/cpu-exec.o accel/tcg/cpu-exec-common.o accel/tcg/translate-all.o accel/tcg/translator.o dump/dump.o hw/9pfs/virtio-9p-device.o hw/block/virtio-blk.o hw/block/vhost-user-blk.o hw/block/dataplane/virtio-blk.o hw/char/virtio-serial-bus.o hw/core/machine-qmp-cmds.o hw/cpu/arm11mpcore.o hw/cpu/realview_mpcore.o hw/cpu/a9mpcore.o hw/c.... (reduced) ... verbs -libumad -lslirp -lglib-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lepoxy -lgbm -lepoxy -lgbm -lepoxy -lgbm -lepoxy -lgbm -lepoxy -lgbm -lpulse -lbrlapi -lcacard -lglib-2.0 -lcacard -lglib-2.0 -lusbredirparser -lusbredirparser -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lX11 -lvte-2.91 -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lepoxy -lgbm -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lX11 -lvte-2.91 -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lepoxy -lgbm -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lX11 -lvte-2.91 -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lepoxy -lgbm -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lX11 -lvte-2.91 -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lepoxy -lgbm -Wl,-Bsymbolic-functions -lncursesw -ltinfo -L/usr/local/lib -lz -lxml2 -laio -liscsi -lcurl -lrbd -lrados -lssh -lpixman-1 -lutil -lnuma -lpng16 -lz -ljpeg -lsasl2 -lxenctrl -lxenstore -lxenguest -lxenforeignmemory -lxengnttab -lxenevtchn -lxendevicemodel -lxentoolcore -lrdmacm -libverbs -libumad -lfdt -lspice-server -lpmem -L/home/ckim/prj/abdsn/ab21sim/ab21tsim/QEMU/qemu-5.1.0/build/capstone -lcapstone -lm -lz -lgthread-2.0 -pthread -lglib-2.0 -lnettle -lgnutls -lrt I reduced the .o file list above. But I could see all the .o files are there with -ldl option. One thing I didn't know was that the final link command was using c++ not cc. The default library and include settings of my c++ are like this : ckim@ckim-ubuntu:~$ echo | c++ -v -x c -E - Using built-in specs. COLLECT_GCC=c++ OFFLOAD_TARGET_NAMES=nvptx-none:hsa OFFLOAD_TARGET_DEFAULT=1 Target: x86_64-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.3.0-17ubuntu1~20.04' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-9-HskZEa/gcc-9-9.3.0/debian/t mp-nvptx/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu Thread model: posix gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04) COLLECT_GCC_OPTIONS='-v' '-E' '-shared-libgcc' '-mtune=generic' '-march=x86-64' /usr/lib/gcc/x86_64-linux-gnu/9/cc1 -E -quiet -v -imultiarch x86_64-linux-gnu - -mtune=generic -march=x86-64 -fasynchronous-unwind-tables -fstack-protector-strong -Wformat -Wformat-security -fstack-clash-protection -fcf-protection ignoring nonexistent directory "/usr/local/include/x86_64-linux-gnu" ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/9/include-fixed" ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include" #include "..." search starts here: #include <...> search starts here: /usr/lib/gcc/x86_64-linux-gnu/9/include /usr/local/include /usr/include/x86_64-linux-gnu /usr/include End of search list. # 1 "" # 1 "" # 1 "" # 31 "" # 1 "/usr/include/stdc-predef.h" 1 3 4 # 32 "" 2 # 1 "" COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/9/:/usr/lib/gcc/x86_64-linux-gnu /9/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/9/:/usr/lib /gcc/x86_64-linux-gnu/ LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/9/:/usr/lib/gcc/x86_64-linux-gnu/ 9/../../../x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/9/../../../../lib /:/lib/x86_64-linux-gnu/:/lib/../lib/:/usr/lib/x86_64-linux-gnu/:/usr/lib/.. /lib/:/usr/lib/gcc/x86_64-linux-gnu/9/../../../:/lib/:/usr/lib/ COLLECT_GCC_OPTIONS='-v' '-E' '-shared-libgcc' '-mtune=generic' '-march=x86-64' I can see the LIBRARY_PATH includes /usr/lib/x86_64-linux-gnu which contains libdl.so file. ckim@ckim-ubuntu:~$ F libdl.so /usr (F: my find function) /usr/lib32/libdl.so /usr/alpha-linux-gnu/lib/libdl.so /usr/libx32/libdl.so /usr/lib/i386-linux-gnu/libdl.so /usr/lib/x86_64-linux-gnu/libdl.so Using "nm -D /usr/lib/x86_64-linux-gnu/libdl.so", I could see it contains correctly dlsym (not __libc_dlsym). (I mistakenly grepped dlsym in libc.so file yesterday and it gave me __libc_dlsym). So yes, it is not being linked to the /usr/lib/x86_64-linux-gnu/libdl.so. (Actually it should be linked during the run time and what I need is the include file with "extern dlsym(...)" declaration. Right? The axpu_if.c file has #include which is in /usr/include/x86_64-linux-gnu directory which is in one of the default include path of cc and c++. (I checked both) So include path is correct and -ldl is used for final link command. The dlsym is in libdl.so but I don't know why it's looking for a symbol 'dlsym@@GLIBC_2.2.5'. Can you give me any suggestions for check? (In the source code I kept them as dlopen, dlsym now, and I guess if this undefined reference error goes, the seg-fault will go too) Thank you! Chan Kim > -----Original Message----- > From: Florian Weimer > Sent: Tuesday, May 25, 2021 4:36 PM > To: Chan Kim via Libc-help > Cc: ckim@etri.re.kr > Subject: Re: undefined reference to symbol 'dlsym@@GLIBC_2.2.5' during > qemu build > > * Chan Kim via Libc-help: > > > LINK aarch64-softmmu/qemu-system-aarch64 > > > > /usr/bin/ld: ../hw/misc/axpu_if.o: undefined reference to symbol > > 'dlsym@@GLIBC_2.2.5' > > > > /usr/bin/ld: > > /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libdl.so: > > error adding symbols: DSO missing from command line > > This error message attempts to tell you to link with -ldl. > > Thanks, > Florian >