Re: puzzling ld behavior, failing to find symbols from .so after adding more .o file

[Andrew Pinski <pinskia@gmail.com>]

On Fri, Mar 25, 2022 at 1:16 PM Michael Tokarev <mjt@tls.msk.ru> wrote:
>
> Hi!
>
> I've got a puzzling issue here which I need help with,
> at least some pointers as of where to dig.
>
> The context is - a debian system, with its current gcc & binutils
> (unstable). I'm trying to update samba in debian to current version.
> And the build fails during linking of one particular .so file (so far).
> And fails in somewhat interesting way.  The whole thing is large with
> a complex build system.
>
> I took out this link command line and work with it only. From a long
> list of .o files which was there, I started with just one .o file,
> which links fine. And am adding the other .o files one by one.  And
> at one point it starts failing.
>
> Basically, it boils down to this.
>
>   gcc -shared -o foo.so a.o -lbar -lbaz  -- works
>   gcc -shared -o foo.so a.o b.o -lbar -lbaz -- fails
>
> but it reports symbols like bar_init which are used by a.o as missing.
>
> It is more.  When I replace -lbar (which is a system library,
> and it uses .so file by default) with static libbar.a, it works
> again even with b.o added to the mix -- "works" means it is now
> able to find bar_init just fine, but it now start listing symbols
> used by a.o from -lbaz too - which, again, can be fixed by replacing
> -lbaz with libbaz.a. And so on.
>
> it *looks* like after adding b.o into the mix, the linker stops
> seeing symbols in shared libraries, only can use static libs.
>
> b.o is a regular object file, I see nothing wrong with it. Yet it
> makes whole thing to fail.
>
> The complete command line is this (I preprocess this with grep -v # to run):


One thing to try is to add -Wl,--no-as-needed (and remove
-Wl,--as-needed) and see if that helps.
My bet is there are some ordering issues.

Thanks,
Andrew

>
> --- cut ---
> gcc
> -Wl,--as-needed
> -Wl,--version-script=source3/smbconf.vscript
> -shared
> -Wl,-h,libsmbconf.so.0
> lib/util/util_pw.c.99.o
> librpc/gen_ndr/ndr_server_id.c.40.o
> lib/async_req/async_sock.c.1.o
> #source3/lib/cluster_support.c.45.o
> source3/lib/dbwrap/dbwrap_ctdb.c.45.o
> #source3/lib/messages_ctdb.c.45.o
> #source3/lib/messages_ctdb_ref.c.45.o
> #source3/lib/ctdbd_conn.c.45.o
> #source3/lib/smbconf/smbconf_init.c.77.o
> #source3/lib/smbconf/smbconf_reg.c.77.o
> source3/lib/system.c.44.o
> source3/lib/sendfile.c.44.o
> source3/lib/recvfile.c.44.o
> source3/lib/time.c.44.o
> source3/lib/util_sid.c.44.o
> source3/lib/util_specialsids.c.44.o
> source3/lib/util_file.c.44.o
> #source3/lib/util.c.44.o
> #source3/lib/util_path.c.44.o
> #source3/lib/util_matching.c.44.o
> source3/lib/util_procid.c.44.o
> #source3/lib/util_sock.c.44.o
> source3/lib/util_tsock.c.44.o
> source3/lib/util_transfer_file.c.44.o
> source3/lib/util_macstreams.c.44.o
> #source3/lib/util_tdb.c.116.o
> source3/param/util.c.1.o
> #source3/lib/messages.c.52.o
> #source3/lib/util_cluster.c.52.o
> #source3/lib/id_cache.c.52.o
> #source3/lib/serverid.c.52.o
> #source3/lib/server_id_watch.c.52.o
> #source3/lib/server_id_db_util.c.52.o
> source3/lib/addrchange.c.52.o
> #lib/util/debug_s3.c.52.o
> #source3/lib/dumpcore.c.52.o
> #source3/lib/interface.c.52.o
> #source3/lib/username.c.52.o
> source3/lib/smbrun.c.52.o
> #source3/lib/wins_srv.c.52.o
> #source3/lib/substitute.c.52.o
> source3/lib/substitute_generic.c.52.o
> source3/lib/ms_fnmatch.c.52.o
> #source3/lib/tallocmsg.c.52.o
> #source3/lib/dmallocmsg.c.52.o
> #source3/lib/gencache.c.52.o
> source3/lib/util_event.c.52.o
> #source3/lib/global_contexts.c.52.o
> source3/lib/ldap_escape.c.52.o
> source3/lib/system_smbd.c.52.o
> source3/lib/audit.c.52.o
> #source3/lib/idmap_cache.c.52.o
> #source3/lib/namemap_cache.c.52.o
> source3/lib/util_ea.c.52.o
> #source3/lib/background.c.52.o
> source3/lib/version.c.109.o
> #source3/lib/dbwrap/dbwrap_open.c.48.o
> #source3/lib/dbwrap/dbwrap_watch.c.48.o
> #source3/lib/g_lock.c.48.o
> lib/smbconf/smbconf.c.1.o
> lib/smbconf/smbconf_txt.c.1.o
> lib/smbconf/smbconf_util.c.1.o
> #source3/registry/reg_backend_smbconf.c.36.o
> #source3/registry/reg_init_smbconf.c.36.o
> source3/registry/reg_util_token.c.36.o
> #source3/registry/reg_api_util.c.36.o
> #source3/registry/reg_api.c.35.o
> source3/registry/reg_dispatcher.c.35.o
> #source3/registry/reg_cachehook.c.35.o
> source3/registry/reg_objects.c.35.o
> source3/registry/reg_util_internal.c.35.o
> source3/lib/util_nttoken.c.35.o
> #source3/registry/reg_backend_db.c.35.o
> #source3/registry/reg_parse_internal.c.35.o
> #source3/registry/reg_parse.c.35.o
> source3/lib/srprs.c.35.o
> #source3/registry/reg_init_basic.c.35.o
> #source3/param/loadparm_ctx.c.2.o
> librpc/gen_ndr/ndr_messaging.c.154.o
> source3/libsmb/errormap.c.126.o
> source3/libsmb/smberr.c.126.o
> source3/lib/errmap_unix.c.126.o
> #source3/param/loadparm.c.76.o
> #source3/lib/sharesec.c.76.o
> #source3/lib/ldap_debug_handler.c.76.o
> #source3/lib/util_names.c.76.o
> -osource3/libsmbconf.so
> -Lthird_party/heimdal_build
> -Lsource4/lib/events
> -Llib/tdb_wrap
> -Llib
> -Llibrpc
> -Llib/replace
> -Llib/socket
> -Llibcli/util
> -Llib/param
> -Llib/messaging
> -Llibcli/registry
> -Llib/dbwrap
> -Llib/util
> -Lsource3
> -Llibcli/security
> -L/usr/local/lib
> -L/usr/local/lib
> -lsamba-security-samba4
> -lmessages-util-samba4
> -lserver-id-db-samba4
> -lsamba-debug-samba4
> -lsamba-util
> -ldbwrap-samba4
> -ltevent-util
> -lsamba3-util-samba4
> -lutil-tdb-samba4
> -lutil-reg-samba4
> -lsys-rw-samba4
> -ltalloc-report-printf-samba4
> -lCHARSET3-samba4
> -lmessages-dgm-samba4
> -lsamba-hostconfig
> -lsamba-errors
> -linterfaces-samba4
> -lreplace-samba4
> -lndr
> -liov-buf-samba4
> -lsamba-sockets-samba4
> -lsmbd-shim-samba4
> -lsocket-blocking-samba4
> -ltdb-wrap-samba4
> -ltime-basic-samba4
> -lutil-setid-samba4
> -lgenrand-samba4
> -lmsghdr-samba4
> -lserver-role-samba4
> -lndr-nbt
> -levents-samba4
> -lroken-samba4
> -lutil
> -lresolv
> -licudata
> -licui18n
> -licuuc
> -lbsd
> -lz
> -ldl
> -lgnutls
> -ltdb
> -lcups
> -ltalloc
> -lpthread
> -llber
> -lldap
> -lcap
> -ltevent
> -ltalloc
> -lsystemd
> -Wl,-z,relro
> -Wl,-z,now
> -Wl,-z,relro,-z,now
> -Wl,-no-undefined
> -Wl,--export-dynamic
> --- cut ---
>
> There, the b.o which makes things fail is, for example,
> source3/lib/dbwrap/dbwrap_ctdb.c.45.o (not only this one,
> some other .o files trigger this same issue too).
>
> Here's the output of the above command:
>
> --- cut ---
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `async_connect_connected':
> ./bin/default/../../lib/async_req/async_sock.c:194: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:197: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_cleanup':
> ./bin/default/../../lib/async_req/async_sock.c:318: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:320: undefined reference to `_talloc_free'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:321: undefined reference to `_talloc_free'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `read_packet_cleanup':
> ./bin/default/../../lib/async_req/async_sock.c:509: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:511: undefined reference to `_talloc_free'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `wait_for_read_cleanup':
> ./bin/default/../../lib/async_req/async_sock.c:641: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:643: undefined reference to `_talloc_free'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `async_connect_cleanup':
> ./bin/default/../../lib/async_req/async_sock.c:167: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:169: undefined reference to `_talloc_free'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_cancel':
> ./bin/default/../../lib/async_req/async_sock.c:326: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:335: undefined reference to `_talloc_free'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:336: undefined reference to `_talloc_free'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_trigger':
> ./bin/default/../../lib/async_req/async_sock.c:380: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `wait_for_read_done':
> ./bin/default/../../lib/async_req/async_sock.c:651: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:654: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_handler':
> ./bin/default/../../lib/async_req/async_sock.c:399: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:402: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o:./bin/default/../../lib/async_req/async_sock.c:518: more undefined references to `_talloc_get_type_abort'
> follow
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `read_packet_handler':
> ./bin/default/../../lib/async_req/async_sock.c:522: undefined reference to `talloc_get_size'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `accept_handler':
> ./bin/default/../../lib/async_req/async_sock.c:732: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:734: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:737: undefined reference to `_talloc_free'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_send':
> ./bin/default/../../lib/async_req/async_sock.c:275: undefined reference to `_talloc_memdup'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_recv':
> ./bin/default/../../lib/async_req/async_sock.c:442: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `read_packet_recv':
> ./bin/default/../../lib/async_req/async_sock.c:588: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `accept_recv':
> ./bin/default/../../lib/async_req/async_sock.c:767: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `read_packet_recv':
> ./bin/default/../../lib/async_req/async_sock.c:596: undefined reference to `talloc_get_size'
> /usr/bin/ld: lib/util/util_pw.c.99.o: in function `tcopy_passwd':
> ./bin/default/../../lib/util/util_pw.c:50: undefined reference to `talloc_strdup'
> /usr/bin/ld: ./bin/default/../../lib/util/util_pw.c:51: undefined reference to `talloc_strdup'
> /usr/bin/ld: ./bin/default/../../lib/util/util_pw.c:54: undefined reference to `talloc_strdup'
> /usr/bin/ld: ./bin/default/../../lib/util/util_pw.c:55: undefined reference to `talloc_strdup'
> /usr/bin/ld: ./bin/default/../../lib/util/util_pw.c:56: undefined reference to `talloc_strdup'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_transaction_cancel':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:876: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:896: undefined reference to `_talloc_free'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_record_destructor':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:563: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:564: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_id':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1772: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `pull_newest_from_marshall_buffer_parser':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:463: undefined reference to `_talloc_memdup'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_transaction_start':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:343: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:359: undefined reference to `_talloc_zero'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:367: undefined reference to `talloc_asprintf'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:378: undefined reference to `g_lock_lock'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:386: undefined reference to `_talloc_set_destructor'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:382: undefined reference to `_talloc_free'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:371: undefined reference to `_talloc_free'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_transaction_destructor':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:325: undefined reference to `g_lock_unlock'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_get_seqnum':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1765: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdbd_traverse':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1569: undefined reference to `lp_ctdb_timeout'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1569: undefined reference to `lp_ctdbd_socket'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1569: undefined reference to `ctdbd_init_connection'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1578: undefined reference to `ctdbd_traverse'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1579: undefined reference to `_talloc_free'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_traverse_read':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1732: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1747: undefined reference to `tdb_traverse_read'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_traverse':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1597: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1612: undefined reference to `tdb_traverse'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1638: undefined reference to `tdb_exists'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1639: undefined reference to `tdb_null'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: relocation R_X86_64_PC32 against undefined hidden symbol `tdb_null' can not be used when making a
> shared object
> /usr/bin/ld: final link failed: bad value
> collect2: error: ld returned 1 exit status
> --- cut ---
>
> The symbols which are now reported as missing (in util_pw.o or async_sock.o) are
> all talloc_* -- al the symbols from the system -ltalloc.  Once again, after
> omitting dbwrap_ctdb.o, it all links fine, finding all talloc symbols without
> an issue.
>
> http://www.corpit.ru/mjt/tmp/dbwrap_ctdb.c.45.o is this .o file (x86_64).
>
> (The tdb_null message is most likely due to it being unable to find this
> symbol in -ltdb the same way as it can't find symbols in -ltalloc).
>
> I'm quite lost here. The question is: why after ecountering dbwrap_ctdb.o,
> ld stops finding symbols in shared libraries which it were finding before,
> but still being able to find symbols in static libs?  How to debug this,
> maybe to make a test case?
>
> Is there a way to tell ld to emit some debugging info about its symbol
> lookup/resolution happenings?
>
> gcc (Debian 11.2.0-19) 11.2.0, from 4:11.2.0-2 debian package.
> binutils 2.38, from 2.38-3 debian package.
>
> Thank you!
>
> /mjt