puzzling ld behavior, failing to find symbols from .so after adding more .o file

puzzling ld behavior, failing to find symbols from .so after adding more .o file

[Michael Tokarev]

Hi!

I've got a puzzling issue here which I need help with,
at least some pointers as of where to dig.

The context is - a debian system, with its current gcc & binutils
(unstable). I'm trying to update samba in debian to current version.
And the build fails during linking of one particular .so file (so far).
And fails in somewhat interesting way.  The whole thing is large with
a complex build system.

I took out this link command line and work with it only. From a long
list of .o files which was there, I started with just one .o file,
which links fine. And am adding the other .o files one by one.  And
at one point it starts failing.

Basically, it boils down to this.

  gcc -shared -o foo.so a.o -lbar -lbaz  -- works
  gcc -shared -o foo.so a.o b.o -lbar -lbaz -- fails

but it reports symbols like bar_init which are used by a.o as missing.

It is more.  When I replace -lbar (which is a system library,
and it uses .so file by default) with static libbar.a, it works
again even with b.o added to the mix -- "works" means it is now
able to find bar_init just fine, but it now start listing symbols
used by a.o from -lbaz too - which, again, can be fixed by replacing
-lbaz with libbaz.a. And so on.

it *looks* like after adding b.o into the mix, the linker stops
seeing symbols in shared libraries, only can use static libs.

b.o is a regular object file, I see nothing wrong with it. Yet it
makes whole thing to fail.

The complete command line is this (I preprocess this with grep -v # to run):

--- cut ---
gcc
-Wl,--as-needed
-Wl,--version-script=source3/smbconf.vscript
-shared
-Wl,-h,libsmbconf.so.0
lib/util/util_pw.c.99.o
librpc/gen_ndr/ndr_server_id.c.40.o
lib/async_req/async_sock.c.1.o
#source3/lib/cluster_support.c.45.o
source3/lib/dbwrap/dbwrap_ctdb.c.45.o
#source3/lib/messages_ctdb.c.45.o
#source3/lib/messages_ctdb_ref.c.45.o
#source3/lib/ctdbd_conn.c.45.o
#source3/lib/smbconf/smbconf_init.c.77.o
#source3/lib/smbconf/smbconf_reg.c.77.o
source3/lib/system.c.44.o
source3/lib/sendfile.c.44.o
source3/lib/recvfile.c.44.o
source3/lib/time.c.44.o
source3/lib/util_sid.c.44.o
source3/lib/util_specialsids.c.44.o
source3/lib/util_file.c.44.o
#source3/lib/util.c.44.o
#source3/lib/util_path.c.44.o
#source3/lib/util_matching.c.44.o
source3/lib/util_procid.c.44.o
#source3/lib/util_sock.c.44.o
source3/lib/util_tsock.c.44.o
source3/lib/util_transfer_file.c.44.o
source3/lib/util_macstreams.c.44.o
#source3/lib/util_tdb.c.116.o
source3/param/util.c.1.o
#source3/lib/messages.c.52.o
#source3/lib/util_cluster.c.52.o
#source3/lib/id_cache.c.52.o
#source3/lib/serverid.c.52.o
#source3/lib/server_id_watch.c.52.o
#source3/lib/server_id_db_util.c.52.o
source3/lib/addrchange.c.52.o
#lib/util/debug_s3.c.52.o
#source3/lib/dumpcore.c.52.o
#source3/lib/interface.c.52.o
#source3/lib/username.c.52.o
source3/lib/smbrun.c.52.o
#source3/lib/wins_srv.c.52.o
#source3/lib/substitute.c.52.o
source3/lib/substitute_generic.c.52.o
source3/lib/ms_fnmatch.c.52.o
#source3/lib/tallocmsg.c.52.o
#source3/lib/dmallocmsg.c.52.o
#source3/lib/gencache.c.52.o
source3/lib/util_event.c.52.o
#source3/lib/global_contexts.c.52.o
source3/lib/ldap_escape.c.52.o
source3/lib/system_smbd.c.52.o
source3/lib/audit.c.52.o
#source3/lib/idmap_cache.c.52.o
#source3/lib/namemap_cache.c.52.o
source3/lib/util_ea.c.52.o
#source3/lib/background.c.52.o
source3/lib/version.c.109.o
#source3/lib/dbwrap/dbwrap_open.c.48.o
#source3/lib/dbwrap/dbwrap_watch.c.48.o
#source3/lib/g_lock.c.48.o
lib/smbconf/smbconf.c.1.o
lib/smbconf/smbconf_txt.c.1.o
lib/smbconf/smbconf_util.c.1.o
#source3/registry/reg_backend_smbconf.c.36.o
#source3/registry/reg_init_smbconf.c.36.o
source3/registry/reg_util_token.c.36.o
#source3/registry/reg_api_util.c.36.o
#source3/registry/reg_api.c.35.o
source3/registry/reg_dispatcher.c.35.o
#source3/registry/reg_cachehook.c.35.o
source3/registry/reg_objects.c.35.o
source3/registry/reg_util_internal.c.35.o
source3/lib/util_nttoken.c.35.o
#source3/registry/reg_backend_db.c.35.o
#source3/registry/reg_parse_internal.c.35.o
#source3/registry/reg_parse.c.35.o
source3/lib/srprs.c.35.o
#source3/registry/reg_init_basic.c.35.o
#source3/param/loadparm_ctx.c.2.o
librpc/gen_ndr/ndr_messaging.c.154.o
source3/libsmb/errormap.c.126.o
source3/libsmb/smberr.c.126.o
source3/lib/errmap_unix.c.126.o
#source3/param/loadparm.c.76.o
#source3/lib/sharesec.c.76.o
#source3/lib/ldap_debug_handler.c.76.o
#source3/lib/util_names.c.76.o
-osource3/libsmbconf.so
-Lthird_party/heimdal_build
-Lsource4/lib/events
-Llib/tdb_wrap
-Llib
-Llibrpc
-Llib/replace
-Llib/socket
-Llibcli/util
-Llib/param
-Llib/messaging
-Llibcli/registry
-Llib/dbwrap
-Llib/util
-Lsource3
-Llibcli/security
-L/usr/local/lib
-L/usr/local/lib
-lsamba-security-samba4
-lmessages-util-samba4
-lserver-id-db-samba4
-lsamba-debug-samba4
-lsamba-util
-ldbwrap-samba4
-ltevent-util
-lsamba3-util-samba4
-lutil-tdb-samba4
-lutil-reg-samba4
-lsys-rw-samba4
-ltalloc-report-printf-samba4
-lCHARSET3-samba4
-lmessages-dgm-samba4
-lsamba-hostconfig
-lsamba-errors
-linterfaces-samba4
-lreplace-samba4
-lndr
-liov-buf-samba4
-lsamba-sockets-samba4
-lsmbd-shim-samba4
-lsocket-blocking-samba4
-ltdb-wrap-samba4
-ltime-basic-samba4
-lutil-setid-samba4
-lgenrand-samba4
-lmsghdr-samba4
-lserver-role-samba4
-lndr-nbt
-levents-samba4
-lroken-samba4
-lutil
-lresolv
-licudata
-licui18n
-licuuc
-lbsd
-lz
-ldl
-lgnutls
-ltdb
-lcups
-ltalloc
-lpthread
-llber
-lldap
-lcap
-ltevent
-ltalloc
-lsystemd
-Wl,-z,relro
-Wl,-z,now
-Wl,-z,relro,-z,now
-Wl,-no-undefined
-Wl,--export-dynamic
--- cut ---

There, the b.o which makes things fail is, for example,
source3/lib/dbwrap/dbwrap_ctdb.c.45.o (not only this one,
some other .o files trigger this same issue too).

Here's the output of the above command:

--- cut ---
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `async_connect_connected':
./bin/default/../../lib/async_req/async_sock.c:194: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:197: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_cleanup':
./bin/default/../../lib/async_req/async_sock.c:318: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:320: undefined reference to `_talloc_free'
/usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:321: undefined reference to `_talloc_free'
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `read_packet_cleanup':
./bin/default/../../lib/async_req/async_sock.c:509: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:511: undefined reference to `_talloc_free'
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `wait_for_read_cleanup':
./bin/default/../../lib/async_req/async_sock.c:641: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:643: undefined reference to `_talloc_free'
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `async_connect_cleanup':
./bin/default/../../lib/async_req/async_sock.c:167: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:169: undefined reference to `_talloc_free'
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_cancel':
./bin/default/../../lib/async_req/async_sock.c:326: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:335: undefined reference to `_talloc_free'
/usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:336: undefined reference to `_talloc_free'
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_trigger':
./bin/default/../../lib/async_req/async_sock.c:380: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `wait_for_read_done':
./bin/default/../../lib/async_req/async_sock.c:651: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:654: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_handler':
./bin/default/../../lib/async_req/async_sock.c:399: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:402: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: lib/async_req/async_sock.c.1.o:./bin/default/../../lib/async_req/async_sock.c:518: more undefined references to `_talloc_get_type_abort' 
follow
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `read_packet_handler':
./bin/default/../../lib/async_req/async_sock.c:522: undefined reference to `talloc_get_size'
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `accept_handler':
./bin/default/../../lib/async_req/async_sock.c:732: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:734: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:737: undefined reference to `_talloc_free'
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_send':
./bin/default/../../lib/async_req/async_sock.c:275: undefined reference to `_talloc_memdup'
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_recv':
./bin/default/../../lib/async_req/async_sock.c:442: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `read_packet_recv':
./bin/default/../../lib/async_req/async_sock.c:588: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `accept_recv':
./bin/default/../../lib/async_req/async_sock.c:767: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `read_packet_recv':
./bin/default/../../lib/async_req/async_sock.c:596: undefined reference to `talloc_get_size'
/usr/bin/ld: lib/util/util_pw.c.99.o: in function `tcopy_passwd':
./bin/default/../../lib/util/util_pw.c:50: undefined reference to `talloc_strdup'
/usr/bin/ld: ./bin/default/../../lib/util/util_pw.c:51: undefined reference to `talloc_strdup'
/usr/bin/ld: ./bin/default/../../lib/util/util_pw.c:54: undefined reference to `talloc_strdup'
/usr/bin/ld: ./bin/default/../../lib/util/util_pw.c:55: undefined reference to `talloc_strdup'
/usr/bin/ld: ./bin/default/../../lib/util/util_pw.c:56: undefined reference to `talloc_strdup'
/usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_transaction_cancel':
./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:876: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:896: undefined reference to `_talloc_free'
/usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_record_destructor':
./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:563: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:564: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_id':
./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1772: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `pull_newest_from_marshall_buffer_parser':
./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:463: undefined reference to `_talloc_memdup'
/usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_transaction_start':
./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:343: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:359: undefined reference to `_talloc_zero'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:367: undefined reference to `talloc_asprintf'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:378: undefined reference to `g_lock_lock'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:386: undefined reference to `_talloc_set_destructor'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:382: undefined reference to `_talloc_free'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:371: undefined reference to `_talloc_free'
/usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_transaction_destructor':
./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:325: undefined reference to `g_lock_unlock'
/usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_get_seqnum':
./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1765: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdbd_traverse':
./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1569: undefined reference to `lp_ctdb_timeout'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1569: undefined reference to `lp_ctdbd_socket'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1569: undefined reference to `ctdbd_init_connection'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1578: undefined reference to `ctdbd_traverse'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1579: undefined reference to `_talloc_free'
/usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_traverse_read':
./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1732: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1747: undefined reference to `tdb_traverse_read'
/usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_traverse':
./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1597: undefined reference to `_talloc_get_type_abort'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1612: undefined reference to `tdb_traverse'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1638: undefined reference to `tdb_exists'
/usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1639: undefined reference to `tdb_null'
/usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: relocation R_X86_64_PC32 against undefined hidden symbol `tdb_null' can not be used when making a 
shared object
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
--- cut ---

The symbols which are now reported as missing (in util_pw.o or async_sock.o) are
all talloc_* -- al the symbols from the system -ltalloc.  Once again, after
omitting dbwrap_ctdb.o, it all links fine, finding all talloc symbols without
an issue.

http://www.corpit.ru/mjt/tmp/dbwrap_ctdb.c.45.o is this .o file (x86_64).

(The tdb_null message is most likely due to it being unable to find this
symbol in -ltdb the same way as it can't find symbols in -ltalloc).

I'm quite lost here. The question is: why after ecountering dbwrap_ctdb.o,
ld stops finding symbols in shared libraries which it were finding before,
but still being able to find symbols in static libs?  How to debug this,
maybe to make a test case?

Is there a way to tell ld to emit some debugging info about its symbol
lookup/resolution happenings?

gcc (Debian 11.2.0-19) 11.2.0, from 4:11.2.0-2 debian package.
binutils 2.38, from 2.38-3 debian package.

Thank you!

/mjt

Re: puzzling ld behavior, failing to find symbols from .so after adding more .o file

[Andrew Bell]

On Fri, Mar 25, 2022 at 4:16 PM Michael Tokarev <mjt@tls.msk.ru> wrote:

> Hi!
>
> I've got a puzzling issue here which I need help with,
> at least some pointers as of where to dig.
>


> ...deleted...
>


> I'm quite lost here. The question is: why after ecountering dbwrap_ctdb.o,
> ld stops finding symbols in shared libraries which it were finding before,
> but still being able to find symbols in static libs?  How to debug this,
> maybe to make a test case?
>

There is certainly the LD_DEBUG environment variable.

--
Andrew Bell
andrew.bell.ia@gmail.com

Re: puzzling ld behavior, failing to find symbols from .so after adding more .o file

[Andrew Pinski]

On Fri, Mar 25, 2022 at 1:16 PM Michael Tokarev <mjt@tls.msk.ru> wrote:
>
> Hi!
>
> I've got a puzzling issue here which I need help with,
> at least some pointers as of where to dig.
>
> The context is - a debian system, with its current gcc & binutils
> (unstable). I'm trying to update samba in debian to current version.
> And the build fails during linking of one particular .so file (so far).
> And fails in somewhat interesting way.  The whole thing is large with
> a complex build system.
>
> I took out this link command line and work with it only. From a long
> list of .o files which was there, I started with just one .o file,
> which links fine. And am adding the other .o files one by one.  And
> at one point it starts failing.
>
> Basically, it boils down to this.
>
>   gcc -shared -o foo.so a.o -lbar -lbaz  -- works
>   gcc -shared -o foo.so a.o b.o -lbar -lbaz -- fails
>
> but it reports symbols like bar_init which are used by a.o as missing.
>
> It is more.  When I replace -lbar (which is a system library,
> and it uses .so file by default) with static libbar.a, it works
> again even with b.o added to the mix -- "works" means it is now
> able to find bar_init just fine, but it now start listing symbols
> used by a.o from -lbaz too - which, again, can be fixed by replacing
> -lbaz with libbaz.a. And so on.
>
> it *looks* like after adding b.o into the mix, the linker stops
> seeing symbols in shared libraries, only can use static libs.
>
> b.o is a regular object file, I see nothing wrong with it. Yet it
> makes whole thing to fail.
>
> The complete command line is this (I preprocess this with grep -v # to run):


One thing to try is to add -Wl,--no-as-needed (and remove
-Wl,--as-needed) and see if that helps.
My bet is there are some ordering issues.

Thanks,
Andrew

>
> --- cut ---
> gcc
> -Wl,--as-needed
> -Wl,--version-script=source3/smbconf.vscript
> -shared
> -Wl,-h,libsmbconf.so.0
> lib/util/util_pw.c.99.o
> librpc/gen_ndr/ndr_server_id.c.40.o
> lib/async_req/async_sock.c.1.o
> #source3/lib/cluster_support.c.45.o
> source3/lib/dbwrap/dbwrap_ctdb.c.45.o
> #source3/lib/messages_ctdb.c.45.o
> #source3/lib/messages_ctdb_ref.c.45.o
> #source3/lib/ctdbd_conn.c.45.o
> #source3/lib/smbconf/smbconf_init.c.77.o
> #source3/lib/smbconf/smbconf_reg.c.77.o
> source3/lib/system.c.44.o
> source3/lib/sendfile.c.44.o
> source3/lib/recvfile.c.44.o
> source3/lib/time.c.44.o
> source3/lib/util_sid.c.44.o
> source3/lib/util_specialsids.c.44.o
> source3/lib/util_file.c.44.o
> #source3/lib/util.c.44.o
> #source3/lib/util_path.c.44.o
> #source3/lib/util_matching.c.44.o
> source3/lib/util_procid.c.44.o
> #source3/lib/util_sock.c.44.o
> source3/lib/util_tsock.c.44.o
> source3/lib/util_transfer_file.c.44.o
> source3/lib/util_macstreams.c.44.o
> #source3/lib/util_tdb.c.116.o
> source3/param/util.c.1.o
> #source3/lib/messages.c.52.o
> #source3/lib/util_cluster.c.52.o
> #source3/lib/id_cache.c.52.o
> #source3/lib/serverid.c.52.o
> #source3/lib/server_id_watch.c.52.o
> #source3/lib/server_id_db_util.c.52.o
> source3/lib/addrchange.c.52.o
> #lib/util/debug_s3.c.52.o
> #source3/lib/dumpcore.c.52.o
> #source3/lib/interface.c.52.o
> #source3/lib/username.c.52.o
> source3/lib/smbrun.c.52.o
> #source3/lib/wins_srv.c.52.o
> #source3/lib/substitute.c.52.o
> source3/lib/substitute_generic.c.52.o
> source3/lib/ms_fnmatch.c.52.o
> #source3/lib/tallocmsg.c.52.o
> #source3/lib/dmallocmsg.c.52.o
> #source3/lib/gencache.c.52.o
> source3/lib/util_event.c.52.o
> #source3/lib/global_contexts.c.52.o
> source3/lib/ldap_escape.c.52.o
> source3/lib/system_smbd.c.52.o
> source3/lib/audit.c.52.o
> #source3/lib/idmap_cache.c.52.o
> #source3/lib/namemap_cache.c.52.o
> source3/lib/util_ea.c.52.o
> #source3/lib/background.c.52.o
> source3/lib/version.c.109.o
> #source3/lib/dbwrap/dbwrap_open.c.48.o
> #source3/lib/dbwrap/dbwrap_watch.c.48.o
> #source3/lib/g_lock.c.48.o
> lib/smbconf/smbconf.c.1.o
> lib/smbconf/smbconf_txt.c.1.o
> lib/smbconf/smbconf_util.c.1.o
> #source3/registry/reg_backend_smbconf.c.36.o
> #source3/registry/reg_init_smbconf.c.36.o
> source3/registry/reg_util_token.c.36.o
> #source3/registry/reg_api_util.c.36.o
> #source3/registry/reg_api.c.35.o
> source3/registry/reg_dispatcher.c.35.o
> #source3/registry/reg_cachehook.c.35.o
> source3/registry/reg_objects.c.35.o
> source3/registry/reg_util_internal.c.35.o
> source3/lib/util_nttoken.c.35.o
> #source3/registry/reg_backend_db.c.35.o
> #source3/registry/reg_parse_internal.c.35.o
> #source3/registry/reg_parse.c.35.o
> source3/lib/srprs.c.35.o
> #source3/registry/reg_init_basic.c.35.o
> #source3/param/loadparm_ctx.c.2.o
> librpc/gen_ndr/ndr_messaging.c.154.o
> source3/libsmb/errormap.c.126.o
> source3/libsmb/smberr.c.126.o
> source3/lib/errmap_unix.c.126.o
> #source3/param/loadparm.c.76.o
> #source3/lib/sharesec.c.76.o
> #source3/lib/ldap_debug_handler.c.76.o
> #source3/lib/util_names.c.76.o
> -osource3/libsmbconf.so
> -Lthird_party/heimdal_build
> -Lsource4/lib/events
> -Llib/tdb_wrap
> -Llib
> -Llibrpc
> -Llib/replace
> -Llib/socket
> -Llibcli/util
> -Llib/param
> -Llib/messaging
> -Llibcli/registry
> -Llib/dbwrap
> -Llib/util
> -Lsource3
> -Llibcli/security
> -L/usr/local/lib
> -L/usr/local/lib
> -lsamba-security-samba4
> -lmessages-util-samba4
> -lserver-id-db-samba4
> -lsamba-debug-samba4
> -lsamba-util
> -ldbwrap-samba4
> -ltevent-util
> -lsamba3-util-samba4
> -lutil-tdb-samba4
> -lutil-reg-samba4
> -lsys-rw-samba4
> -ltalloc-report-printf-samba4
> -lCHARSET3-samba4
> -lmessages-dgm-samba4
> -lsamba-hostconfig
> -lsamba-errors
> -linterfaces-samba4
> -lreplace-samba4
> -lndr
> -liov-buf-samba4
> -lsamba-sockets-samba4
> -lsmbd-shim-samba4
> -lsocket-blocking-samba4
> -ltdb-wrap-samba4
> -ltime-basic-samba4
> -lutil-setid-samba4
> -lgenrand-samba4
> -lmsghdr-samba4
> -lserver-role-samba4
> -lndr-nbt
> -levents-samba4
> -lroken-samba4
> -lutil
> -lresolv
> -licudata
> -licui18n
> -licuuc
> -lbsd
> -lz
> -ldl
> -lgnutls
> -ltdb
> -lcups
> -ltalloc
> -lpthread
> -llber
> -lldap
> -lcap
> -ltevent
> -ltalloc
> -lsystemd
> -Wl,-z,relro
> -Wl,-z,now
> -Wl,-z,relro,-z,now
> -Wl,-no-undefined
> -Wl,--export-dynamic
> --- cut ---
>
> There, the b.o which makes things fail is, for example,
> source3/lib/dbwrap/dbwrap_ctdb.c.45.o (not only this one,
> some other .o files trigger this same issue too).
>
> Here's the output of the above command:
>
> --- cut ---
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `async_connect_connected':
> ./bin/default/../../lib/async_req/async_sock.c:194: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:197: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_cleanup':
> ./bin/default/../../lib/async_req/async_sock.c:318: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:320: undefined reference to `_talloc_free'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:321: undefined reference to `_talloc_free'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `read_packet_cleanup':
> ./bin/default/../../lib/async_req/async_sock.c:509: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:511: undefined reference to `_talloc_free'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `wait_for_read_cleanup':
> ./bin/default/../../lib/async_req/async_sock.c:641: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:643: undefined reference to `_talloc_free'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `async_connect_cleanup':
> ./bin/default/../../lib/async_req/async_sock.c:167: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:169: undefined reference to `_talloc_free'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_cancel':
> ./bin/default/../../lib/async_req/async_sock.c:326: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:335: undefined reference to `_talloc_free'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:336: undefined reference to `_talloc_free'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_trigger':
> ./bin/default/../../lib/async_req/async_sock.c:380: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `wait_for_read_done':
> ./bin/default/../../lib/async_req/async_sock.c:651: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:654: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_handler':
> ./bin/default/../../lib/async_req/async_sock.c:399: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:402: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o:./bin/default/../../lib/async_req/async_sock.c:518: more undefined references to `_talloc_get_type_abort'
> follow
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `read_packet_handler':
> ./bin/default/../../lib/async_req/async_sock.c:522: undefined reference to `talloc_get_size'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `accept_handler':
> ./bin/default/../../lib/async_req/async_sock.c:732: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:734: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../lib/async_req/async_sock.c:737: undefined reference to `_talloc_free'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_send':
> ./bin/default/../../lib/async_req/async_sock.c:275: undefined reference to `_talloc_memdup'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `writev_recv':
> ./bin/default/../../lib/async_req/async_sock.c:442: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `read_packet_recv':
> ./bin/default/../../lib/async_req/async_sock.c:588: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `accept_recv':
> ./bin/default/../../lib/async_req/async_sock.c:767: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: lib/async_req/async_sock.c.1.o: in function `read_packet_recv':
> ./bin/default/../../lib/async_req/async_sock.c:596: undefined reference to `talloc_get_size'
> /usr/bin/ld: lib/util/util_pw.c.99.o: in function `tcopy_passwd':
> ./bin/default/../../lib/util/util_pw.c:50: undefined reference to `talloc_strdup'
> /usr/bin/ld: ./bin/default/../../lib/util/util_pw.c:51: undefined reference to `talloc_strdup'
> /usr/bin/ld: ./bin/default/../../lib/util/util_pw.c:54: undefined reference to `talloc_strdup'
> /usr/bin/ld: ./bin/default/../../lib/util/util_pw.c:55: undefined reference to `talloc_strdup'
> /usr/bin/ld: ./bin/default/../../lib/util/util_pw.c:56: undefined reference to `talloc_strdup'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_transaction_cancel':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:876: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:896: undefined reference to `_talloc_free'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_record_destructor':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:563: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:564: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_id':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1772: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `pull_newest_from_marshall_buffer_parser':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:463: undefined reference to `_talloc_memdup'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_transaction_start':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:343: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:359: undefined reference to `_talloc_zero'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:367: undefined reference to `talloc_asprintf'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:378: undefined reference to `g_lock_lock'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:386: undefined reference to `_talloc_set_destructor'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:382: undefined reference to `_talloc_free'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:371: undefined reference to `_talloc_free'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_transaction_destructor':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:325: undefined reference to `g_lock_unlock'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_get_seqnum':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1765: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdbd_traverse':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1569: undefined reference to `lp_ctdb_timeout'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1569: undefined reference to `lp_ctdbd_socket'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1569: undefined reference to `ctdbd_init_connection'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1578: undefined reference to `ctdbd_traverse'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1579: undefined reference to `_talloc_free'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_traverse_read':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1732: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1747: undefined reference to `tdb_traverse_read'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: in function `db_ctdb_traverse':
> ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1597: undefined reference to `_talloc_get_type_abort'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1612: undefined reference to `tdb_traverse'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1638: undefined reference to `tdb_exists'
> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1639: undefined reference to `tdb_null'
> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: relocation R_X86_64_PC32 against undefined hidden symbol `tdb_null' can not be used when making a
> shared object
> /usr/bin/ld: final link failed: bad value
> collect2: error: ld returned 1 exit status
> --- cut ---
>
> The symbols which are now reported as missing (in util_pw.o or async_sock.o) are
> all talloc_* -- al the symbols from the system -ltalloc.  Once again, after
> omitting dbwrap_ctdb.o, it all links fine, finding all talloc symbols without
> an issue.
>
> http://www.corpit.ru/mjt/tmp/dbwrap_ctdb.c.45.o is this .o file (x86_64).
>
> (The tdb_null message is most likely due to it being unable to find this
> symbol in -ltdb the same way as it can't find symbols in -ltalloc).
>
> I'm quite lost here. The question is: why after ecountering dbwrap_ctdb.o,
> ld stops finding symbols in shared libraries which it were finding before,
> but still being able to find symbols in static libs?  How to debug this,
> maybe to make a test case?
>
> Is there a way to tell ld to emit some debugging info about its symbol
> lookup/resolution happenings?
>
> gcc (Debian 11.2.0-19) 11.2.0, from 4:11.2.0-2 debian package.
> binutils 2.38, from 2.38-3 debian package.
>
> Thank you!
>
> /mjt

Re: puzzling ld behavior, failing to find symbols from .so after adding more .o file

[Michael Tokarev]

26.03.2022 01:43, Andrew Bell wrote:
> On Fri, Mar 25, 2022 at 4:16 PM Michael Tokarev <mjt@tls.msk.ru <mailto:mjt@tls.msk.ru>> wrote:
> 
>     Hi!
> 
>     I've got a puzzling issue here which I need help with,
>     at least some pointers as of where to dig.
> 
>     ...deleted...
> 
>     I'm quite lost here. The question is: why after ecountering dbwrap_ctdb.o,
>     ld stops finding symbols in shared libraries which it were finding before,
>     but still being able to find symbols in static libs?  How to debug this,
>     maybe to make a test case?
> 
> There is certainly the LD_DEBUG environment variable.

Hm. LD_DEBUG seems to be something about ld.so (the runtime linker), not ld
(the build-time linker).  If I set LD_DEBUG when invoking ld, it will print
how *ld* symbols are resolved, not how ld resolves symbols in the executable
being build. Or am I wrong?

Thank you!

/mjt

Re: puzzling ld behavior, failing to find symbols from .so after adding more .o file

[Michael Tokarev]

26.03.2022 03:25, Andrew Pinski wrote:
> 
> One thing to try is to add -Wl,--no-as-needed (and remove
> -Wl,--as-needed) and see if that helps.

adding --no-as-needed to the beginning and removing --as-needed
at the end does not change anything.

> My bet is there are some ordering issues.

in my tests, I add independent object files to the link line
one by one. More, I keep all the -lfoo at their places, -
in order to replace libfooo.so with libfoo.a, I *remove*
libfoo.so file, so the linker finds only the .a variant.
All the other stuff is here, order stays the same.

..
>> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1639: undefined reference to `tdb_null'
>> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: relocation R_X86_64_PC32 against undefined hidden symbol `tdb_null' can not be used when making a
>> shared object

Someone on the IRC told me this:

<trofi> R_X86_64_PC32 is is a relocation required to be local to the module. Note it's not GOT-relative.
<trofi> you need to check how tdb_null is declared when it's compiled

I don't know what is a relocation required to be local or
what is GOT-relative. But I do know there are several
different possibilities for libtdb compilation.

The thing is: it is samba, and the libraries in question
are libraries internal to samba, which were built separately,
and samba is told to use system version of these libs. Maybe
the whole thing is because the build "mixes" system libs and
internal headers with different symbol flags/definitions?

So far, it *seems* the whole thing breaks once I add any .o
file which uses tdb.h/libtdb.  Not yet sure about this, since
I can't add any object file into the mix due to inter-object-file
deps (so I'm adding only the simplest self-contained ones),
but all self-containing tdb-using ones gives this surprize
result.

Where one can see which variant of the symbol an object file
wants compared to the variant which is provided by an .so file?
Which variants are possible?

Thank you!

/mjt

Re: puzzling ld behavior, failing to find symbols from .so after adding more .o file

[Michael Tokarev]

26.03.2022 18:06, Michael Tokarev wrote:
..
>>> /usr/bin/ld: ./bin/default/../../source3/lib/dbwrap/dbwrap_ctdb.c:1639: undefined reference to `tdb_null'
>>> /usr/bin/ld: source3/lib/dbwrap/dbwrap_ctdb.c.45.o: relocation R_X86_64_PC32 against undefined hidden symbol `tdb_null' can not be used when making a
>>> shared object
> 
> Someone on the IRC told me this:
> 
> <trofi> R_X86_64_PC32 is is a relocation required to be local to the module. Note it's not GOT-relative.
> <trofi> you need to check how tdb_null is declared when it's compiled
> 
> I don't know what is a relocation required to be local or
> what is GOT-relative. But I do know there are several
> different possibilities for libtdb compilation.

I think I got it sorted out.

in the new version the tdb.h file (which declares tdb_null
and other stuff) added _PUBLIC_ attribute before all symbols,
which is by default evaluates to attribute(visibility(default)).
But when compiling samba, _PUBLIC_ is defined to
attribute(visibility(hidden)) before including tdb.h.
And things goes the wrong way from here...

It is interesting still why ld behaves this way, but the
mystery is now solved.

Thanks,

/mjt

Re: puzzling ld behavior, failing to find symbols from .so after adding more .o file

[Andreas Schwab]

On Mär 25 2022, Michael Tokarev wrote:

> Is there a way to tell ld to emit some debugging info about its symbol
> lookup/resolution happenings?

Try generating a linker map file with -Wl,-Map=mapfile.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: puzzling ld behavior, failing to find symbols from .so after adding more .o file

[Alan Modra]

On Sat, Mar 26, 2022 at 06:19:21PM +0300, Michael Tokarev wrote:
> I think I got it sorted out.
> 
> in the new version the tdb.h file (which declares tdb_null
> and other stuff) added _PUBLIC_ attribute before all symbols,
> which is by default evaluates to attribute(visibility(default)).
> But when compiling samba, _PUBLIC_ is defined to
> attribute(visibility(hidden)) before including tdb.h.
> And things goes the wrong way from here...
> 
> It is interesting still why ld behaves this way, but the
> mystery is now solved.

A hidden visibility symbol will not be put in .dynsym.  So any newly
built shared library defining one of the affected symbols won't
provide a definition that can be used outside of that shared library.

-- 
Alan Modra
Australia Development Lab, IBM