Re: RTLD_DEEPBIND interaction with LD_PRELOAD

[Jonathon Anderson <janderson@rice.edu>]

Hello all,

Wrapping symbols is of interest to us (the HPCToolkit folks), so I thought I would jump in here and bring up a 3rd option that we are excited about: LD_AUDIT-powered symbol injection.

We currently use LD_PRELOAD + dlsym(RTLD_NEXT) to wrap critical symbols, however we have also encountered unavoidable limitations with the approach for some applications in the wild. We haven't run into RTLD_DEEPBIND before, but we have found many other issues:
 - dlsym(): If the symbol is fetched directly with dlsym() LD_PRELOAD does not apply. (And yes, there is code out there that does `dlsym(dlopen(libc.so.6))`. :/)
 - dlopen(RTLD_LOCAL): dlsym(RTLD_NEXT) fails if the "victim" symbol is only loaded as the dependency of a library loaded with dlopen(RTLD_LOCAL).
 - dlmopen() namespaces: LD_PRELOAD only applies to the main namespace, symbols in private dlmopen() namespaces are unaffected.

The alternative we are considering uses LD_AUDIT's la_symbind hook to inject our wrappers. This hook fires *every* time a symbol gets bound or on dlsym(), avoiding the narrow application issues with LD_PRELOAD. The hook also receives the to-be-bound target as an argument, avoiding the issues with dlsym(RTLD_NEXT). The high power of this approach makes it a very appealing alternative to LD_PRELOAD.

We have not yet tried LD_AUDIT-powered symbol injection in the wild, but I did write [a small test matrix with some possible wrapper implementations][1] for preliminary research. So far, the basic LD_AUDIT-powered implementations are very promising and avoid the issues we see with LD_PRELOAD.

[1]: https://gitlab.com/blue42u/ldaudit-power-tests/-/tree/main/symbol-wrapping

-Jonathon