A public interface for library search

A public interface for library search

[Kozlukov Sergei]

Hi! I wasn't sure if this was worth a ticket on BugZilla yet, I hope libc-help will be an appropriate venue.

I'm trying to assess if there's such a thing as an agreeable interface for locating the host platform's shared libraries. Specifically, if there's a "conventional" way of predicting, given a soname, what specific file the `dlopen()` would have loaded, but without actually loading it.

This task is, I think, precisely what certain container-related projects try to accomplish by peeking into the /etc/ld.so.cache, with the purpose of locating the userspace drivers that need to be mounted into the container (think mesa or, most realistically, libcuda.so):

- https://github.com/NVIDIA/libnvidia-container/blob/1eb5a30a6ad0415550a9df632ac8832bf7e2bbba/src/nvc_ldcache.c#L370
- https://github.com/NVIDIA/nvidia-container-toolkit/blob/da0755769fee3ab071aaa3b9fac6fe085da7071d/internal/ldcache/ldcache.go#L36

These tools are then used by many downstream projects, e.g. apptainer/sylabs-singularity, docker, etc.

As far as I can tell, /etc/ld.so.cache is glibc internals and isn't meant to be used as a public interface. In particular, NixOS does not populate that file, and the nixpkgs simply patch all software that refers to it.

I think this behaviour has to change upstream, in those projects, and for that we need an interface that could be considered "conventional", "maintstream", and "supported".

The combination of `dlopen` and `RTLD_DI_ORIGIN` are almost perfect for the task: they respect LD_LIBRARY_PATH, LD_PRELOAD, and even the inquiring process's RUNPATH. Unfortunately, that comes with the side-effect of executing the loaded libraries' constructors. I understand that this shouldn't be a security concern, at least in the specific case of nvidia's container-toolkit and from glibc's perspective. However, there's no good reason to actually "load" the libraries being searched for, so I expect that this solution would be, generally, hard to sell.

Thanks!

Re: A public interface for library search

[Szabolcs Nagy]

The 11/14/2023 00:11, Kozlukov Sergei via Libc-help wrote:
> Hi! I wasn't sure if this was worth a ticket on BugZilla yet, I hope libc-help will be an appropriate venue.
> 
> I'm trying to assess if there's such a thing as an agreeable interface for locating the host platform's shared libraries. Specifically, if there's a "conventional" way of predicting, given a soname, what specific file the `dlopen()` would have loaded, but without actually loading it.

i think posix allows names without / to be treated specially
(e.g. ld.so could recognize 'libc.so.6' and not load anything
from the filesystem).

the path lookup can change at runtime (e.g. file system can
change) if you are fine with a static view then

ld.so --list exe

can give you the mappings. (won't cover dlopened modules
unless you create an exe that directly depends on those,
but that's not guaranteed to be possible given the runtime
dependent behaviour).

> This task is, I think, precisely what certain container-related projects try to accomplish by peeking into the /etc/ld.so.cache, with the purpose of locating the userspace drivers that need to be mounted into the container (think mesa or, most realistically, libcuda.so):
> 
> - https://github.com/NVIDIA/libnvidia-container/blob/1eb5a30a6ad0415550a9df632ac8832bf7e2bbba/src/nvc_ldcache.c#L370
> - https://github.com/NVIDIA/nvidia-container-toolkit/blob/da0755769fee3ab071aaa3b9fac6fe085da7071d/internal/ldcache/ldcache.go#L36
> 
> These tools are then used by many downstream projects, e.g. apptainer/sylabs-singularity, docker, etc.
> 
> As far as I can tell, /etc/ld.so.cache is glibc internals and isn't meant to be used as a public interface. In particular, NixOS does not populate that file, and the nixpkgs simply patch all software that refers to it.
> 
> I think this behaviour has to change upstream, in those projects, and for that we need an interface that could be considered "conventional", "maintstream", and "supported".
> 
> The combination of `dlopen` and `RTLD_DI_ORIGIN` are almost perfect for the task: they respect LD_LIBRARY_PATH, LD_PRELOAD, and even the inquiring process's RUNPATH. Unfortunately, that comes with the side-effect of executing the loaded libraries' constructors. I understand that this shouldn't be a security concern, at least in the specific case of nvidia's container-toolkit and from glibc's perspective. However, there's no good reason to actually "load" the libraries being searched for, so I expect that this solution would be, generally, hard to sell.

this assumes that the container somehow has the same setup as the
host outside, which is not necessarily the case (same glibc, same
config files, same env, same paths,..)

note that the libc can dlopen internal components you don't know
about so i don't see how this can ever be robust. (not to mention
portability to bionic, musl, etc systems)

so while an api can be designed for the path mapping it seems it
won't fully solve the problem.