Re: [Libguestfs] alternatives for hooking dlopen() without LD_LIBRARY_PATH or LD_AUDIT?

[Eric Blake <eblake@redhat.com>]

On 2/21/20 6:19 AM, Florian Weimer wrote:
> * Eric Blake:
> 
>> So with that said, here's a question I just thought of:
>>
>> If your patch for glibc support for DT_AUDIT is incorporated, is it
>> possible to mark a shared library as its own audit library via
>> DT_AUDIT? That is, if nbdkit-vddk-plugin.so can provide entry points
>> for _both_ the nbdkit interface (which satisfies dlopen() from the
>> nbdkit binary) and la_version/la_objsearch() (which satisfy the
>> requirements for use from the audit code in ld.so), _and_ during the
>> compilation of nbdkit-vddk-plugin.so, we marked the library as its own
>> DT_AUDIT entry, would the mere act of dlopen("nbdkit-vddk-plugin.so")
>> from nbdkit be sufficient to trigger audit actions such as
>> la_objsearch() on all subsequent shared loads (whether by dlopen() or
>> DT_NEEDED) performed by nbdkit-vddk-plugin.so and its descendant
>> loaded libraries?
> 
> So you want to dlopen nbdkit-vddk-plugin.so and launch a new auditor
> even if the process so far hasn't used auditing?  And the main program
> (which links agains a library which eventually makes this dlopen call)
> would not know anything about the existence of this specific plugin and
> auditing?

Yes, you interpreted my question correctly.

> 
> This isn't currently supported.  It's not just that the glibc
> implementation cannot do it.  The audit API (as sketched in <link.h>) is
> not a good fit for late loading where you have never observed open
> events.  It pretty much assumes that auditors are loaded magically
> *before* program start, so that they can observe all open calls and set
> up their own data structures along the way.

The concern is not about nbdkit loading nbdkit-vddk-plugin.so, but 
nbdkit-vddk-plugin.so doing subsequent loads of libvixDiskLib.so and its 
bare dependencies on libstdc++.so and such that were incorrectly built 
without DT_RUNPATH, but where we can't rewrite libvixDiskLib.so because 
it is proprietary, so the best we can do is hook the loading environment 
(either by la_objsearch or by re-exec with LD_LIBRARY_PATH).

> 
> I think what confuses me is that keep talking about a single binary, but
> clearly there is this separate vddk DSO, and there is talk of plugins.
> So it seems to me that multiple files are involved already?

You are correct that there are multiple files involved:

The nbdkit project currently has 2 relevant files: 'nbdkit' and 
'nbdkit-vddk-plugin.so' (and various other plugins, but those are not 
relevant to the VDDK use case)

The VDDK project from VMware: multiple files: libvixDiskLib.so (primary 
interface), which dlopen()s libdiskLibPlugin.so, which in turn has 
DT_NEEDED on libstdc++.so and several other recompiled system libraries. 
'find vmware-vix-disklib-distrib/lib64/ -type f | wc' shows 23 libraries 
total, but the end user installs it as a single tarball from VMware.

We can't change what VDDK ships, but we want to avoid making the nbdkit 
portion change from 2 files into 3, as every additional file required 
beyond what VMware ships is that much more burden for a user to choose 
to use nbdkit for accessing their VMware disks.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org