what is the dlopen criterion used to decide if library needs to be loaded?

what is the dlopen criterion used to decide if library needs to be loaded?

[Daniel Villeneuve]

Hi,

I am compiling/linking/loading libraries from a long-running program.
I tend to reuse the same build directory and names.

But it seems that dlopen skips loading a new library if I reuse the same name as a previously (still loaded) library.
I could check that with the RTLD_NOLOAD option.
This happens even if the library itself has changed.

By looking at the code in glibc, there seems to be a test using inode/xdev, which does not rely on the name.
However, inodes can be reused immediately after unlink, so it seems fragile to infer equality of contents just comparing inodes.

In the end, I've rebuilt the library using the same name (not being sure whether the inode would be the same or not), and before dlopen, I create a hard link with a new unique name on the library and use that as arg to dlopen (and then delete the hard link).

Is this a safe way to ensure a newly built library is really loaded?
Would a symlink work? It seems that in /proc/$pid/maps, symlinks are resolved to canonical file names, so a new symlink name might not be sufficient...

Regards,
--
Daniel Villeneuve

Re: what is the dlopen criterion used to decide if library needs to be loaded?

[Florian Weimer]

* Daniel Villeneuve via Libc-help:

> By looking at the code in glibc, there seems to be a test using
> inode/xdev, which does not rely on the name.

> However, inodes can be reused immediately after unlink, so it seems
> fragile to infer equality of contents just comparing inodes.

The files are stilled mapped while the objects are still loaded, so they
must exist unnamed in the the file system.  Their content is also
accessible via /proc/PID/map_files.  I believe the inode/device pair is
indeed unique under these circumstances.

> In the end, I've rebuilt the library using the same name (not being
> sure whether the inode would be the same or not), and before dlopen, I
> create a hard link with a new unique name on the library and use that
> as arg to dlopen (and then delete the hard link).
>
> Is this a safe way to ensure a newly built library is really loaded?

It depends on what the soname of the library is.  If you set it to a
fixed value, the new library may be opened, but not loaded eventually
because the soname is already known to the system.

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

Re: what is the dlopen criterion used to decide if library needs to be loaded?

[Daniel Villeneuve]

On 11/13/20 4:41 PM, Florian Weimer wrote:
> * Daniel Villeneuve via Libc-help:
>
>> By looking at the code in glibc, there seems to be a test using inode/xdev, which does not rely on the name.  However, inodes can be reused immediately after unlink, so it seems fragile to infer equality of contents just comparing inodes.
> The files are stilled mapped while the objects are still loaded, so they
> must exist unnamed in the the file system.  Their content is also
> accessible via /proc/PID/map_files.  I believe the inode/device pair is
> indeed unique under these circumstances.
Ok, thanks.
>> In the end, I've rebuilt the library using the same name (not being
>> sure whether the inode would be the same or not), and before dlopen, I
>> create a hard link with a new unique name on the library and use that
>> as arg to dlopen (and then delete the hard link).
>>
>> Is this a safe way to ensure a newly built library is really loaded?
> It depends on what the soname of the library is.  If you set it to a
> fixed value, the new library may be opened, but not loaded eventually
> because the soname is already known to the system.
This internal test about soname (dlopen skipping loading a library) is new to me.
So loading two different library files, with different names, could end up in skipping the second load because of same soname?
My tests show that even with the same soname, dlopen/dlsym use the new library (loaded with the unique name).

My understanding of ld -hSONAME is for registering at link-time in an executable which arg to use for an eventual dlopen.
Not sure about the connection with calling dlopen on a specific path...

Regards,
--
Daniel Villeneuve

Re: what is the dlopen criterion used to decide if library needs to be loaded?

[Daniel Villeneuve]

On 11/13/20 5:18 PM, Daniel Villeneuve wrote:
> On 11/13/20 4:41 PM, Florian Weimer wrote:
>>> In the end, I've rebuilt the library using the same name (not being
>>> sure whether the inode would be the same or not), and before dlopen, I
>>> create a hard link with a new unique name on the library and use that
>>> as arg to dlopen (and then delete the hard link).
>>>
>>> Is this a safe way to ensure a newly built library is really loaded?
>> It depends on what the soname of the library is.  If you set it to a
>> fixed value, the new library may be opened, but not loaded eventually
>> because the soname is already known to the system.
> This internal test about soname (dlopen skipping loading a library) is new to me.
> So loading two different library files, with different names, could end up in skipping the second load because of same soname?
> My tests show that even with the same soname, dlopen/dlsym use the new library (loaded with the unique name).
>
> My understanding of ld -hSONAME is for registering at link-time in an executable which arg to use for an eventual dlopen.
> Not sure about the connection with calling dlopen on a specific path...

I extended my search in glibc source from dlfcn to elf, and found in elf/dl-load.c (_dl_map_object) the part that compares the name passed to dlopen and previously registered sonames.

Based on that, I could trigger the problem you allude to above, by using a specially crafted soname for "ld -hSONAME" that ends up matching a unique name I will generate in the future: in this case, the library with this specific unique name is not loaded.

This explains my successful tests as well, since the unique names passed to dlopen are different from any soname used before, so the test in _dl_map_object necessarily fails.

Thanks for pointing me to the right path.

Regards,
--
Daniel Villeneuve

Re: what is the dlopen criterion used to decide if library needs to be loaded?

[Florian Weimer]

* Daniel Villeneuve:

> On 11/13/20 5:18 PM, Daniel Villeneuve wrote:
>> On 11/13/20 4:41 PM, Florian Weimer wrote:
>>>> In the end, I've rebuilt the library using the same name (not being
>>>> sure whether the inode would be the same or not), and before dlopen, I
>>>> create a hard link with a new unique name on the library and use that
>>>> as arg to dlopen (and then delete the hard link).
>>>>
>>>> Is this a safe way to ensure a newly built library is really loaded?
>>> It depends on what the soname of the library is.  If you set it to a
>>> fixed value, the new library may be opened, but not loaded eventually
>>> because the soname is already known to the system.
>>
>> This internal test about soname (dlopen skipping loading a library)
>> is new to me.  So loading two different library files, with different
>> names, could end up in skipping the second load because of same
>> soname?  My tests show that even with the same soname, dlopen/dlsym
>> use the new library (loaded with the unique name).
>>
>> My understanding of ld -hSONAME is for registering at link-time in an
>> executable which arg to use for an eventual dlopen.  Not sure about
>> the connection with calling dlopen on a specific path...
>
> I extended my search in glibc source from dlfcn to elf, and found in
> elf/dl-load.c (_dl_map_object) the part that compares the name passed
> to dlopen and previously registered sonames.
>
> Based on that, I could trigger the problem you allude to above, by
> using a specially crafted soname for "ld -hSONAME" that ends up
> matching a unique name I will generate in the future: in this case,
> the library with this specific unique name is not loaded.
>
> This explains my successful tests as well, since the unique names
> passed to dlopen are different from any soname used before, so the
> test in _dl_map_object necessarily fails.

There are some suggestions that we should not load an object if its
soname matches one already known to the system.

Unfortunately I can't find the patch reference right now.  If we make
this change in glibc, I believe your application would behave
differently.

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

Re: what is the dlopen criterion used to decide if library needs to be loaded?

[Daniel Villeneuve]

On 11/23/20 5:39 AM, Florian Weimer wrote:
> * Daniel Villeneuve:
>
>> On 11/13/20 5:18 PM, Daniel Villeneuve wrote:
>>> On 11/13/20 4:41 PM, Florian Weimer wrote:
>>>>> In the end, I've rebuilt the library using the same name (not being
>>>>> sure whether the inode would be the same or not), and before dlopen, I
>>>>> create a hard link with a new unique name on the library and use that
>>>>> as arg to dlopen (and then delete the hard link).
>>>>>
>>>>> Is this a safe way to ensure a newly built library is really loaded?
>>>> It depends on what the soname of the library is.  If you set it to a
>>>> fixed value, the new library may be opened, but not loaded eventually
>>>> because the soname is already known to the system.
>>> This internal test about soname (dlopen skipping loading a library)
>>> is new to me.  So loading two different library files, with different
>>> names, could end up in skipping the second load because of same
>>> soname?  My tests show that even with the same soname, dlopen/dlsym
>>> use the new library (loaded with the unique name).
>>>
>>> My understanding of ld -hSONAME is for registering at link-time in an
>>> executable which arg to use for an eventual dlopen.  Not sure about
>>> the connection with calling dlopen on a specific path...
>> I extended my search in glibc source from dlfcn to elf, and found in
>> elf/dl-load.c (_dl_map_object) the part that compares the name passed
>> to dlopen and previously registered sonames.
>>
>> Based on that, I could trigger the problem you allude to above, by
>> using a specially crafted soname for "ld -hSONAME" that ends up
>> matching a unique name I will generate in the future: in this case,
>> the library with this specific unique name is not loaded.
>>
>> This explains my successful tests as well, since the unique names
>> passed to dlopen are different from any soname used before, so the
>> test in _dl_map_object necessarily fails.
> There are some suggestions that we should not load an object if its
> soname matches one already known to the system.
>
> Unfortunately I can't find the patch reference right now.  If we make
> this change in glibc, I believe your application would behave
> differently.
>
Thanks for the warning.
I understand I was inferring correctness from implementation-defined behavior (vs published spec).

Is there a doc on the stable part of dynamic-loading rules regarding a filesystem library name, its  internal soname and already loaded shared libraries?
In my case, something about when the argument to dlopen could/will be reused internally without being loaded vs must be loaded anew.
I could not find this information except by reading the source code and asking on the list.

Regards,
--
Daniel Villeneuve

Re: what is the dlopen criterion used to decide if library needs to be loaded?

[Florian Weimer]

* Daniel Villeneuve:

> Is there a doc on the stable part of dynamic-loading rules regarding a
> filesystem library name, its  internal soname and already loaded
> shared libraries?

I don't think there is such documentation yet, sorry.

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill