[Bug dynamic-link/27977] New: Unexpected behavior when using DT_FILTER

[Bug dynamic-link/27977] New: Unexpected behavior when using DT_FILTER

[tdiff at yandex dot ru]

https://sourceware.org/bugzilla/show_bug.cgi?id=27977

            Bug ID: 27977
           Summary: Unexpected behavior when using DT_FILTER
           Product: glibc
           Version: 2.34
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: dynamic-link
          Assignee: unassigned at sourceware dot org
          Reporter: tdiff at yandex dot ru
  Target Milestone: ---

Created attachment 13489
  --> https://sourceware.org/bugzilla/attachment.cgi?id=13489&action=edit
Problem repro

Hi,

I am trying to use DT_FILTER to hide conflicting symbol implementation from one
of two libraries linked into the main executable.

In the attached example I have following setup:
lib1.so: func1(), func2()
lib2.so: func1(), func3()
lib2f.so: func3() //< this is a filter for lib2.so
main(): calls func1(), func2(), func3().

lib2f.so and lib1.so are passed to ld in the following order:
  gcc main.c -L. -l2f -l1 -o $@

Expected:
main calls func1() from lib1.so

Actual:
main calls func1() from lib2.so

Reasoning:
>From man ld:
       -F name
       --filter=name
           ... 
           If you later link a program against this filter object, then,
           when you run the program, the dynamic linker will see the
           DT_FILTER field.  The dynamic linker will resolve symbols
           according to the symbol table of the filter object as usual,
           but it will actually link to the definitions found in the
           shared object name. Thus the filter object can be used to
           select a subset of the symbols provided by the object name.

This clearly refers to runtime linker and mentions that dynsym of filter.so
should be used for symbol lookup. However, it looks like runtime liker does not
respect filter object.

I observe same behavior on glibc 1.17 @ Suse and 1.34 @ Ubuntu so looks like it
is not a distro-specific issue.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27977] Unexpected behavior when using DT_FILTER

[tdiff at yandex dot ru]

https://sourceware.org/bugzilla/show_bug.cgi?id=27977

A.Dmitrovsky <tdiff at yandex dot ru> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tdiff at yandex dot ru

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27977] Unexpected behavior when using DT_FILTER

[alex.ameen.tx at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27977

Alex Ameen <alex.ameen.tx at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |alex.ameen.tx at gmail dot com

--- Comment #1 from Alex Ameen <alex.ameen.tx at gmail dot com> ---
Oddly enough I bumped into this exact issue yesterday with a nearly identical
example when I was trying to experiment with filters.

The behavior is the same on RHEL 7, and NixOS. I got the same behavior when
trying to use alternative linkers ( `ld.gold' and `ld.bfd' ).

I confirmed the issue by tracing `LD_DEBUG' outputs and found that the linker
essentially treats `DT_FILTER' entries as `DT_NEEDED' entries except that they
don't need to be explicitly linked during a build - at runtime they are barely
more useful than a symlink with their current implementation.

In any case I agree that either this is a mistake in the implementation, or
misleading documentation.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27977] Unexpected behavior when using DT_FILTER

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27977

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fweimer at redhat dot com

--- Comment #2 from Florian Weimer <fweimer at redhat dot com> ---
This came up when we fixed bug 16272: DT_FILTER does not actually filter
anything (it just reorders name resolution), so the observed glibc behavior is
more or less as expected.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27977] Unexpected behavior when using DT_FILTER

[tdiff at yandex dot ru]

https://sourceware.org/bugzilla/show_bug.cgi?id=27977

--- Comment #3 from A.Dmitrovsky <tdiff at yandex dot ru> ---
(In reply to Florian Weimer from comment #2)
> This came up when we fixed bug 16272: DT_FILTER does not actually filter
> anything (it just reorders name resolution), so the observed glibc behavior
> is more or less as expected.

Could you please elaborate on "it just reorders name resolution"? What is the
intended use case for the --filter option? 

Is there a bug in documentation for this option since currently it very
specifically mentions "selection of a subset of symbols"?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27977] Unexpected behavior when using DT_FILTER

[alex.ameen.tx at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27977

--- Comment #4 from Alex Ameen <alex.ameen.tx at gmail dot com> ---
Yeah my understanding based on the SunOS docs, which I only referenced because
GNU has basically no documentation on this feature, is the following :

Say you have a big library `libfoo.so' which you cannot modify or rebuild, for
the sake of argument let's say its a commercial 3rd party lib which is licensed
in a way that disallows the use of `objdump' to directly modify the symbol
table.

In our usage we want to link ONLY the symbol `foo' from `libfoo.so', but
nothing else because other global symbols in `libfoo.so' conflict with our own
implementations, listed later in the link. 

To ensure that only `foo' is linked we create a filter library `libfoomin.so'
with only `foo' in its symbol table, listing `libfoo.so' as a `DT_NEEDED'
entry.

The expected behavior is that regardless of what order `libfoomin.so' appears
in our link we will not have to worry about any other symbols except for `foo'
from being bound either during pre-linking or at runtime.

I admit that the use case is niche but this is precisely the situation I've
been trying to solve, making it is a useful feature if implemented as SunOS
described.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27977] Unexpected behavior when using DT_FILTER

[alex.ameen.tx at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27977

--- Comment #5 from Alex Ameen <alex.ameen.tx at gmail dot com> ---
Apologies I meant to say "DT_FILTER" entry in my response.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27977] Unexpected behavior when using DT_FILTER

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27977

--- Comment #6 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to A.Dmitrovsky from comment #3)
> (In reply to Florian Weimer from comment #2)
> > This came up when we fixed bug 16272: DT_FILTER does not actually filter
> > anything (it just reorders name resolution), so the observed glibc behavior
> > is more or less as expected.
> 
> Could you please elaborate on "it just reorders name resolution"? What is
> the intended use case for the --filter option? 

DT_NEEDED puts the referenced object after the current object in the search
scope. DT_FILTER puts it before it. I believe this is the only difference in
the current glibc implementation. There is simply no filtering. It has been
this way since basically forever, so if we want to change DT_FILTER now, we
would have to make it dependent on some additional flag.

But to be clear, I do not see us fixing this issue any time soon. Ideally, the
change in lookup behavior would also be reflect in GDB, so that it can find the
same symbol the process uses at run time. And I do not see a way to implement a
properly filtering DT_FILTER without significant changes to the search scope
data structures.

Unfortunately, dlmopen is quite buggy, too, but maybe it will fit your use case
once we implement RTLD_SHARED. My hunch is that this will be ready before a
filtering DT_FILTER. (I assume RTLD_LOCAL is too limited for your use case.)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27977] Unexpected behavior when using DT_FILTER

[tdiff at yandex dot ru]

https://sourceware.org/bugzilla/show_bug.cgi?id=27977

--- Comment #7 from A.Dmitrovsky <tdiff at yandex dot ru> ---
(In reply to Florian Weimer from comment #6)
> 
> DT_NEEDED puts the referenced object after the current object in the search
> scope. DT_FILTER puts it before it. I believe this is the only difference in
> the current glibc implementation. There is simply no filtering. It has been
> this way since basically forever, so if we want to change DT_FILTER now, we
> would have to make it dependent on some additional flag.
> 

Thank you for clarification.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27977] Unexpected behavior when using DT_FILTER

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27977

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sam at gentoo dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.