[Bug dynamic-link/28218] ld.so: ifunc resolver calls a lazy PLT. When does it work?

["i at maskray dot me" <sourceware-bugzilla@sourceware.org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=28218

--- Comment #3 from Fangrui Song <i at maskray dot me> ---
(In reply to Florian Weimer from comment #2)
> (In reply to Alan Modra from comment #1)
> > > FreeBSD rtld resolves non-IRELATIVE relocations in all modules, then
> > > IRELATIVE relocations in all modules.
> > Yes, this is the correct way to do things, and the only general way to
> > support cross-module ifunc resolvers that themselves might need to be
> > relocated.  We have known this for a long time.  See for example
> > https://gcc.gnu.org/legacy-ml/gcc-patches/2009-07/msg01307.html
> 
> Cross-module IFUNC resolvers that have relocation dependencies need more
> deferral than just IRELATIVE relocations. Regular relocations which target
> an IFUNC symbol need to be deferred, and copy relocations as well. Even that
> does not completely address all cases.
> 
> My impression is that the glibc maintainers think that IFUNC resolvers must
> not use have relocation dependencies. This could mean that they can only be
> implemented in assembler on some targets.
> 
> Initial comments on a proposal were positive:
> https://sourceware.org/legacy-ml/libc-alpha/2017-01/msg00468.html But I
> think during later discussions, this approach was rejected.

I have read
https://libc-alpha.sourceware.narkive.com/EDssYfrx/ifunc-resolver-scheduling-bugs-21041-20019

I think using a dependency based topological sort is probably over-engineering.
A two-pass approach (used by FreeBSD, mentioned by Szabolcs Nagy) works well.
Cross-DSO calls are via JUMP_SLOT relocs and cross-DSO variable accesses are
via GLOB_DAT or absolute relocations. Such relocs have been properly set up in
the first pass of relocation resolving. Here is a somewhat complex case.

cat > ./a.c <<eof
  #include <stdio.h>

  int a_impl() { return 42; }
  void *a_resolver() {
    puts("a_resolver");
    return (void *)a_impl;
  }
  int a() __attribute__((ifunc("a_resolver")));

  // .rela.dyn.rel => R_X86_64_64 referencing STT_GNU_IFUNC in .rela.dyn
  int (*fptr_a)() = a;
  int b(); extern int (*fptr_b)();
  int c(); extern int (*fptr_c)();

  int main() {
    printf("%d\n", a());
    printf("b: %p %p\n", fptr_b, b);
    printf("c: %p %p\n", fptr_c, c);
  }
eof
cat > ./b.c <<eof
  #include <stdio.h>
  void c();
  int b_impl() { return 43; }
  void *b_resolver() {
    puts("b_resolver");
    c(); // c is defined in c.so
    return (void *)b_impl;
  }
  int b() __attribute__((ifunc("b_resolver")));
  int (*fptr_b)() = b;
eof
cat > ./c.c <<eof
  #include <stdio.h>
  int c_impl() { return 44; }
  void *c_resolver() {
    puts("c_resolver");
    return (void *)c_impl;
  }
  int c() __attribute__((ifunc("c_resolver")));
  int (*fptr_c)() = c;
eof
cat > ./Makefile <<'eof'
a: a.c b.so c.so
        cc -pie -fpie a.c ./b.so ./c.so -l dl -o $@

b.so: b.c
        cc -shared -fpic b.c -o $@

c.so: c.c
        cc -shared -fpic c.c -o $@
eof
```

Note that in `b.so`, the ifunc resolver has a PLT call to `c.so`.
In real world applications, `c` may be some performance critical functions like
`memset`.

`./a` obviously crashes with glibc. `./a` works on FreeBSD.
```text
c_resolver
b_resolver
c_resolver
a_resolver
b_resolver
c_resolver
42
b: 0x80106e670 0x80106e670
c: 0x801072630 0x801072630
```

Also works when `a` is built with `-no-pie -fno-pic` where there are copy
relocations.
We can let `b.so` depend on `c.so`, swap the link order of `./b.so` and
`./c.so`. Still works.

The approach also works with various examples where an ifunc resolver returns a
variable referencing an implementation.


> The main problem with the two-pass approach is that there is no place
> where it can cache symbol lookup results. If we assume that relocation
> performance is dominated by symbol lookups (which is reasonable IMHO),
> we are looking at a very substantial performance hit for the two-pass
> approach. [...]

We can handle just IRELATIVE and ignore R_X86_64_64 referencing STT_GNU_IFUNC.

* Check whether a module has IRELATIVE. If not, skip the second pass.
* If yes, handle IRELATIVE in the second pass.

-- 
You are receiving this mail because:
You are on the CC list for the bug.