Using dlmope/dlopem with RTLD_GLOBAL

Using dlmope/dlopem with RTLD_GLOBAL

[Nima Hamidi]

[-- Attachment #1: Type: text/plain, Size: 4552 bytes --]

Hello all,

I’m trying to dlmopen a shared library which in turn dlopen’s another shared library with RTLD_GLOBAL flag set. On some platforms, it works as I expect it, but on some other platforms it crashes. Here is the simplest example I could make to show that crash:

I create a docker container containing my example using:

////////////////////////////////////////////////////////////
docker run --rm -it ubuntu:devel bash
apt-get update
apt install g++-10
cd ~

cat >a.cpp <<EOF
// a.cpp
#include <iostream>
#include <dlfcn.h>

using namespace std;

int main () {
 void * handle = nullptr;
 bool mopen = true;

 if (mopen) {
 cout << "mopening libb.so" << endl;
 handle = dlmopen(LM_ID_NEWLM, "libb.so", RTLD_NOW | RTLD_LOCAL);
 } else {
 cout << "opening libb.so" << endl;
 handle = dlopen("libb.so", RTLD_NOW | RTLD_LOCAL);
 }

 void (*bfunc)();
 *(void**)(&bfunc) = dlsym(handle, "bfunc");
 bfunc();

 return 0;
}
EOF


cat >b.cpp <<EOF
// b.cpp
#include <iostream>
#include <dlfcn.h>

using namespace std;

extern "C" void bfunc() {
 cout << "bfunc in b.cpp is being called..." << endl;
 void * handle = dlopen("libc.so", RTLD_NOW | RTLD_GLOBAL);
 //void * handle = dlopen("libc.so", RTLD_NOW | RTLD_LOCAL);

 void (*cfunc)();
 *(void**)(&cfunc) = dlsym(handle, "cfunc");
 cfunc();
}
EOF

cat >c.cpp <<EOF
// c.cpp
#include <iostream>

using namespace std;

extern "C" void cfunc() {
 cout << "cfunc in c.cpp is being called..." << endl;
}
EOF

g++-10 a.cpp -o a -ldl -Wl,-rpath `pwd`
g++-10 b.cpp -o libb.so -shared -fPIC -ldl -Wl,-rpath `pwd`
g++-10 c.cpp -o libc.so -shared -fPIC

////////////////////////////////////////////////////////////


Finally, when I run LD_DEBUG=1 ./a, it crashes with the following log:

 902:	symbol=_ZNSt8ios_base4InitD1Ev; lookup in file=/root/libb.so [1]
 902:	symbol=_ZNSt8ios_base4InitD1Ev; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=__cxa_finalize; lookup in file=/root/libb.so [1]
 902:	symbol=__cxa_finalize; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=__cxa_finalize; lookup in file=/lib/aarch64-linux-gnu/libc.so.6 [1]
 902:	symbol=_ZStlsISt11char_traitsIcEERSt13basic_ostreamIcT_ES5_PKc; lookup in file=/root/libb.so [1]
 902:	symbol=_ZStlsISt11char_traitsIcEERSt13basic_ostreamIcT_ES5_PKc; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=_ZNSolsEPFRSoS_E; lookup in file=/root/libb.so [1]
 902:	symbol=_ZNSolsEPFRSoS_E; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=__cxa_atexit; lookup in file=/root/libb.so [1]
 902:	symbol=__cxa_atexit; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=__cxa_atexit; lookup in file=/lib/aarch64-linux-gnu/libc.so.6 [1]
 902:	symbol=_ZNSt8ios_base4InitC1Ev; lookup in file=/root/libb.so [1]
 902:	symbol=_ZNSt8ios_base4InitC1Ev; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=__gmon_start__; lookup in file=/root/libb.so [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libc.so.6 [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/ld-linux-aarch64.so.1 [0]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libm.so.6 [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libgcc_s.so.1 [1]
 902:	symbol=__gmon_start__; lookup in file=/root/libc.so [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libc.so.6 [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libm.so.6 [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/ld-linux-aarch64.so.1 [0]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libgcc_s.so.1 [1]
Segmentation fault

The libc version that I’m using is 2.36 as outputted by

$ ldd --version
ldd (Ubuntu GLIBC 2.36-0ubuntu4) 2.36
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper

The error won’t happen if I dlopen with RTLD_LOCAL in b.cpp rather than RTLD_GLOBAL. I would appreciate any help on what I’m doing wrong here.

Best,
Nima

Re: Using dlmope/dlopem with RTLD_GLOBAL

[Florian Weimer]

* Nima Hamidi via Libc-help:

> The error won’t happen if I dlopen with RTLD_LOCAL in b.cpp rather
> than RTLD_GLOBAL. I would appreciate any help on what I’m doing wrong
> here.

I think I see what's going on.  The add_to_global_resize function does
not take into account that the global scope does not exist yet when it
is called.  Would you please file a bug in Bugzilla?

  <https://sourceware.org/bugzilla/enter_bug.cgi?product=glibc&component=dynamic-link>

Thanks,
Florian

Re: Using dlmope/dlopem with RTLD_GLOBAL

[Nima Hamidi]

[-- Attachment #1: Type: text/plain, Size: 692 bytes --]

Thanks for your quick response. Will do that once I can create an account on Bugzilla.
On Feb 13, 2023 at 4:30 AM -0600, Florian Weimer <fweimer@redhat.com>, wrote:
> * Nima Hamidi via Libc-help:
>
> > The error won’t happen if I dlopen with RTLD_LOCAL in b.cpp rather
> > than RTLD_GLOBAL. I would appreciate any help on what I’m doing wrong
> > here.
>
> I think I see what's going on. The add_to_global_resize function does
> not take into account that the global scope does not exist yet when it
> is called. Would you please file a bug in Bugzilla?
>
> <https://sourceware.org/bugzilla/enter_bug.cgi?product=glibc&component=dynamic-link>
>
> Thanks,
> Florian
>

Re: Using dlmope/dlopem with RTLD_GLOBAL

[Nima Hamidi]

[-- Attachment #1: Type: text/plain, Size: 884 bytes --]

FYI, filed the issue: https://sourceware.org/bugzilla/show_bug.cgi?id=30118
On Feb 13, 2023 at 4:40 AM -0600, Nima Hamidi <nimaa.hamidi@gmail.com>, wrote:
> Thanks for your quick response. Will do that once I can create an account on Bugzilla.
> On Feb 13, 2023 at 4:30 AM -0600, Florian Weimer <fweimer@redhat.com>, wrote:
> > * Nima Hamidi via Libc-help:
> >
> > > The error won’t happen if I dlopen with RTLD_LOCAL in b.cpp rather
> > > than RTLD_GLOBAL. I would appreciate any help on what I’m doing wrong
> > > here.
> >
> > I think I see what's going on. The add_to_global_resize function does
> > not take into account that the global scope does not exist yet when it
> > is called. Would you please file a bug in Bugzilla?
> >
> > <https://sourceware.org/bugzilla/enter_bug.cgi?product=glibc&component=dynamic-link>
> >
> > Thanks,
> > Florian
> >