Using dlmope/dlopem with RTLD_GLOBAL

[Nima Hamidi <nimaa.hamidi@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4552 bytes --]

Hello all,

I’m trying to dlmopen a shared library which in turn dlopen’s another shared library with RTLD_GLOBAL flag set. On some platforms, it works as I expect it, but on some other platforms it crashes. Here is the simplest example I could make to show that crash:

I create a docker container containing my example using:

////////////////////////////////////////////////////////////
docker run --rm -it ubuntu:devel bash
apt-get update
apt install g++-10
cd ~

cat >a.cpp <<EOF
// a.cpp
#include <iostream>
#include <dlfcn.h>

using namespace std;

int main () {
 void * handle = nullptr;
 bool mopen = true;

 if (mopen) {
 cout << "mopening libb.so" << endl;
 handle = dlmopen(LM_ID_NEWLM, "libb.so", RTLD_NOW | RTLD_LOCAL);
 } else {
 cout << "opening libb.so" << endl;
 handle = dlopen("libb.so", RTLD_NOW | RTLD_LOCAL);
 }

 void (*bfunc)();
 *(void**)(&bfunc) = dlsym(handle, "bfunc");
 bfunc();

 return 0;
}
EOF


cat >b.cpp <<EOF
// b.cpp
#include <iostream>
#include <dlfcn.h>

using namespace std;

extern "C" void bfunc() {
 cout << "bfunc in b.cpp is being called..." << endl;
 void * handle = dlopen("libc.so", RTLD_NOW | RTLD_GLOBAL);
 //void * handle = dlopen("libc.so", RTLD_NOW | RTLD_LOCAL);

 void (*cfunc)();
 *(void**)(&cfunc) = dlsym(handle, "cfunc");
 cfunc();
}
EOF

cat >c.cpp <<EOF
// c.cpp
#include <iostream>

using namespace std;

extern "C" void cfunc() {
 cout << "cfunc in c.cpp is being called..." << endl;
}
EOF

g++-10 a.cpp -o a -ldl -Wl,-rpath `pwd`
g++-10 b.cpp -o libb.so -shared -fPIC -ldl -Wl,-rpath `pwd`
g++-10 c.cpp -o libc.so -shared -fPIC

////////////////////////////////////////////////////////////


Finally, when I run LD_DEBUG=1 ./a, it crashes with the following log:

 902:	symbol=_ZNSt8ios_base4InitD1Ev; lookup in file=/root/libb.so [1]
 902:	symbol=_ZNSt8ios_base4InitD1Ev; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=__cxa_finalize; lookup in file=/root/libb.so [1]
 902:	symbol=__cxa_finalize; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=__cxa_finalize; lookup in file=/lib/aarch64-linux-gnu/libc.so.6 [1]
 902:	symbol=_ZStlsISt11char_traitsIcEERSt13basic_ostreamIcT_ES5_PKc; lookup in file=/root/libb.so [1]
 902:	symbol=_ZStlsISt11char_traitsIcEERSt13basic_ostreamIcT_ES5_PKc; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=_ZNSolsEPFRSoS_E; lookup in file=/root/libb.so [1]
 902:	symbol=_ZNSolsEPFRSoS_E; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=__cxa_atexit; lookup in file=/root/libb.so [1]
 902:	symbol=__cxa_atexit; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=__cxa_atexit; lookup in file=/lib/aarch64-linux-gnu/libc.so.6 [1]
 902:	symbol=_ZNSt8ios_base4InitC1Ev; lookup in file=/root/libb.so [1]
 902:	symbol=_ZNSt8ios_base4InitC1Ev; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=__gmon_start__; lookup in file=/root/libb.so [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libc.so.6 [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/ld-linux-aarch64.so.1 [0]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libm.so.6 [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libgcc_s.so.1 [1]
 902:	symbol=__gmon_start__; lookup in file=/root/libc.so [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libstdc++.so.6 [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libc.so.6 [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libm.so.6 [1]
 902:	symbol=__gmon_start__; lookup in file=/lib/ld-linux-aarch64.so.1 [0]
 902:	symbol=__gmon_start__; lookup in file=/lib/aarch64-linux-gnu/libgcc_s.so.1 [1]
Segmentation fault

The libc version that I’m using is 2.36 as outputted by

$ ldd --version
ldd (Ubuntu GLIBC 2.36-0ubuntu4) 2.36
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper

The error won’t happen if I dlopen with RTLD_LOCAL in b.cpp rather than RTLD_GLOBAL. I would appreciate any help on what I’m doing wrong here.

Best,
Nima