question on glibc interposition

question on glibc interposition

[christopher.aoki]

I have a question on glibc interposition.   Should calls to an
interposed function (e.g., malloc) from within a shared library
(e.g., libgfortran.so) bind to the interposing library, as opposed
to the glibc version?

For example I have a library that interposes malloc/free/etc. and am
finding cases in the Fortran runtime library libgfortran.so for which the
answer to the above question is “no”:

(gdb) where
#0  __GI___libc_malloc (bytes=5) at malloc.c:3030
#1  0x00007ffff76bdc28 in ?? () from /lib64/libgfortran.so.3
#2  0x00007ffff7799f5d in _gfortran_string_trim () from /lib64/libgfortran.so.3
#3  0x000000000046c083 in __textfile_module_MOD_open_for_read ()
#4  0x000000000046cf38 in __textfile_module_MOD_open_1 ()
#5  0x00000000004023b2 in main ()
(gdb) ^Z
Suspended
(xen.x86_64)% ps
  PID TTY          TIME CMD
  556 pts/1    00:00:00 ps
29977 pts/1    00:00:00 tcsh
32580 pts/1    00:00:00 gdb
32645 pts/1    00:00:00 tonto_base.glib
(xen.x86_64)% ldd tonto_base.glibc_mmheap 
	linux-vdso.so.1 =>  (0x00007ffd4b69d000)
	libmmheap_malloc.so.1 => /home/caoki/work/glibc/proto/lib64/libmmheap_malloc.so.1 (0x00007efeef3e5000)
	libmmheap.so.1 => /home/caoki/work/glibc/proto/lib64/libmmheap.so.1 (0x00007efeef1d4000)
	libgfortran.so.3 => /lib64/libgfortran.so.3 (0x00007efeeee9d000)
	libm.so.6 => /home/caoki/work/glibc/proto/lib64/libm.so.6 (0x00007efeeeb0b000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007efeee8f4000)
	libquadmath.so.0 => /lib64/libquadmath.so.0 (0x00007efeee6b8000)
	libc.so.6 => /home/caoki/work/glibc/proto/lib64/libc.so.6 (0x00007efeee305000)
	/home/caoki/work/glibc/proto/lib64/ld-2.26.9000.so => /lib64/ld-linux-x86-64.so.2 (0x00005574b1671000)
(xen.x86_64)% nm  /home/caoki/work/glibc/proto/lib64/libmmheap_malloc.so.1 | grep -w T
00000000000011d0 T aligned_alloc
0000000000000e90 T calloc
0000000000000f40 T free
0000000000000e00 T malloc
0000000000000f50 T memalign
0000000000001170 T posix_memalign
0000000000001000 T realloc
(xen.x86_64)% 

The problem is that in other cases in the same process, e.g., in
the main program, calls to malloc/free/etc are bound to the interposition
library.   This doesn’t work well when a block allocated by glibc malloc
is passed to the interposition library’s version of free().

Any clues appreciated.   I will gladly supply additional information 
if requested.

-chris

Re: question on glibc interposition

[Alexander Monakov]

On Sun, 11 Feb 2018, Alexander Monakov wrote:
> and after that LD_DEBUG stream between the 'run' and 'fin' commands should
> show you how the malloc call was bound.

Ugh, no, not that part. The relatively small part produced after 'fin' and
the gdb prompt after that (that's why LD_BIND_NOT is needed - to see bindings
after invoking _gfortran_string_trim even if some were already computed).

Alexander

Re: question on glibc interposition

[Alexander Monakov]

On Sat, 10 Feb 2018, christopher.aoki@oracle.com wrote:

> I have a question on glibc interposition.   Should calls to an
> interposed function (e.g., malloc) from within a shared library
> (e.g., libgfortran.so) bind to the interposing library, as opposed
> to the glibc version?

In general yes. There are exceptions related to dlopen or libc listed
earlier in DT_NEEDED tags (check readelf -d ./tonto*), but that seems
unlikely to be your case.

To investigate your situation, have a look at LD_DEBUG functionality
as documented in 'man ld.so'. You can do something like

  LD_DEBUG_OUTPUT=/tmp/lddbg.txt LD_DEBUG=all ./tonto...

and then peruse the output, or alternatively interactively in gdb like

  gdb --args ./tonto...
  (gdb) set env LD_DEBUG=all
  (gdb) set env LD_BIND_NOT=1
  (gdb) break _gfortran_string_trim
  (gdb) run
  [breakpoint hit]
  (gdb) fin

and after that LD_DEBUG stream between the 'run' and 'fin' commands should
show you how the malloc call was bound.

Hope that helps.
Alexander

Re: question on glibc interposition

[christopher.aoki]

> On Feb 11, 2018, at 1:05 AM, Alexander Monakov <amonakov@ispras.ru> wrote:
> 
> On Sun, 11 Feb 2018, Alexander Monakov wrote:
>> and after that LD_DEBUG stream between the 'run' and 'fin' commands should
>> show you how the malloc call was bound.
> 
> Ugh, no, not that part. The relatively small part produced after 'fin' and
> the gdb prompt after that (that's why LD_BIND_NOT is needed - to see bindings
> after invoking _gfortran_string_trim even if some were already computed).
> 
> Alexander

Yes, that was helpful.  It turned out that  my interposition library was misusing
symbol versioning, which apparently interferes with interposition of the intended
routines (malloc/free/etc.)  Thanks!

-chris

Re: question on glibc interposition

[Carlos O'Donell]

On 02/13/2018 11:39 AM, christopher.aoki@oracle.com wrote:
> 
>> On Feb 11, 2018, at 1:05 AM, Alexander Monakov <amonakov@ispras.ru> wrote:
>>
>> On Sun, 11 Feb 2018, Alexander Monakov wrote:
>>> and after that LD_DEBUG stream between the 'run' and 'fin' commands should
>>> show you how the malloc call was bound.
>>
>> Ugh, no, not that part. The relatively small part produced after 'fin' and
>> the gdb prompt after that (that's why LD_BIND_NOT is needed - to see bindings
>> after invoking _gfortran_string_trim even if some were already computed).
>>
>> Alexander
> 
> Yes, that was helpful.  It turned out that  my interposition library was misusing
> symbol versioning, which apparently interferes with interposition of the intended
> routines (malloc/free/etc.)  Thanks!

Glad to hear that.

In glibc we make sure malloc remains interposable for all callers.

The GNU toolchain does not have anything like group binding from Solaris which might
have allowed you to avoid the interposition. The only alternative would have been
a DSO that provided it's own memory allocator, but in that case you'd never see
any undefined references to malloc family symbols after the static linker ran.

-- 
Cheers,
Carlos.