[Bug dynamic-link/14379] New: shared object constructors are called in the wrong order

[Bug dynamic-link/14379] New: shared object constructors are called in the wrong order

[luto at mit dot edu]

http://sourceware.org/bugzilla/show_bug.cgi?id=14379

             Bug #: 14379
           Summary: shared object constructors are called in the wrong
                    order
           Product: glibc
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: dynamic-link
        AssignedTo: unassigned@sourceware.org
        ReportedBy: luto@mit.edu
    Classification: Unclassified


[This is not clearly a bug in the strictest sense, as the ELF spec, AFAICT, has
nothing useful to say.  It's rather unfortunate behavior, though, that makes
writing efficient and reliable malloc replacements very difficult, for
example.]

glibc's dynamic elf loader makes some effort to initialize an SO's DT_NEEDED
dependencies before initializing the SO that depended on them.  This is nice.  

Unfortunately, for sibling dependencies, it appears that the first library is
searched first for symbols (as required by the ELF spec) but the *last* library
is initialized first.  This means that, if an earlier library overrides a
symbol called by the constructor of a later library, then the earlier library
is called before it is initialized.

This is quite unfortunate for malloc replacements.  I have one that is intended
to be used with LD_PRELOAD, which causes its symbols to be searched first (as
desired) but causes it to be constructed last (which is bad).

Fixing this would be more complicated than just reversing the order of
initialization: it would still be good for deeper dependencies to be
initialized first.

As a demonstration, create lib.c and bin.c, as below.  Then do:

$ for name in lib1 lib2 preload_lib; do gcc -DLIBNAME="\"$name\"" -fPIC -shared
-o ${name}.so lib.c -Wl,--no-undefined; done
$ gcc -Wl,--no-as-needed -o bin_12 bin.c lib1.so lib2.so -Wl,-rpath -Wl,.
$ gcc -Wl,--no-as-needed -o bin_1 bin.c lib1.so -Wl,-rpath -Wl,.

$ ./bin_12
lib2: ctor called. calling the_func
  lib1: the_func called
  lib1: NOT INITED!
lib1: ctor called. calling the_func
  lib1: the_func called
main: calling the_func
  lib1: the_func called

$ LD_PRELOAD=preload_lib.so ./bin_1
lib1: ctor called. calling the_func
  preload_lib: the_func called
  preload_lib: NOT INITED!
preload_lib: ctor called. calling the_func
  preload_lib: the_func called
main: calling the_func
  preload_lib: the_func called


-- start of lib.c --

#include <unistd.h>
#define PRINT(x) do { static const char __s[] = x; write(1, __s,
sizeof(__s)-1); } while(0)

static int inited = 0;

void the_func(void)
{
  PRINT("  " LIBNAME ": the_func called\n");
  if (!inited)
    PRINT("  " LIBNAME ": NOT INITED!\n");
}

__attribute__((constructor)) static void ctor(void)
{
  inited = 1;
  PRINT(LIBNAME ": ctor called. calling the_func\n");
  the_func();
}

-- end of lib.c --

-- start of bin.c --

#include <unistd.h>
#define PRINT(x) do { static const char __s[] = x; write(1, __s,
sizeof(__s)-1); } while(0)

extern void the_func(void);

int main()
{
  PRINT("main: calling the_func\n");
  the_func();
  return 0;
}

-- end of bin.c --

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug dynamic-link/14379] shared object constructors are called in the wrong order

[ppluzhnikov at google dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=14379

Paul Pluzhnikov <ppluzhnikov at google dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ppluzhnikov at google dot
                   |                            |com

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug dynamic-link/14379] shared object constructors are called in the wrong order

[carlos_odonell at mentor dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=14379

Carlos O'Donell <carlos_odonell at mentor dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |WAITING
                 CC|                            |carlos_odonell at mentor
                   |                            |dot com

--- Comment #1 from Carlos O'Donell <carlos_odonell at mentor dot com> 2012-07-24 01:40:49 UTC ---
Could you please provide a concrete example including:

(a) Use case where the current ordering causes a problem.

(b) New ordering where the problem is solved. Please describe in detail what
you mean by "it would still be good for deeper dependencies to be
initialized first." 

We need to see more detailed use cases before we can make an informed judgement
here.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug dynamic-link/14379] shared object constructors are called in the wrong order

[luto at mit dot edu]

http://sourceware.org/bugzilla/show_bug.cgi?id=14379

--- Comment #2 from Andy Lutomirski <luto at mit dot edu> 2012-07-24 15:21:19 UTC ---
Here's a silly malloc replacement library:

#include <sys/mman.h>
#include <stdlib.h>

static char *arena;

__attribute__((constructor)) static void init()
{
  arena = (char*)mmap(0, 1 << 24, PROT_READ | PROT_WRITE, MAP_PRIVATE |
MAP_ANONYMOUS, -1, 0);
}

void *malloc(size_t n)
{
  void *ret = arena;
  arena += n;
  return ret;
}

void *realloc(void *ptr, size_t size)
{
  abort();
}

void free(void *p) {}


IMO it should work (as long as no one tries to use realloc, calloc, or threads,
but this is just an example).  Here's a rather ordinary shared library:

#include <malloc.h>

__attribute__((constructor)) static void init()
{
    *(char*)malloc(1) = 0;
}

This is, of course, silly, but in C++, dynamic allocation due to global object
constructors is common.

If I try to use that library with the malloc replacement LD_PRELOADed, it
crashes:

   19797:    calling init: /lib64/ld-linux-x86-64.so.2
     19797:    
     19797:    
     19797:    calling init: /lib64/libc.so.6
     19797:    
     19797:    
     19797:    calling init: /lib64/libgcc_s.so.1
     19797:    
     19797:    
     19797:    calling init: /lib64/libm.so.6
     19797:    
     19797:    
     19797:    calling init: /lib64/libstdc++.so.6
     19797:    
     19797:    
     19797:    calling init: ./test_lib.so
     19797:    
Segmentation fault

The malloc library (silly_malloc.so in this case) wasn't initialized.  I think
that, in any sensible initialization order, LD_PRELOADed libraries would
initialize first, not last.  (Maybe libraries in the DT_NEEDED list of the
LD_PRELOADed library should initialize even sooner, but that's irrelevant
here.)

My comment about initializing deeper dependencies first is something that
current glibc does right.  I don't think that should change.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug dynamic-link/14379] shared object constructors are called in the wrong order

[luto at mit dot edu]

http://sourceware.org/bugzilla/show_bug.cgi?id=14379

Andy Lutomirski <luto at mit dot edu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |UNCONFIRMED
     Ever Confirmed|1                           |0

--- Comment #3 from Andy Lutomirski <luto at mit dot edu> 2012-09-11 23:41:20 UTC ---
[marking as UNCONFIRMED because I don't think this is waiting for anything]

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug dynamic-link/14379] shared object constructors are called in the wrong order

[carlos_odonell at mentor dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=14379

--- Comment #4 from Carlos O'Donell <carlos_odonell at mentor dot com> 2012-09-12 02:10:44 UTC ---
(In reply to comment #3)
> [marking as UNCONFIRMED because I don't think this is waiting for anything]

That's right, you've provided an example and we need someone to review the
example. Thanks for updating the ticket.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug dynamic-link/14379] shared object constructors are called in the wrong order

[carlos at systemhalted dot org]

http://sourceware.org/bugzilla/show_bug.cgi?id=14379

Carlos O'Donell <carlos at systemhalted dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|carlos_odonell at mentor    |carlos at systemhalted dot
                   |dot com                     |org

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug dynamic-link/14379] shared object constructors are called in the wrong order

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=14379

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|                            |security-

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/14379] shared object constructors are called in the wrong order

[luto at mit dot edu]

https://sourceware.org/bugzilla/show_bug.cgi?id=14379

--- Comment #5 from Andy Lutomirski <luto at mit dot edu> ---
I think this bug is involved in a fairly reliable deadlock when a multithreaded
application that preloads tcmalloc and links to GnuTLS 3.2 or earlier calls
fork:

https://bugs.freedesktop.org/show_bug.cgi?id=84567

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/14379] shared object constructors are called in the wrong order

[luto at mit dot edu]

https://sourceware.org/bugzilla/show_bug.cgi?id=14379

--- Comment #6 from Andy Lutomirski <luto at mit dot edu> ---
FWIW, in the comments to your patch, you mention that pthread_atfork is nearly
impossible to use cleanly without atomics.  Unless I'm missing something, this
is true for the prepare handler, but not for the parent and child handlers --
there are no threads when the parent and child handlers are called.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/14379] shared object constructors are called in the wrong order

[luto at mit dot edu]

https://sourceware.org/bugzilla/show_bug.cgi?id=14379

--- Comment #7 from Andy Lutomirski <luto at mit dot edu> ---
(In reply to Andy Lutomirski from comment #6)

Sorry, ignore that comment.  Wrong BZ entry.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/14379] shared object constructors are called in the wrong order (with interposition)

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=14379

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|shared object constructors  |shared object constructors
                   |are called in the wrong     |are called in the wrong
                   |order                       |order (with interposition)
         Resolution|---                         |INVALID
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #14 from Florian Weimer <fweimer at redhat dot com> ---
I'm closing this as INVALID per the discussion here: interposition necessarily
goes in the opposite direction of DT_NEEDED dependencies, and interposing
objects need to be written in a way that reflects that (e.g., using lazy
initialization instead of ELF constructors).

-- 
You are receiving this mail because:
You are on the CC list for the bug.