[Bug libstdc++/67791] New: Crash using std::thread and iostream with dynamic loading of a shared library

[Bug libstdc++/67791] New: Crash using std::thread and iostream with dynamic loading of a shared library

[nexyon at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

            Bug ID: 67791
           Summary: Crash using std::thread and iostream with dynamic
                    loading of a shared library
           Product: gcc
           Version: 5.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libstdc++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: nexyon at gmail dot com
  Target Milestone: ---

Created attachment 36430
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=36430&action=edit
the source of the shared library

If you think the summary of this bug is long/strange: yes, to me this seems
like the most weird bug, I have ever found.

What I am doing is the following:

* I have a main program (main.cpp) that loads a dynamic library thread.so and
calls a function in there.
* main.cpp also includes <iostream> without actually using this library.
(sounds weird, eh?)
* The function in thread.so creates a std::thread and runs it.
* The application crashes when the thread should be started.

Let me show you what happens (or: how to reproduce):

--------------

% g++ -g -fPIC -shared -o thread.so thread.cpp -std=c++11 -lpthread
% g++ -g -ldl -o main main.cpp
%  gdb main
        GNU gdb (GDB) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from main...done.
(gdb) r
Starting program: /data/Work/Computer/Programming/Audaspace/bug/main 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt full
#0  0x0000000000000000 in ?? ()
No symbol table info available.
#1  0x00007ffff790e493 in __gthread_create (__args=<optimized out>,
__func=0x7ffff790e330 <std::(anonymous
namespace)::execute_native_thread_routine(void*)>, __threadid=0x7fffffffdd60)
    at
/build/gcc-multilib/src/gcc-build/x86_64-unknown-linux-gnu/libstdc++-v3/include/x86_64-unknown-linux-gnu/bits/gthr-default.h:662
No locals.
#2  std::thread::_M_start_thread (this=0x7fffffffdd60, __b=...) at
/build/gcc-multilib/src/gcc-5.2.0/libstdc++-v3/src/c++11/thread.cc:149
No locals.
#3  0x00007ffff6d99826 in std::thread::thread<void (&)(int, unsigned long
long*), int, unsigned long long*&>(void (&)(int, unsigned long long*), int&&,
unsigned long long*&) (this=0x7fffffffdd60, __f=
    @0x7ffff6d993e2: {void (int, unsigned long long *)} 0x7ffff6d993e2
<factorial(int, unsigned long long*)>) at /usr/include/c++/5.2.0/thread:137
No locals.
#4  0x00007ffff6d99463 in run () at thread.cpp:17
        a = 4196233
        c = 0x7fffffffdd70
        t1 = {_M_id = {_M_thread = 0}}
#5  0x0000000000400793 in main (argc=1, argv=0x7fffffffdea8) at main.cpp:9
        handle = 0x612c40
        run_func = 0x7ffff6d99430 <run()>
(gdb)

--------------

Now why is this weird/funny?

There are two ways you can get rid of the crash:

* the first is to add -lpthread to the second g++ line (the one that compiles
main.cpp and links the main program) OR
* you remove the line #include <iostream> in main.cpp

I found this bug in a way bigger code base as you can imagine and I already
spent a lot of time to get it down to this minimal example. I hope you can
find, what is the problem here! Thanks!

[Bug libstdc++/67791] Crash using std::thread and iostream with dynamic loading of a shared library

[nexyon at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

--- Comment #1 from nexyon at gmail dot com ---
Created attachment 36431
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=36431&action=edit
the source of the main program

[Bug libstdc++/67791] Crash using std::thread and iostream with dynamic loading of a shared library

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
So the problem here is libstdc++ first tries to figures out if you are linked
against pthreads and if you are not then it skips locks in some cases.  In this
case since the main program was not linked against pthreads and you initialized
libstdc++ via the include of iostream, it sets the single threaded use case.

[Bug libstdc++/67791] Crash using std::thread and iostream with dynamic loading of a shared library

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

--- Comment #3 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Yes, this is a known problem with the __gthread_active_p() trick we use, which
doesn't work when libpthread gets loaded later via a dlopen call.

[Bug libstdc++/67791] Crash using std::thread and iostream with dynamic loading of a shared library

[nexyon at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

--- Comment #4 from nexyon at gmail dot com ---
Thanks for the quick responses! I already expected some sort of side effect
like this. Maybe it's possible to reevaluate whether pthread is linked or not
during the first use of std::thread? In any case std::thread should not crash I
guess, so just to fix that it would be necessary to check whether the standard
library was inizialized without threads. Or shouldn't there be locks in any
case? What if another threading library is used but pthreads, that libstdc++ is
not expecting? (Not sure though, how practical that is, or how likely that's
going to happen)

As the problem seems to be known, I wonder if I can expect this to be solved
any time soon? For now I guess my options are to tell the users of my library
and it's plugins to always link pthread to their main program or maybe
artificially add a function in my library that uses std::thread so that my
library gets linked against pthread. Then linking dynamically with it in the
executable should not cause problems, only symloading the library would still
cause the same problem. Or do you have other ideas?

Cheers!

[Bug libstdc++/67791] Crash using std::thread and iostream with dynamic loading of a shared library

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2020-11-23
             Status|UNCONFIRMED                 |NEW
     Ever confirmed|0                           |1

--- Comment #7 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to nexyon from comment #4)
> Thanks for the quick responses! I already expected some sort of side effect
> like this. Maybe it's possible to reevaluate whether pthread is linked or
> not during the first use of std::thread?

No, because the result is cached in a static const variable, which can't be
changed after its first use.

> In any case std::thread should not
> crash I guess, so just to fix that it would be necessary to check whether
> the standard library was inizialized without threads.

We used to check for the pthread_create weak symbol, and throw an exception
instead of crashing. Since the fix for PR libstdc++/61841 we don't check it,
and crash instead of throwing. That should be fixed.

> Or shouldn't there be
> locks in any case? What if another threading library is used but pthreads,
> that libstdc++ is not expecting? (Not sure though, how practical that is, or
> how likely that's going to happen)

That's not supported. The only supported way to create new threads is via
pthreads.

> As the problem seems to be known, I wonder if I can expect this to be solved
> any time soon? For now I guess my options are to tell the users of my
> library and it's plugins to always link pthread to their main program or

Right.

> maybe artificially add a function in my library that uses std::thread so
> that my library gets linked against pthread. Then linking dynamically with
> it in the executable should not cause problems, only symloading the library
> would still cause the same problem. Or do you have other ideas?

If the main executable is not linked to libpthread, using dlopen to load a
library that depends on libpthread doesn't work. This is currently unsupported,
and unsupportable on GNU/Linux.

If your library depends on libpthread.so and the main executable links to your
library that should be fine. You don't need to use std::thread for that, just
link to it without using --as-needed e.g. -Wl,--no-as-needed -lpthread
-Wl,--as-needed

Late-loading of libpthread.so via dlopen should start working with a future
version of glibc, when this is implemented:
https://sourceware.org/legacy-ml/libc-alpha/2019-10/msg00080.html

[Bug libstdc++/67791] [8/9/10/11 Regression] Crash using std::thread and iostream with dynamic loading of a shared library

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Crash using std::thread and |[8/9/10/11 Regression]
                   |iostream with dynamic       |Crash using std::thread and
                   |loading of a shared library |iostream with dynamic
                   |                            |loading of a shared library
      Known to work|                            |4.9.4
           Assignee|unassigned at gcc dot gnu.org      |redi at gcc dot gnu.org
             Status|NEW                         |ASSIGNED
      Known to fail|                            |10.2.0, 11.0, 5.1.0, 8.4.0,
                   |                            |9.3.0
   Target Milestone|---                         |8.5

--- Comment #8 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Jonathan Wakely from comment #7)
> (In reply to nexyon from comment #4)
> > In any case std::thread should not
> > crash I guess, so just to fix that it would be necessary to check whether
> > the standard library was inizialized without threads.
> 
> We used to check for the pthread_create weak symbol, and throw an exception
> instead of crashing. Since the fix for PR libstdc++/61841 we don't check it,
> and crash instead of throwing. That should be fixed.

Marking as a regression, because GCC 4.9 printed this error instead of
crashing:

terminate called after throwing an instance of 'std::system_error'
  what():  Enable multithreading to use std::thread: Operation not permitted
Aborted (core dumped)

That's better than segfaulting with no explanation.

I incorrectly assumed that the use of &pthread_create added for PR 61841 would
make checking __gthread_active_p unnecessary. As the examples here show, that's
not true. The program calls __gthread_active_p() before main() runs, and at
that time libpthread.so is not loaded. When libpthread.so is loaded later, it's
too late for __gthread_active_p to notice.

[Bug libstdc++/67791] [8/9/10/11 Regression] Crash using std::thread and iostream with dynamic loading of a shared library

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

--- Comment #9 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jonathan Wakely <redi@gcc.gnu.org>:

https://gcc.gnu.org/g:4bbd5d0c5fb2b7527938ad44a6d8a2f2ef8bbe12

commit r11-5315-g4bbd5d0c5fb2b7527938ad44a6d8a2f2ef8bbe12
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Tue Nov 24 12:48:31 2020 +0000

    libstdc++: Throw instead of segfaulting in std::thread constructor [PR
67791]

    This turns a mysterious segfault into an exception with a more useful
    message. If the exception isn't caught, the user sees this instead of
    just a segfault:

    terminate called after throwing an instance of 'std::system_error'
      what():  Enable multithreading to use std::thread: Operation not
permitted
    Aborted (core dumped)

    libstdc++-v3/ChangeLog:

            PR libstdc++/67791
            * src/c++11/thread.cc (thread::_M_start_thread(_State_ptr, void
(*)())):
            Check that gthreads is available before calling __gthread_create.

[Bug libstdc++/67791] [8/9/10 Regression] Crash using std::thread and iostream with dynamic loading of a shared library

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|redi at gcc dot gnu.org            |unassigned at gcc dot gnu.org
             Status|ASSIGNED                    |NEW

[Bug libstdc++/67791] [9/10 Regression] Crash using std::thread and iostream with dynamic loading of a shared library

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|8.5                         |9.4

--- Comment #10 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
GCC 8 branch is being closed.

[Bug libstdc++/67791] [9/10 Regression] Crash using std::thread and iostream with dynamic loading of a shared library

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|9.4                         |9.5

--- Comment #11 from Richard Biener <rguenth at gcc dot gnu.org> ---
GCC 9.4 is being released, retargeting bugs to GCC 9.5.

[Bug libstdc++/67791] [10 Regression] Crash using std::thread and iostream with dynamic loading of a shared library

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|9.5                         |10.4

--- Comment #12 from Richard Biener <rguenth at gcc dot gnu.org> ---
GCC 9 branch is being closed

[Bug libstdc++/67791] [10 Regression] Crash using std::thread and iostream with dynamic loading of a shared library

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|10.4                        |10.5

--- Comment #13 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
GCC 10.4 is being released, retargeting bugs to GCC 10.5.

[Bug libstdc++/67791] [10 Regression] Crash using std::thread and iostream with dynamic loading of a shared library

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

--- Comment #14 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-10 branch has been updated by Jonathan Wakely
<redi@gcc.gnu.org>:

https://gcc.gnu.org/g:e2babb0540cfa66752214f3f111461e1da4b26ce

commit r10-11325-ge2babb0540cfa66752214f3f111461e1da4b26ce
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Tue Nov 24 12:48:31 2020 +0000

    libstdc++: Throw instead of segfaulting in std::thread constructor [PR
67791]

    This turns a mysterious segfault into an exception with a more useful
    message. If the exception isn't caught, the user sees this instead of
    just a segfault:

    terminate called after throwing an instance of 'std::system_error'
      what():  Enable multithreading to use std::thread: Operation not
permitted
    Aborted (core dumped)

    libstdc++-v3/ChangeLog:

            PR libstdc++/67791
            * src/c++11/thread.cc (thread::_M_start_thread(_State_ptr, void
(*)())):
            Check that gthreads is available before calling __gthread_create.

    (cherry picked from commit 4bbd5d0c5fb2b7527938ad44a6d8a2f2ef8bbe12)

[Bug libstdc++/67791] [10 Regression] Crash using std::thread and iostream with dynamic loading of a shared library

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #15 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Fixed for 10.5 and up.

[Bug libstdc++/67791] [10 Regression] Crash using std::thread and iostream with dynamic loading of a shared library

[colin.davidson at codeplay dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

colin.davidson at codeplay dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |colin.davidson at codeplay dot com

--- Comment #16 from colin.davidson at codeplay dot com ---
I've started seeing this throw error in the last few days (both in dockers and
locally on ubuntu 20.04). This was in previously working code which used dlopen
on a library with pthreads. We used RTLD_LAZY | RTLD_GLOBAL to work around
previous crashes but now we can't get the code to work unless we LD_PRELOAD the
pthreads library first.

I'm on libstdc++.so.6.0.28,  gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0

[Bug libstdc++/67791] [10 Regression] Crash using std::thread and iostream with dynamic loading of a shared library

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67791

--- Comment #17 from Jonathan Wakely <redi at gcc dot gnu.org> ---
GCC 9 hasn't changed, so it must be something in your Ubuntu environment or
your code.

I don't think the RTLD_xxx approach was ever robust, the libpthread.so library
needs to be loaded before the first __gthread_active_p() check in the whole
executable. That can be achieved by linking the executable to libpthread or
preloading it. Or using a newer glibc where libpthread.so is empty and all the
pthreads functions live in libc.so anyway.