C++ exceptions with many threads and no memory.

C++ exceptions with many threads and no memory.

[John Steele Scott]

I'm assessing the issues involved with migrating a largish project from C to C++. One of the things which has come up is exception handling. This application allocates a number of error-context structures on startup, in order to ensure that they are available even under out-of-memory conditions.

When comparing this to GCC's exception handling, I find that libstdc++ calls malloc() whenever an exception is thrown, and falls back to statically allocated memory only when malloc fails. Only 64 statically allocated exceptions may be in flight at once, and if this limit is violated, std::terminate() is called.

This seems like quite a brittle behaviour to me. I guess not often seen as a problem in practice because Linux overcommit usually ensures malloc() won't fail (although if it does, then you have to take your chances with the oom-killer anyway).

There doesn't seem to be any way around this, is there? Do GCC developers consider it a bug?

The code below demonstrates the issue. With 64 threads it runs to completion, with 65 threads it triggers the following backtrace, with GCC 4.8.1.

(gdb) bt
#0  0x00007ffff7319425 in __GI_raise (sig=<optimised out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x00007ffff731cb8b in __GI_abort () at abort.c:91
#2  0x00007ffff7b36b05 in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff7b34c76 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff7b34ca3 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff7b339c5 in __cxa_allocate_exception () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x0000000000401467 in f () at libstdc++-exception-malloc.cpp:37
#7  0x00007ffff7b87c00 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#8  0x00007ffff76aae9a in start_thread (arg=0x7fffd77a7700) at pthread_create.c:308
#9  0x00007ffff73d73fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#10 0x0000000000000000 in ?? ()

Cheers,

John

#include <chrono>
#include <condition_variable>
#include <exception>
#include <stdlib.h>
#include <mutex>
#include <thread>
#include <vector>

std::mutex m;
std::condition_variable cv;
static bool all_threads_have_spawned = false;

class MyClass
{
public:
  ~MyClass ()
  {
    /* Simulate the stack unwind taking a long time. */
    std::this_thread::sleep_for(std::chrono::milliseconds(1000));
  }
};

static void f ()
{
  try
  {
    MyClass obj;

    {
      /* Wait for the main thread to signal us, in order to avoid throwing until
       * malloc has been told to fail.
       */
      std::unique_lock<std::mutex> lock(m);
      cv.wait(lock, []{ return all_threads_have_spawned;} );
    }

    throw std::bad_alloc();
  }
  catch (std::bad_alloc)
  {

  }
}

/* Declare glibc's internal malloc(), so we can call it from our malloc() wrapper. */
extern "C" void *__libc_malloc(size_t size);

static bool fail_malloc = false;

extern "C" void *malloc (size_t size)
{
  return fail_malloc ? NULL : __libc_malloc(size);
}

int main ()
{
  std::vector<std::thread> threads;

  /* With 64 threads this program runs to completion. With 65 threads it dies
   * with std::terminate() called from __cxa_allocate_exception().
   */
  const int THREAD_COUNT = 65;

  for (int i = 0; i < THREAD_COUNT; i++)
  {
    threads.emplace_back(f);
  }

  /* All the threads have been created. Tell our version of malloc() to fail,
   * from now on, simulating out-of-memory.
   */
  fail_malloc = true;

  {
    /* Wake all our threads. */
    all_threads_have_spawned = true;
    std::unique_lock<std::mutex> lock(m);
    cv.notify_all();
  }

  for (auto &thread : threads)
  {
    thread.join();
  }

  return 0;
}

Re: C++ exceptions with many threads and no memory.

[Ian Lance Taylor]

On Mon, Mar 3, 2014 at 3:53 AM, John Steele Scott <toojays@toojays.net> wrote:
>
> When comparing this to GCC's exception handling, I find that libstdc++ calls malloc() whenever an exception is thrown, and falls back to statically allocated memory only when malloc fails. Only 64 statically allocated exceptions may be in flight at once, and if this limit is violated, std::terminate() is called.
>
> This seems like quite a brittle behaviour to me. I guess not often seen as a problem in practice because Linux overcommit usually ensures malloc() won't fail (although if it does, then you have to take your chances with the oom-killer anyway).
>
> There doesn't seem to be any way around this, is there? Do GCC developers consider it a bug?

If you have a good proposal for how to handle this situation, I'm sure
that GCC developers would consider it.

It seems pretty tough to handle large numbers of exceptions when
malloc fails.  And since it is such an unusual condition, I don't
think it would be a good idea to address it in a manner that penalizes
performance in ordinary conditions.

Ian

Re: C++ exceptions with many threads and no memory.

[John Steele Scott]

On 04/03/14 17:05, Ian Lance Taylor wrote:
> On Mon, Mar 3, 2014 at 3:53 AM, John Steele Scott <toojays@toojays.net> wrote:
>>
>> When comparing this to GCC's exception handling, I find that libstdc++ calls malloc() whenever an exception is thrown, and falls back to statically allocated memory only when malloc fails. Only 64 statically allocated exceptions may be in flight at once, and if this limit is violated, std::terminate() is called.
>>
>> This seems like quite a brittle behaviour to me. I guess not often seen as a problem in practice because Linux overcommit usually ensures malloc() won't fail (although if it does, then you have to take your chances with the oom-killer anyway).
>>
>> There doesn't seem to be any way around this, is there? Do GCC developers consider it a bug?
> 
> If you have a good proposal for how to handle this situation, I'm sure
> that GCC developers would consider it.
> 
> It seems pretty tough to handle large numbers of exceptions when
> malloc fails.  And since it is such an unusual condition, I don't
> think it would be a good idea to address it in a manner that penalizes
> performance in ordinary conditions.

Thanks Ian for your reply.

I agree, it's tough to find a suitable general solution which scales with the
number of threads, and I don't have a solution for the general case. In my case,
we have a thread budget, it's just that the number of threads we use is much
larger than the 64 which GCC can guarantee to safely handle.

I'd settle for being able to supply my own pair of allocation/deallocation
functions for this specific use case, and having enough documentation to be able
to confidently estimate the amount (size and shape, i.e. N blocks of M bytes
each) of memory I'd need to reserve for this.

From what I've seen so far, it seems that as long as I know the maximum number
of threads in my program, and the maximum size of the exceptions in my program,
this should be doable, if I could override the memory allocator just for
__cxa_allocate_exception() and friends.

I admit to not having a deep understanding of GCC/libstdc++ implementation
details or C++ exceptions. If there are other reasons as to why this wouldn't
work, I'd appreciate knowing about them.

Cheers,

John

Re: C++ exceptions with many threads and no memory.

[John Steele Scott]

On 04/03/14 17:05, Ian Lance Taylor wrote:
> On Mon, Mar 3, 2014 at 3:53 AM, John Steele Scott <toojays@toojays.net> wrote:
>>
>> When comparing this to GCC's exception handling, I find that libstdc++ calls malloc() whenever an exception is thrown, and falls back to statically allocated memory only when malloc fails. Only 64 statically allocated exceptions may be in flight at once, and if this limit is violated, std::terminate() is called.
>>
>> This seems like quite a brittle behaviour to me. I guess not often seen as a problem in practice because Linux overcommit usually ensures malloc() won't fail (although if it does, then you have to take your chances with the oom-killer anyway).
>>
>> There doesn't seem to be any way around this, is there? Do GCC developers consider it a bug?
> 
> If you have a good proposal for how to handle this situation, I'm sure
> that GCC developers would consider it.
> 
> It seems pretty tough to handle large numbers of exceptions when
> malloc fails.  And since it is such an unusual condition, I don't
> think it would be a good idea to address it in a manner that penalizes
> performance in ordinary conditions.

Thanks Ian for your reply.

I agree, it's tough to find a suitable general solution which scales with the
number of threads, and I don't have a solution for the general case. In my case,
we have a thread budget, it's just that the number of threads we use is much
larger than the 64 which GCC can guarantee to safely handle.

I'd settle for being able to supply my own pair of allocation/deallocation
functions for this specific use case, and having enough documentation to be able
to confidently estimate the amount (size and shape, i.e. N blocks of M bytes
each) of memory I'd need to reserve for this.

From what I've seen so far, it seems that as long as I know the maximum number
of threads in my program, and the maximum size of the exceptions in my program,
this should be doable, if I could override the memory allocator just for
__cxa_allocate_exception() and friends.

I admit to not having a deep understanding of GCC/libstdc++ implementation
details or C++ exceptions. If there are other reasons as to why this wouldn't
work, I'd appreciate knowing about them.

Cheers,

John