Re: GDB abort on glibc detected file descriptor overflow

["Ananthakrishna Sowda (asowda)" <asowda@cisco.com>]

I looked in /proc/<pid>/fd for Gdb process and I see 1535 files open. Most of them are pointing to libfoo.so and libfoo.so.debug , where libfoo.so is a library loaded by the process which crashed and generated a core-file. Normally, these file descriptors are re-used and maximum number of open file descriptors stays around 150, even for a process which could load several hundreds of .so libraries.


From: Simon Marchi <simon.marchi@polymtl.ca>
Date: Wednesday, September 1, 2021 at 6:05 PM
To: Ananthakrishna Sowda (asowda) <asowda@cisco.com>, gdb@sourceware.org <gdb@sourceware.org>
Subject: Re: GDB abort on glibc detected file descriptor overflow


On 2021-09-01 6:37 p.m., Ananthakrishna Sowda (asowda) via Gdb wrote:
> I’m observing abort in GDB 9.2.1 version, and same issue is present in git://sourceware.org/git/binutils-gdb.git tip.
>
> The full call trace is shown at the end of this message.
> In frame 7,  call to FD_SET is causing buffer overflow when commands from a GDB macro file are processed.
>
> (gdb) frame 7
> #7  0x000000000076978b in gdb_readline_no_editing (prompt=<optimized out>) at /auto/swtools/prod-builds/src/gdb-9.2.1/gdb/gdb/top.c:850
> 850               FD_SET (fd, &readfds);
> (gdb) p fd
> $1 = 1533
>
> GDB is processing split dwarf  “.dwp” file for the main executable and processing some “.dwo” files in the workspace, which may have something to do with it.  GDB is opening a bunch of .debug files , one each for every  library  and the open file descriptors  go  past 1024.  This results in buffer overflow when gdb.macros file is opened and processed in frame 7 ( file descriptor 1533).
>
> The bfd file descriptor caching code which tries to limit no of open descriptors is not effective in this case.
> Does this explanation make  sense? Any ideas to fix this issue are greatly appreciated.

It won't fix the problem, but I think we could start by adding an
assertion before calling FD_SET (everywhere where we do call it):

  gdb_assert (fd < FD_SETSIZE);

Your build happened to catch it, but other builds could just fail
silently or crash in less clear ways.

As for the solution, maybe this code should be converted to use poll or
other more modern APIs to avoid this limit?

It would be interesting if you could show what are the open file
descriptors at this point (list /proc/<pid>/fd), just to see what uses
the most fds.

Simon