[Bug libgcc/109685] New: Memory leak in `__deregister_frame`

["markus.boeck02 at gmail dot com" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109685

            Bug ID: 109685
           Summary: Memory leak in `__deregister_frame`
           Product: gcc
           Version: 13.1.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libgcc
          Assignee: unassigned at gcc dot gnu.org
          Reporter: markus.boeck02 at gmail dot com
  Target Milestone: ---

Sorry that I can't ship a proper reproducer, since I believe this essentially
requires a JIT or extracts of binary sections I am not yet familiar with. If I
do figure out a viable minimal reproducer I'll post them later.

With the recent release of GCC 13 landing on my fedora machine I have suddenly
started getting memory leaks reports by the leak sanitizer within a JIT
application of mine using `__register_frame` and `__deregister_frame`, pointing
to memory allocated by libgcc. I have then gone through debugging sessions with
GDB and found following oddities which I believe should be the causes of the
leak:

First of all, the memory allocation being leaked happens in `start_fde_sort`
https://github.com/gcc-mirror/gcc/blob/12de8da8961d294904d6af90b9cc27a5ba1ccfd0/libgcc/unwind-dw2-fde.c#L507
```
if ((accu->linear = malloc (size)))
    {
      accu->linear->count = 0;
      if ((accu->aux = malloc (size)))
        accu->aux->count = 0;
      return 1;
    }
```

Specifically the assignment to `accu->linear`. `accu->aux` is only temporarily
working memory that gets properly freed later. 
`accu->linear` instead gets put into an `object` that is inserted into a global
btree
(pointer is assigned to `u.sort`
https://github.com/gcc-mirror/gcc/blob/12de8da8961d294904d6af90b9cc27a5ba1ccfd0/libgcc/unwind-dw2-fde.c#L918)

The above call chains happens the first time unwinding happens since objects
are lazily initialized.

Later during JIT shutdown, `__deregsiter_frame` is called to erase all the
unwind information that has been produced.

This leads us to following code:
```
#ifdef ATOMIC_FDE_FAST_PATH
  ...
  uintptr_type range[2];
  get_pc_range (&lookupob, range);

  // And remove
  ob = btree_remove (&registered_frames, range[0]);
#else
  ...
#endif

  gcc_assert (in_shutdown || ob);
  return (void *) ob;
```
https://github.com/gcc-mirror/gcc/blob/12de8da8961d294904d6af90b9cc27a5ba1ccfd0/libgcc/unwind-dw2-fde.c#L242

with the caller calling `free` on the returned `ob`. 
Problem is that the `ob` may still have the pointer previously set by
`init_object` within its `u.sort` field. No attempt to free it is done within
the `ATOMIC_FDE_FAST_PATH` region however (something that does happen in the
#else region, which is seemingly not the default or maybe not enabled by the
distribution).

This therefore leads to the memory pointed to by `ob->u.sort` to become
unreachable and leak. 
The `ATOMIC_FDE_FAST_PATH` fast path was only added after the GCC 12 release
which would also explain why the LSAN only caught the leak after the GCC 13
release