[Bug libgcc/109685] New: Memory leak in `__deregister_frame`

[Bug libgcc/109685] New: Memory leak in `__deregister_frame`

[markus.boeck02 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109685

            Bug ID: 109685
           Summary: Memory leak in `__deregister_frame`
           Product: gcc
           Version: 13.1.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libgcc
          Assignee: unassigned at gcc dot gnu.org
          Reporter: markus.boeck02 at gmail dot com
  Target Milestone: ---

Sorry that I can't ship a proper reproducer, since I believe this essentially
requires a JIT or extracts of binary sections I am not yet familiar with. If I
do figure out a viable minimal reproducer I'll post them later.

With the recent release of GCC 13 landing on my fedora machine I have suddenly
started getting memory leaks reports by the leak sanitizer within a JIT
application of mine using `__register_frame` and `__deregister_frame`, pointing
to memory allocated by libgcc. I have then gone through debugging sessions with
GDB and found following oddities which I believe should be the causes of the
leak:

First of all, the memory allocation being leaked happens in `start_fde_sort`
https://github.com/gcc-mirror/gcc/blob/12de8da8961d294904d6af90b9cc27a5ba1ccfd0/libgcc/unwind-dw2-fde.c#L507
```
if ((accu->linear = malloc (size)))
    {
      accu->linear->count = 0;
      if ((accu->aux = malloc (size)))
        accu->aux->count = 0;
      return 1;
    }
```

Specifically the assignment to `accu->linear`. `accu->aux` is only temporarily
working memory that gets properly freed later. 
`accu->linear` instead gets put into an `object` that is inserted into a global
btree
(pointer is assigned to `u.sort`
https://github.com/gcc-mirror/gcc/blob/12de8da8961d294904d6af90b9cc27a5ba1ccfd0/libgcc/unwind-dw2-fde.c#L918)

The above call chains happens the first time unwinding happens since objects
are lazily initialized.

Later during JIT shutdown, `__deregsiter_frame` is called to erase all the
unwind information that has been produced.

This leads us to following code:
```
#ifdef ATOMIC_FDE_FAST_PATH
  ...
  uintptr_type range[2];
  get_pc_range (&lookupob, range);

  // And remove
  ob = btree_remove (&registered_frames, range[0]);
#else
  ...
#endif

  gcc_assert (in_shutdown || ob);
  return (void *) ob;
```
https://github.com/gcc-mirror/gcc/blob/12de8da8961d294904d6af90b9cc27a5ba1ccfd0/libgcc/unwind-dw2-fde.c#L242

with the caller calling `free` on the returned `ob`. 
Problem is that the `ob` may still have the pointer previously set by
`init_object` within its `u.sort` field. No attempt to free it is done within
the `ATOMIC_FDE_FAST_PATH` region however (something that does happen in the
#else region, which is seemingly not the default or maybe not enabled by the
distribution).

This therefore leads to the memory pointed to by `ob->u.sort` to become
unreachable and leak. 
The `ATOMIC_FDE_FAST_PATH` fast path was only added after the GCC 12 release
which would also explain why the LSAN only caught the leak after the GCC 13
release

[Bug libgcc/109685] [13/14 Regression] Memory leak in `__deregister_frame`

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109685

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org,
                   |                            |tneumann at users dot sourceforge.
                   |                            |net
            Summary|Memory leak in              |[13/14 Regression] Memory
                   |`__deregister_frame`        |leak in
                   |                            |`__deregister_frame`
   Target Milestone|---                         |13.2

[Bug libgcc/109685] [13/14 Regression] Memory leak in `__deregister_frame`

[tneumann at users dot sourceforge.net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109685

--- Comment #1 from Thomas Neumann <tneumann at users dot sourceforge.net> ---
Created attachment 54969
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=54969&action=edit
fix for the issue

[Bug libgcc/109685] [13/14 Regression] Memory leak in `__deregister_frame`

[tneumann at users dot sourceforge.net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109685

--- Comment #2 from Thomas Neumann <tneumann at users dot sourceforge.net> ---
I can reproduce the issue. The attached patch fixes the problem, I will send it
for reviewing.

[Bug libgcc/109685] [13/14 Regression] Memory leak in `__deregister_frame`

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109685

--- Comment #3 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Thomas Neumann <tneumann@gcc.gnu.org>:

https://gcc.gnu.org/g:5cf60b6ba111f4169305c7832b063b000e9ec36a

commit r14-1514-g5cf60b6ba111f4169305c7832b063b000e9ec36a
Author: Thomas Neumann <tneumann@users.sourceforge.net>
Date:   Tue May 2 16:21:09 2023 +0200

    release the sorted FDE array when deregistering a frame [PR109685]

    The atomic fastpath bypasses the code that releases the sort
    array which was lazily allocated during unwinding. We now
    check after deregistering if there is an array to free.

    libgcc/ChangeLog:
            PR libgcc/109685
            * unwind-dw2-fde.c: Free sort array in atomic fast path.

[Bug libgcc/109685] [13/14 Regression] Memory leak in `__deregister_frame`

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109685

--- Comment #4 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-13 branch has been updated by Thomas Neumann
<tneumann@gcc.gnu.org>:

https://gcc.gnu.org/g:fd68f60c4ca5d23e4f42c447a2fd1d33030301bf

commit r13-7410-gfd68f60c4ca5d23e4f42c447a2fd1d33030301bf
Author: Thomas Neumann <tneumann@users.sourceforge.net>
Date:   Tue May 2 16:21:09 2023 +0200

    release the sorted FDE array when deregistering a frame [PR109685]

    The atomic fastpath bypasses the code that releases the sort
    array which was lazily allocated during unwinding. We now
    check after deregistering if there is an array to free.

    libgcc/ChangeLog:
            PR libgcc/109685
            * unwind-dw2-fde.c: Free sort array in atomic fast path.

[Bug libgcc/109685] [13/14 Regression] Memory leak in `__deregister_frame`

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109685

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|13.2                        |13.3

--- Comment #5 from Richard Biener <rguenth at gcc dot gnu.org> ---
GCC 13.2 is being released, retargeting bugs to GCC 13.3.

[Bug libgcc/109685] [13/14 Regression] Memory leak in `__deregister_frame`

[tneumann at users dot sourceforge.net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109685

--- Comment #6 from Thomas Neumann <tneumann at users dot sourceforge.net> ---
> GCC 13.2 is being released, retargeting bugs to GCC 13.3.

the bug should be closed as fixed, the bug fix is already in the 13.2 branch.
(I do not have permissions to do that, though).

[Bug libgcc/109685] [13/14 Regression] Memory leak in `__deregister_frame`

[markus.boeck02 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109685

Markus Böck <markus.boeck02 at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |FIXED

--- Comment #7 from Markus Böck <markus.boeck02 at gmail dot com> ---
Fixed

[Bug libgcc/109685] [13/14 Regression] Memory leak in `__deregister_frame`

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109685

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|13.3                        |13.2