[Bug malloc/23328] Remove malloc hooks and ensure related APIs return no data.

[Bug malloc/23328] Remove malloc hooks and ensure related APIs return no data.

[siddhesh at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=23328

Siddhesh Poyarekar <siddhesh at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |siddhesh at sourceware dot org
           Assignee|unassigned at sourceware dot org   |siddhesh at sourceware dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/23328] Remove malloc hooks and ensure related APIs return no data.

[siddhesh at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=23328

Siddhesh Poyarekar <siddhesh at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED
   Target Milestone|---                         |2.34

--- Comment #5 from Siddhesh Poyarekar <siddhesh at sourceware dot org> ---
commit 1e5a5866cb9541b5231dba3d86c8a1a35d516de9
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
Date:   Thu Jul 22 18:38:12 2021 +0530

    Remove malloc hooks [BZ #23328]

    Make malloc hooks symbols compat-only so that new applications cannot
    link against them and remove the declarations from the API.  Also
    remove the unused malloc-hooks.h.

    Finally, mark all symbols in libc_malloc_debug.so as compat so that
    the library cannot be linked against.

    Add a note about the deprecation in NEWS.

    Reviewed-by: Carlos O'Donell <carlos@redhat.com>
    Tested-by: Carlos O'Donell <carlos@redhat.com>

commit 0552fd2c7d4e8a570cb4fe4dfe65e96f6d24b0cd
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
Date:   Thu Jul 22 18:38:10 2021 +0530

    Move malloc_{g,s}et_state to libc_malloc_debug

    These deprecated functions are only safe to call from
    __malloc_initialize_hook and as a result, are not useful in the
    general case.  Move the implementations to libc_malloc_debug so that
    existing binaries that need it will now have to preload the debug DSO
    to work correctly.

    This also allows simplification of the core malloc implementation by
    dropping all the undumping support code that was added to make
    malloc_set_state work.

    One known breakage is that of ancient emacs binaries that depend on
    this.  They will now crash when running with this libc.  With
    LD_BIND_NOW=1, it will terminate immediately because of not being able
    to find malloc_set_state but with lazy binding it will crash in
    unpredictable ways.  It will need a preloaded libc_malloc_debug.so so
    that its initialization hook is executed to allow its malloc
    implementation to work properly.

    Reviewed-by: Carlos O'Donell <carlos@redhat.com>
    Tested-by: Carlos O'Donell <carlos@redhat.com>

commit b5bd5bfe88f496463ec9fab680a8edf64d7c2a42
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
Date:   Thu Jul 22 18:38:08 2021 +0530

    glibc.malloc.check: Wean away from malloc hooks

    The malloc-check debugging feature is tightly integrated into glibc
    malloc, so thanks to an idea from Florian Weimer, much of the malloc
    implementation has been moved into libc_malloc_debug.so to support
    malloc-check.  Due to this, glibc malloc and malloc-check can no
    longer work together; they use altogether different (but identical)
    structures for heap management.  This should not make a difference
    though since the malloc check hook is not disabled anywhere.
    malloc_set_state does, but it does so early enough that it shouldn't
    cause any problems.

    The malloc check tunable is now in the debug DSO and has no effect
    when the DSO is not preloaded.

    Reviewed-by: Carlos O'Donell <carlos@redhat.com>
    Tested-by: Carlos O'Donell <carlos@redhat.com>

commit 9dad716d4d2993f50b165747781244bd7c43bc95
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
Date:   Thu Jul 22 18:38:06 2021 +0530

    mtrace: Wean away from malloc hooks

    Wean mtrace away from the malloc hooks and move them into the debug
    DSO.  Split the API away from the implementation so that we can add
    the API to libc.so as well as libc_malloc_debug.so, with the libc
    implementations being empty.

    Update localplt data since memalign no longer has any callers after
    this change.

    Reviewed-by: Carlos O'Donell <carlos@redhat.com>
    Tested-by: Carlos O'Donell <carlos@redhat.com>

commit cc35896ea3e4532919ec81b17f36299117debe79
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
Date:   Thu Jul 22 18:38:04 2021 +0530

    Simplify __malloc_initialized

    Now that mcheck no longer needs to check __malloc_initialized (and no
    other third party hook can since the symbol is not exported), make the
    variable boolean and static so that it is used strictly within malloc.

    Reviewed-by: Carlos O'Donell <carlos@redhat.com>
    Tested-by: Carlos O'Donell <carlos@redhat.com>

commit c142eb253f3814f46527e9b37484041dd85702cf
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
Date:   Thu Jul 22 18:38:02 2021 +0530

    mcheck: Wean away from malloc hooks [BZ #23489]

    Split the mcheck implementation into the debugging hooks and API so
    that the API can be replicated in libc and libc_malloc_debug.so.  The
    libc APIs always result in failure.

    The mcheck implementation has also been moved entirely into
    libc_malloc_debug.so and with it, all of the hook initialization code
    can now be moved into the debug library.  Now the initialization can
    be done independently of libc internals.

    With this patch, libc_malloc_debug.so can no longer be used with older
    libcs, which is not its goal anyway.  tst-vfork3 breaks due to this
    since it spawns shell scripts, which in turn execute using the system
    glibc.  Move the test to tests-container so that only the built glibc
    is used.

    This move also fixes bugs in the mcheck version of memalign and
    realloc, thus allowing removal of the tests from tests-mcheck
    exclusion list.

    Reviewed-by: Carlos O'Donell <carlos@redhat.com>
    Tested-by: Carlos O'Donell <carlos@redhat.com>

commit 2d2d9f2b48a943fa556301db532103d09800da4d
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
Date:   Thu Jul 22 18:37:59 2021 +0530

    Move malloc hooks into a compat DSO

    Remove all malloc hook uses from core malloc functions and move it
    into a new library libc_malloc_debug.so.  With this, the hooks now no
    longer have any effect on the core library.

    libc_malloc_debug.so is a malloc interposer that needs to be preloaded
    to get hooks functionality back so that the debugging features that
    depend on the hooks, i.e. malloc-check, mcheck and mtrace work again.
    Without the preloaded DSO these debugging features will be nops.
    These features will be ported away from hooks in subsequent patches.

    Similarly, legacy applications that need hooks functionality need to
    preload libc_malloc_debug.so.

    The symbols exported by libc_malloc_debug.so are maintained at exactly
    the same version as libc.so.

    Finally, static binaries will no longer be able to use malloc
    debugging features since they cannot preload the debugging DSO.

    Reviewed-by: Carlos O'Donell <carlos@redhat.com>
    Tested-by: Carlos O'Donell <carlos@redhat.com>

commit 55a4dd39308951da4b0da84b19e415c2bb451b60
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
Date:   Thu Jul 22 18:37:57 2021 +0530

    Remove __morecore and __default_morecore

    Make the __morecore and __default_morecore symbols compat-only and
    remove their declarations from the API.  Also, include morecore.c
    directly into malloc.c; this should ideally get merged into malloc in
    a future cleanup.

    Reviewed-by: Carlos O'Donell <carlos@redhat.com>
    Tested-by: Carlos O'Donell <carlos@redhat.com>

commit 57b07bede12635bd6d6aa0e488824bb510bbeca4
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
Date:   Thu Jul 22 18:37:54 2021 +0530

    Remove __after_morecore_hook

    Remove __after_morecore_hook from the API and finalize the symbol so
    that it can no longer be used in new applications.  Old applications
    using __after_morecore_hook will find that their hook is no longer
    called.

    Reviewed-by: Carlos O'Donell <carlos@redhat.com>
    Tested-by: Carlos O'Donell <carlos@redhat.com>

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/23328] Remove malloc hooks and ensure related APIs return no data.

[i at maskray dot me]

https://sourceware.org/bugzilla/show_bug.cgi?id=23328

Fangrui Song <i at maskray dot me> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |i at maskray dot me

--- Comment #6 from Fangrui Song <i at maskray dot me> ---
These hook symbols should also be removed from malloc/Versions .
The linked output __free_hook@GLIBC_2.2.5 will be unchanged, but it will do
gold/ld.lld a favor.


In GNU ld, if an object file defines __free_hook@GLIBC_2.2.5 and __free_hook,
and --version-script has `GLIBC_2.2.5 { __free_hook; }`,
the output will be the non-default version __free_hook@GLIBC_2.2.5

This is a dark corner of symbol versioning:
https://www.airs.com/blog/archives/220 Combining Versions
Object files other than malloc/malloc-debug.c are using the unversioned symbol
'__free_hook'.
Their '__free_hook' references and the __free_hook@GLIBC_2.2.5 definition are
combined in a strange/unspecified way.

It'd be good to remove non-default version symbols from the version script to
fix the issue.

---

I report it here because this change is the root cause of `FAIL:
malloc/tst-compathooks-on` when I build glibc with ld.lld 13.0.0
https://sourceware.org/pipermail/libc-alpha/2021-July/129450.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/23328] Remove malloc hooks and ensure related APIs return no data.

[i at maskray dot me]

https://sourceware.org/bugzilla/show_bug.cgi?id=23328

--- Comment #7 from Fangrui Song <i at maskray dot me> ---
(In reply to Fangrui Song from comment #6)
> These hook symbols should also be removed from malloc/Versions .
> The linked output __free_hook@GLIBC_2.2.5 will be unchanged, but it will do
> gold/ld.lld a favor.
> [...]

I added a workaround to ld.lld so that
foo@v1 and (foo or foo@@v1) can be combined: https://reviews.llvm.org/D107235
Good for ld.lld now.

For gold linked libc.so, the dynamic symbol is __free_hook@@GLIBC_2.2.5 because
of PR gold/28196

I think using `.symver *, *, remove` in glibc is the way forward. Created PR
glibc/28197

Once implemented, it can make all linkers happy. (Probably more to GNU ld which
has lots of indirection symbol complexity due to support
https://www.airs.com/blog/archives/220 in a contrived way).

-- 
You are receiving this mail because:
You are on the CC list for the bug.