[Bug malloc/20646] sysmalloc incorrectly fails with custom morecore function

[Bug malloc/20646] sysmalloc incorrectly fails with custom morecore function

[siddhesh at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=20646

Siddhesh Poyarekar <siddhesh at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |siddhesh at sourceware dot org
           Assignee|unassigned at sourceware dot org   |siddhesh at sourceware dot org

--- Comment #5 from Siddhesh Poyarekar <siddhesh at sourceware dot org> ---
Support for custom morecore functions is going to be removed in 2.34.  Please
use malloc interposition to do this now.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/20646] sysmalloc incorrectly fails with custom morecore function

[siddhesh at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=20646

Siddhesh Poyarekar <siddhesh at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |WONTFIX

--- Comment #6 from Siddhesh Poyarekar <siddhesh at sourceware dot org> ---
Oops, I missed updating bug status with my last comment.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/20646] sysmalloc incorrectly fails with custom morecore function

[guillaume at morinfr dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=20646

--- Comment #7 from Guillaume Morin <guillaume at morinfr dot org> ---
Removing morecore will break all users of libhugetlbfs. There is no other
existing solution to get hugetlb pages with glibc. What are you expecting users
to do?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/20646] sysmalloc incorrectly fails with custom morecore function

[siddhesh at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=20646

--- Comment #8 from Siddhesh Poyarekar <siddhesh at sourceware dot org> ---
Malloc interposing (i.e. preloading a custom malloc implementation DSO that
allocates memory using hugetlb pages) is the right way to do this because these
hooks in malloc have been a long time security hazard.  Also as you've
discovered, support for custom morecore is also broken in subtle ways, which
makes it quite hard to maintain within glibc.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/20646] sysmalloc incorrectly fails with custom morecore function

[guillaume at morinfr dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=20646

--- Comment #9 from Guillaume Morin <guillaume at morinfr dot org> ---
That is not an equivalent solution though. What you're effectively saying now
that all libhugetlbfs users need to find, test and validate an entire malloc
implementation that suits them to use hugetlb (i.e we can't use glibc's malloc
and we can't use libhugetblfs). Or are you expecting all existing users to
write their own custom malloc implementation?

You're mentioning hooking in morecore() is a security hazard. But from a
security point of view, wouldn't it just as good to support morecore()
interposition instead of the entire malloc implementation? And this would give
current users a rather easy fix and a documented interface (unless I am missing
anything).

I must say I find your other point about being subtly broken is a little hard
to digest. I put together a reproducer and a simple fix for this issue almost 5
years ago... Yes, things will be broken if nothing is done to fix them :-) 

Though this is the only issue with morecore() that I know of (and can only be
reached if the morecore implementation allows trimming). Keep in mind that
there are a lot of libhugetlbfs users and all of them are using morecore().
We've been using libhugetblfs in production for a long time (10+ years) without
any issues. 

Are there some unfixable issues with the morecore() scheme that we never
encountered besides the (understandable) security concerns?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/20646] sysmalloc incorrectly fails with custom morecore function

[adhemerval.zanella at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=20646

Adhemerval Zanella <adhemerval.zanella at linaro dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |adhemerval.zanella at linaro dot o
                   |                            |rg

--- Comment #10 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
To give libhugetlsfs users a way forward, I think we should move the huge pages
morecore functionatily to a tunable [1].  The __morecore returning
non-contiguous memory is not really well tests and have corner cases
specifically with memory trimming (specially because we set MORECORE_CONTIGUOUS
as default).

The approach I did [1] was that same as used for aarch64 memory tagging, which
forces __glibc_morecore to always fails and makes sysmalloc fallbacks to
mmap().  I still need to check if all corner cases are handled correclty
(specially trimming).

[1] https://patchwork.sourceware.org/project/glibc/list/?series=2732
[2]
https://patchwork.sourceware.org/project/glibc/patch/20210830185215.449572-8-adhemerval.zanella@linaro.org/

-- 
You are receiving this mail because:
You are on the CC list for the bug.