[Bug malloc/29709] New: malloc: inappropriate allocation retry when exceeding sane request sizes

[Bug malloc/29709] New: malloc: inappropriate allocation retry when exceeding sane request sizes

[milod2048 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29709

            Bug ID: 29709
           Summary: malloc: inappropriate allocation retry when exceeding
                    sane request sizes
           Product: glibc
           Version: 2.36
            Status: UNCONFIRMED
          Severity: minor
          Priority: P2
         Component: malloc
          Assignee: unassigned at sourceware dot org
          Reporter: milod2048 at gmail dot com
  Target Milestone: ---

version:           master 10/17/2022 02:00 PM (MESZ)
h/b/t:             x86_64-pc-linux-gnu
configure options: --prefix=$(pwd)/install/ --disable-experimental-malloc
kernel:            Linux cbox 5.19.14-200.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC
Wed Oct 5
                   21:31:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
gcc version:       gcc version 12.2.1 20220819 (Red Hat 12.2.1-2) (GCC)
ld version:        GNU ld version 2.37-36.fc36

A rather minor issue, but consider following scenario: A multithreaded process
is linked against a build of glibc without tcache support
(--disable-experimental-malloc) and makes a request for an invalid amount of
memory (i.e. greater than PTRDIFF_MAX) to __libc_malloc.

This would result in a failure within _int_malloc due to the sanity checks
implemented in checked_request2size (req > PTRDIFF_MAX) and _int_malloc would
return NULL. This in turn leads to a second allocation attempt within
__libc_malloc, but this time with a different arena, which does not make much
sense. 

A retry with a different arena wont result in a successful allocation since
previous failure was not caused by the arena but the invalid argument to
__libc_malloc. And an invalid request will never be serviced by malloc,
therefore we can skip the call to arena_get_retry plus the second allocation
attempt.

This behavior can also be found in the __libc_memalign function.

Proposal: Differentiate between allocation failure due to actual memory
shortage and allocation failure due to an invalid memory request. A second
attempt should only be made if the request size is reasonable. This would
prevent malloc from unnecessarily switching/creating arenas.

I have already prepared a patch which should fix this minor issue.

-- 
You are receiving this mail because:
You are on the CC list for the bug.