[PATCH] malloc_stats(): Fix `unsigned int` overflow

[PATCH] malloc_stats(): Fix `unsigned int` overflow

[Niklas Hambüchen]

Fixes malloc_stats() returning vastly wrong information for programs
that use more than 2 GiB memory.

`man mallinfo` documents that it uses `int` and wraps around, but
`man malloc_stats` does not, and should not.

Fixes https://sourceware.org/bugzilla/show_bug.cgi?id=21556

Signed-off-by: Niklas Hambüchen <mail@nh2.me>
---
 malloc/malloc.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/malloc/malloc.c b/malloc/malloc.c
index 095d97a3be..cdc9aff9d5 100644
--- a/malloc/malloc.c
+++ b/malloc/malloc.c
@@ -5151,7 +5151,7 @@ __malloc_stats (void)
 {
   int i;
   mstate ar_ptr;
-  unsigned int in_use_b = mp_.mmapped_mem, system_b = in_use_b;
+  size_t in_use_b = mp_.mmapped_mem, system_b = in_use_b;
 
   if (!__malloc_initialized)
     ptmalloc_init ();
@@ -5166,8 +5166,8 @@ __malloc_stats (void)
       __libc_lock_lock (ar_ptr->mutex);
       int_mallinfo (ar_ptr, &mi);
       fprintf (stderr, "Arena %d:\n", i);
-      fprintf (stderr, "system bytes     = %10u\n", (unsigned int) mi.arena);
-      fprintf (stderr, "in use bytes     = %10u\n", (unsigned int) mi.uordblks);
+      fprintf (stderr, "system bytes     = %10lu\n", mi.arena);
+      fprintf (stderr, "in use bytes     = %10lu\n", mi.uordblks);
 #if MALLOC_DEBUG > 1
       if (i > 0)
         dump_heap (heap_for_ptr (top (ar_ptr)));
@@ -5180,9 +5180,9 @@ __malloc_stats (void)
         break;
     }
   fprintf (stderr, "Total (incl. mmap):\n");
-  fprintf (stderr, "system bytes     = %10u\n", system_b);
-  fprintf (stderr, "in use bytes     = %10u\n", in_use_b);
-  fprintf (stderr, "max mmap regions = %10u\n", (unsigned int) mp_.max_n_mmaps);
+  fprintf (stderr, "system bytes     = %10lu\n", system_b);
+  fprintf (stderr, "in use bytes     = %10lu\n", in_use_b);
+  fprintf (stderr, "max mmap regions = %10d\n", mp_.max_n_mmaps);
   fprintf (stderr, "max mmap bytes   = %10lu\n",
            (unsigned long) mp_.max_mmapped_mem);
   stderr->_flags2 = old_flags2;
-- 
2.31.1

Re: [PATCH] malloc_stats(): Fix `unsigned int` overflow

[DJ Delorie]

Fails 32-bit build:

https://www.delorie.com/trybots/32bit/4291/make.tail.txt

Re: [PATCH] malloc_stats(): Fix `unsigned int` overflow

[Niklas Hambüchen]

The error:

> malloc.c:5169:48: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'size_t' {aka 'unsigned int'} [-Werror=format=]

glibc doesn't require C99, so I guess "%zu" is not an option, right?

Since this is just in an `fprintf`, would casting to `(unsigned long long)` and using "%llu" be considered acceptable?

Niklas

Re: [PATCH] malloc_stats(): Fix `unsigned int` overflow

[Siddhesh Poyarekar]

On 10/30/21 06:06, Niklas Hambüchen via Libc-alpha wrote:
> The error:
> 
>> malloc.c:5169:48: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'size_t' {aka 'unsigned int'} [-Werror=format=]
> 
> glibc doesn't require C99, so I guess "%zu" is not an option, right?

%zu is already being used in the file you're changing, so it's perfectly 
fine to use.

Siddhesh

Re: [PATCH] malloc_stats(): Fix `unsigned int` overflow

[Andreas Schwab]

On Okt 30 2021, Niklas Hambüchen via Libc-alpha wrote:

> glibc doesn't require C99, so I guess "%zu" is not an option, right?

glibc _implements_ C99, so you can use all its features.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: [PATCH] malloc_stats(): Fix `unsigned int` overflow

[Adhemerval Zanella]

On 29/10/2021 21:36, Niklas Hambüchen via Libc-alpha wrote:
> The error:
> 
>> malloc.c:5169:48: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'size_t' {aka 'unsigned int'} [-Werror=format=]
> 
> glibc doesn't require C99, so I guess "%zu" is not an option, right?

We build glibc with -std=gnu11, so we do require a compile that at least support
C11.

> 
> Since this is just in an `fprintf`, would casting to `(unsigned long long)` and using "%llu" be considered acceptable?

Either way would be fine.

[PATCH v2] malloc_stats(): Fix `unsigned int` overflow

[Niklas Hambüchen]

Fixes malloc_stats() returning vastly wrong information for programs
that use more than 2 GiB memory.

`man mallinfo` documents that it uses `int` and wraps around, but
`man malloc_stats` does not, and should not.

Fixes https://sourceware.org/bugzilla/show_bug.cgi?id=21556

Signed-off-by: Niklas Hambüchen <mail@nh2.me>
---
 malloc/malloc.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/malloc/malloc.c b/malloc/malloc.c
index 78a531bc7a..dfb266a226 100644
--- a/malloc/malloc.c
+++ b/malloc/malloc.c
@@ -5374,7 +5374,7 @@ __malloc_stats (void)
 {
   int i;
   mstate ar_ptr;
-  unsigned int in_use_b = mp_.mmapped_mem, system_b = in_use_b;
+  size_t in_use_b = mp_.mmapped_mem, system_b = in_use_b;
 
   if (!__malloc_initialized)
     ptmalloc_init ();
@@ -5389,8 +5389,8 @@ __malloc_stats (void)
       __libc_lock_lock (ar_ptr->mutex);
       int_mallinfo (ar_ptr, &mi);
       fprintf (stderr, "Arena %d:\n", i);
-      fprintf (stderr, "system bytes     = %10u\n", (unsigned int) mi.arena);
-      fprintf (stderr, "in use bytes     = %10u\n", (unsigned int) mi.uordblks);
+      fprintf (stderr, "system bytes     = %10zu\n", mi.arena);
+      fprintf (stderr, "in use bytes     = %10zu\n", mi.uordblks);
 #if MALLOC_DEBUG > 1
       if (i > 0)
         dump_heap (heap_for_ptr (top (ar_ptr)));
@@ -5403,11 +5403,10 @@ __malloc_stats (void)
         break;
     }
   fprintf (stderr, "Total (incl. mmap):\n");
-  fprintf (stderr, "system bytes     = %10u\n", system_b);
-  fprintf (stderr, "in use bytes     = %10u\n", in_use_b);
-  fprintf (stderr, "max mmap regions = %10u\n", (unsigned int) mp_.max_n_mmaps);
-  fprintf (stderr, "max mmap bytes   = %10lu\n",
-           (unsigned long) mp_.max_mmapped_mem);
+  fprintf (stderr, "system bytes     = %10zu\n", system_b);
+  fprintf (stderr, "in use bytes     = %10zu\n", in_use_b);
+  fprintf (stderr, "max mmap regions = %10d\n", mp_.max_n_mmaps);
+  fprintf (stderr, "max mmap bytes   = %10zu\n", mp_.max_mmapped_mem);
   stderr->_flags2 = old_flags2;
   _IO_funlockfile (stderr);
 }
-- 
2.40.1

Re: [PATCH v2] malloc_stats(): Fix `unsigned int` overflow

[DJ Delorie]

Niklas Hambchen <mail@nh2.me> writes:
> @@ -5374,7 +5374,7 @@ __malloc_stats (void)
>  {
>    int i;
>    mstate ar_ptr;
> -  unsigned int in_use_b = mp_.mmapped_mem, system_b = in_use_b;
> +  size_t in_use_b = mp_.mmapped_mem, system_b = in_use_b;

Ok.

> @@ -5389,8 +5389,8 @@ __malloc_stats (void)
>        __libc_lock_lock (ar_ptr->mutex);
>        int_mallinfo (ar_ptr, &mi);
>        fprintf (stderr, "Arena %d:\n", i);
> -      fprintf (stderr, "system bytes     = %10u\n", (unsigned int) mi.arena);
> -      fprintf (stderr, "in use bytes     = %10u\n", (unsigned int) mi.uordblks);
> +      fprintf (stderr, "system bytes     = %10zu\n", mi.arena);
> +      fprintf (stderr, "in use bytes     = %10zu\n", mi.uordblks);

Ok.

> -  fprintf (stderr, "system bytes     = %10u\n", system_b);
> -  fprintf (stderr, "in use bytes     = %10u\n", in_use_b);
> -  fprintf (stderr, "max mmap bytes   = %10lu\n",
> -           (unsigned long) mp_.max_mmapped_mem);
> +  fprintf (stderr, "system bytes     = %10zu\n", system_b);
> +  fprintf (stderr, "in use bytes     = %10zu\n", in_use_b);
> +  fprintf (stderr, "max mmap bytes   = %10zu\n", mp_.max_mmapped_mem);

This needs a cast to (size_t) as max_mmapped_mem is INTERNAL_SIZE_T,
which may be smaller than size_t.

> -  fprintf (stderr, "max mmap regions = %10u\n", (unsigned int) mp_.max_n_mmaps);
> +  fprintf (stderr, "max mmap regions = %10d\n", mp_.max_n_mmaps);

This causes a negative number to be printed when max_n_mmaps increments
past 2^31, where using "u" defers bad results until 2^32.  So IMHO this
one is a regression, not a fix.

Re: [PATCH v2] malloc_stats(): Fix `unsigned int` overflow

[Niklas Hambüchen]

> This needs a cast to (size_t) as max_mmapped_mem is INTERNAL_SIZE_T,
> which may be smaller than size_t.

Thanks, will fix!

>> +  fprintf (stderr, "max mmap regions = %10d\n", mp_.max_n_mmaps);
> 
> This causes a negative number to be printed when max_n_mmaps increments
> past 2^31, where using "u" defers bad results until 2^32.  So IMHO this
> one is a regression, not a fix.

I'm not sure I understand:
That statement holds for all `int`s, so following this logic, all `int`s should be casted to `unsigned int` first and printed with `%u`, and `%d` would be quite useless.

Doesn't it make more sense to print an `int` with `%d` as normal, so that you can _see_ in `malloc_stats()` when it overflows?

The non-`malloc_stats()` code that uses the `int` will likely also behave strangely if this overflow happens, so it seems weird to pretend the value is of a different type in the malloc-internals inspection function.

(Besides, `max_n_mmaps` sounds like a count of mmaps, not a byte count, so it should be much less likely to overflow.)

Thanks!

Re: [PATCH v2] malloc_stats(): Fix `unsigned int` overflow

[DJ Delorie]

Niklas Hambchen <mail@nh2.me> writes:
>>> +  fprintf (stderr, "max mmap regions = %10d\n", mp_.max_n_mmaps);
>> 
>> This causes a negative number to be printed when max_n_mmaps increments
>> past 2^31, where using "u" defers bad results until 2^32.  So IMHO this
>> one is a regression, not a fix.
>
> I'm not sure I understand:
> That statement holds for all `int`s, so following this logic, all
> `int`s should be casted to `unsigned int` first and printed with `%u`,
> and `%d` would be quite useless.
>
> Doesn't it make more sense to print an `int` with `%d` as normal, so
> that you can _see_ in `malloc_stats()` when it overflows?

It only really overflows when it wraps around 2^32.  Until then, the
value is still useful, as long as you present it properly.

An alternative would be to print out "(overflowed)" if the value exceeds 2^31

It just didn't seem right to take something that prints 0..2^32-1 and
reduce it to only print 0..2^31-1.

The other cases were stored as size_t and potentially truncated, which
is a different sort of overflow bug.