Getting more info about a heap corruption

Getting more info about a heap corruption

[Roger Phillips]

[-- Attachment #1: Type: text/plain, Size: 1029 bytes --]

Greetings,

currently I'm analysing a number of heap corruptions in a large application which usually results in a coredump and one of these malloc_printerr :

https://elixir.bootlin.com/glibc/glibc-2.31/source/malloc/malloc.c#L3744
malloc.c - malloc/malloc.c - Glibc source code (glibc-2.31) - Bootlin<https://elixir.bootlin.com/glibc/glibc-2.31/source/malloc/malloc.c#L3744>
Elixir Cross Referencer - Explore source code in your browser - Particularly useful for the Linux kernel and other low-level projects in C/C++ (bootloaders, C libraries...)
elixir.bootlin.com


Problem is that variables needed for this check like victim, bck and next are optimized out and thus not accessible in gdb. Only av and its contents are accessible. I cannot use asan binaries as it is a production test. Changing environment variables is possible though.

So can you tell me if there is any sort of debugging functionality in glibc-2.31 that will show me those missing variables or narrow down the corrupted chunk?

Regards

Re: Getting more info about a heap corruption

[Florian Weimer]

* Roger Phillips via Libc-help:

> Problem is that variables needed for this check like victim, bck and
> next are optimized out and thus not accessible in gdb. Only av and its
> contents are accessible. I cannot use asan binaries as it is a
> production test. Changing environment variables is possible though.

You could use LD_PRELOAD to install debugging malloc (such as the ASAN
allocator), or use LD_PRELOAD=libc.so.6 with a build of glibc (from the
same sources as the system glibc version) with reduced optimization.

I think there are also GDB scripts out there that traverse the various
heap data structures and report inconsistencies.

Thanks,
Florian

Re: Getting more info about a heap corruption

[Roger Phillips]

[-- Attachment #1: Type: text/plain, Size: 1159 bytes --]

Thanks for the answer! Do you mean just LD_PRELOAD libasan without linking against it?

Regarding less optimized libc, do you know if Ubuntu produces such versions for their packages?

Regards

________________________________
From: Florian Weimer <fweimer@redhat.com>
Sent: Wednesday, June 14, 2023 9:24 AM
To: Roger Phillips via Libc-help <libc-help@sourceware.org>
Cc: Roger Phillips <heidegg@hotmail.com>
Subject: Re: Getting more info about a heap corruption

* Roger Phillips via Libc-help:

> Problem is that variables needed for this check like victim, bck and
> next are optimized out and thus not accessible in gdb. Only av and its
> contents are accessible. I cannot use asan binaries as it is a
> production test. Changing environment variables is possible though.

You could use LD_PRELOAD to install debugging malloc (such as the ASAN
allocator), or use LD_PRELOAD=libc.so.6 with a build of glibc (from the
same sources as the system glibc version) with reduced optimization.

I think there are also GDB scripts out there that traverse the various
heap data structures and report inconsistencies.

Thanks,
Florian

Re: Getting more info about a heap corruption

[Florian Weimer]

* Roger Phillips:

> Thanks for the answer! Do you mean just LD_PRELOAD libasan without
> linking against it?

Yes, I think this will work in at least some cases.  There are other
debugging memory allocators that could be used in a similar way, of
course.

> Regarding less optimized libc, do you know if Ubuntu produces such
> versions for their packages?

I don't think so.  You'd have to rebuild glibc yourself with different
build flags, using their build process, but you don't have to install
the result.  Just copy the libc-2.31.so file and LD_PRELOAD it on the
target system.  Note that you need to rebuild the exact same glibc
version, otherwise it might not work.  And you have to do the rebuild
for each libc6 package update.

Thanks,
Florian

Re: Getting more info about a heap corruption

[Roger Phillips]

[-- Attachment #1: Type: text/plain, Size: 1167 bytes --]

Do you know which tests the asan allocator performs out of the box? Or is it more about variable visbility?

________________________________
From: Florian Weimer <fweimer@redhat.com>
Sent: Wednesday, June 14, 2023 10:17 AM
To: Roger Phillips <heidegg@hotmail.com>
Cc: Roger Phillips via Libc-help <libc-help@sourceware.org>
Subject: Re: Getting more info about a heap corruption

* Roger Phillips:

> Thanks for the answer! Do you mean just LD_PRELOAD libasan without
> linking against it?

Yes, I think this will work in at least some cases.  There are other
debugging memory allocators that could be used in a similar way, of
course.

> Regarding less optimized libc, do you know if Ubuntu produces such
> versions for their packages?

I don't think so.  You'd have to rebuild glibc yourself with different
build flags, using their build process, but you don't have to install
the result.  Just copy the libc-2.31.so file and LD_PRELOAD it on the
target system.  Note that you need to rebuild the exact same glibc
version, otherwise it might not work.  And you have to do the rebuild
for each libc6 package update.

Thanks,
Florian