[RFC][PATCH 0/4] malloc_info: minor fixes and improvements [BZ #17039]

[RFC][PATCH 0/4] malloc_info: minor fixes and improvements [BZ #17039]

[Ken Milmore]

Now that the FSF have accepted my copyright assigment for glibc, I
thought it might be a good time to resubmit these patches for review.

To recap, the intention is to rectify a couple of problems with the
output of malloc_info(), to make it useful as a more detailed
alternative to mallinfo() and malloc_stats(). Namely:

1. The reported chunk sizes are typically off by a few bytes (BZ#17039)
2. The size(s) of the "top" heap chunk(s) are not accounted for.

There is also a new unit test to check that the respective outputs of
malloc_info and mallinfo agree exactly.

Feedback appreciated.

-Ken.

Re: [RFC][PATCH 0/4] malloc_info: minor fixes and improvements [BZ #17039]

[DJ Delorie]

Ken Milmore <ken.milmore@gmail.com> writes:
> To recap, the intention is to rectify a couple of problems with the
> output of malloc_info(), to make it useful as a more detailed
> alternative to mallinfo() and malloc_stats(). Namely:
>
> 1. The reported chunk sizes are typically off by a few bytes (BZ#17039)
> 2. The size(s) of the "top" heap chunk(s) are not accounted for.
>
> There is also a new unit test to check that the respective outputs of
> malloc_info and mallinfo agree exactly.

These all look OK to me.

Re: [RFC][PATCH 0/4] malloc_info: minor fixes and improvements [BZ #17039]

[Carlos O'Donell]

On 04/30/2018 05:00 PM, Ken Milmore wrote:
> I've been looking into the malloc_info() function as a way of
> examining heap fragmentation of running programs. Unfortunately, it
> has a couple of annoying problems which have both been mentioned
> already in BZ #17039.

Overall I'm excited to see someone looking at this and your changes
look like they go in the right direction.

Have you given a look through the "Contribution Checklist?"
https://sourceware.org/glibc/wiki/Contribution%20checklist

Do you have a copyright assignment with the FSF?

We can actually probably accept your changes without an assignment
because they are ~15 lines, but that's the limit, we would not be
able to accept your totally awesome testcase without some kind of
assignment.

Everything else on the contribution checklist is fairly straight
forward stuff and we can help with that.

The FSF does digital assignments now around the world, so a copyright
assignment should be really quick. I usually recommend a future's assignment
if you are at all interested in continuing to contribute patches here and
there to the project.

http://git.savannah.gnu.org/cgit/gnulib.git/plain/doc/Copyright/request-assign.future

Please feel free to contact me off-list if you have any questions about
how to fill out the form.

If you're wondering why we go to the trouble:
https://www.gnu.org/licenses/why-assign.en.html

-- 
Cheers,
Carlos.

[RFC][PATCH 0/4] malloc_info: minor fixes and improvements [BZ #17039]

[Ken Milmore]

I've been looking into the malloc_info() function as a way of examining heap fragmentation of running programs. Unfortunately, it has a couple of annoying problems which have both been mentioned already in BZ #17039.

Problem 1: Reported free chunk sizes include the flag bits which are stored in the LSBs of the chunk header. This means they are typically off by a byte or two. These small errors get rolled up into the totals which can lead to a significant amount of error on a heap with lots of free chunks. Exactly how much error is difficult to tell because there is also...

Problem 2: Sizes of the top blocks in each arena are completely omitted from the output. This makes it impossible to tell exactly how much of the heap is allocated and how much is free. This can be seen particularly easily by calling malloc_info() before and after a malloc() near the start of a program: Because the allocation typically comes straight from the top chunk, which is not listed in the output, the reported state of the heap doesn't change at all between the two calls.

The upshot of the above is that despite providing quite detailed information on heap fragmentation, the output of malloc_info() is less accurate than that provided by the older APIs mallinfo() and malloc_stats(), and is difficult to reconcile with them.

The following minor patches are an attempt to rectify the above problems. Included is a unit test which verifies exact agreement between the byte and block counts produced by malloc_info() and mallinfo() after exercising the heap for a bit.

There are several possible ways that the "top" chunk could be represented in the output: I have chosen to add a separate total element, with type="top", for each arena. It could instead be rolled into the <sizes> list anonymously as just another free chunk, which would have the advantage of not changing the schema at all but seems to go against the purpose of providing detailed information. Suggestions or comments welcome.