Re: [PATCH] malloc: Deprecate hook variables, __default_morecore, <mcheck.h>

[Florian Weimer <fweimer@redhat.com>]

On 11/17/2016 03:50 PM, Steve Vormwald wrote:
> On 11/17/2016 08:00 AM, Florian Weimer wrote:
>> In my experience, it is not useful at all for finding the cause of leaks
>> because you only get the address of the immediate caller of malloc,
>> which is often a wrapper.  The address is subject to ASLR as well.
>>
>> Based on the comments in this thread, I'm not sure if people find mtrace
>> theoretically useful, or actually use it. :)
>
> I make use of mtrace extensively when debugging oomkills on Cray
> systems.  Traditional memory leaks are typically not as big a problem as
> heap fragmentation.  Special hardware needs require non-default
> mallopt settings, notably severly restricting use of mmap, so fragmentation
> is a much larger problem for us than for many others.

This is very interesting, thanks.

I've been considering for a while to remove the use of sbrk from the 
allocator, both to simplify it and to take away the possibility for 
attacks to tweak the NON_MAIN_ARENA flag to trick the allocator into 
doing something bad.  Some embedded targets appear to strongly prefer 
the sbrk as well, so maybe this approach is doomed.

Would your configuration have problems if we handed all allocations 
which are larger than 1 MiB or so straight to mmap?

> In my experience,
> mtrace is lightweight enough to not noticeably perturb the heap structure
> over the course of the program, usually allowing me to pinpoint which allocations
> are leading to fragmentation.  Heavier instrumentation, such as valgrind,
> messes with it far too much to isolate which allocations from the application
> are actually causing fragmentation when the instrumentation is not present,
> making them much less useful to me.

DJ's tracer should cover that well because it's designed to alter heap 
layout at all (its own allocations use mmap).

We would also be interested in general patterns of fragmentation you 
have encountered, if any.  Maybe there's something we can do about them.

>> In my proposed NEWS entry, I forgot to mention Address Sanitizer (as was
>> pointed out in the GDB discussion).  It has leak detection capabilities
>> as well:
>>
>> ==7440==ERROR: LeakSanitizer: detected memory leaks
>>
>> Direct leak of 32 byte(s) in 1 object(s) allocated from:
>>     #0 0x7fc52f77097a in malloc (/lib64/libasan.so.2+0x9897a)
>>     #1 0x400703 in main (/tmp/a.out+0x400703)
>>     #2 0x7fc52f33757f in __libc_start_main (/lib64/libc.so.6+0x2057f)
>>
>> And it runs significantly faster than valgrind.
>
> Does it have a noticeable impact on the layout of the heap compared to
> non-instrumented runs?

Eh, yes.  It's a totally different allocator.  So I can see why mtrace 
is attractive.

Thanks,
Florian