From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 113029 invoked by alias); 18 Nov 2016 09:13:44 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Received: (qmail 113006 invoked by uid 89); 18 Nov 2016 09:13:43 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-4.8 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD,SPF_HELO_PASS autolearn=ham version=3.3.2 spammy=attacks X-HELO: mx1.redhat.com Subject: Re: [PATCH] malloc: Deprecate hook variables, __default_morecore, To: Steve Vormwald , Joseph Myers References: <20161026150218.3F1A4439942E0@oldenburg.str.redhat.com> <11f59117-e5a4-4c62-739c-e05e01ee43b3@redhat.com> <1a524603-2966-df5a-edd9-8045aa256837@redhat.com> Cc: "libc-alpha@sourceware.org" From: Florian Weimer Message-ID: <915065cd-079b-0f7b-eebf-7ca835ca4030@redhat.com> Date: Fri, 18 Nov 2016 09:13:00 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-SW-Source: 2016-11/txt/msg00649.txt.bz2 On 11/17/2016 03:50 PM, Steve Vormwald wrote: > On 11/17/2016 08:00 AM, Florian Weimer wrote: >> In my experience, it is not useful at all for finding the cause of leaks >> because you only get the address of the immediate caller of malloc, >> which is often a wrapper. The address is subject to ASLR as well. >> >> Based on the comments in this thread, I'm not sure if people find mtrace >> theoretically useful, or actually use it. :) > > I make use of mtrace extensively when debugging oomkills on Cray > systems. Traditional memory leaks are typically not as big a problem as > heap fragmentation. Special hardware needs require non-default > mallopt settings, notably severly restricting use of mmap, so fragmentation > is a much larger problem for us than for many others. This is very interesting, thanks. I've been considering for a while to remove the use of sbrk from the allocator, both to simplify it and to take away the possibility for attacks to tweak the NON_MAIN_ARENA flag to trick the allocator into doing something bad. Some embedded targets appear to strongly prefer the sbrk as well, so maybe this approach is doomed. Would your configuration have problems if we handed all allocations which are larger than 1 MiB or so straight to mmap? > In my experience, > mtrace is lightweight enough to not noticeably perturb the heap structure > over the course of the program, usually allowing me to pinpoint which allocations > are leading to fragmentation. Heavier instrumentation, such as valgrind, > messes with it far too much to isolate which allocations from the application > are actually causing fragmentation when the instrumentation is not present, > making them much less useful to me. DJ's tracer should cover that well because it's designed to alter heap layout at all (its own allocations use mmap). We would also be interested in general patterns of fragmentation you have encountered, if any. Maybe there's something we can do about them. >> In my proposed NEWS entry, I forgot to mention Address Sanitizer (as was >> pointed out in the GDB discussion). It has leak detection capabilities >> as well: >> >> ==7440==ERROR: LeakSanitizer: detected memory leaks >> >> Direct leak of 32 byte(s) in 1 object(s) allocated from: >> #0 0x7fc52f77097a in malloc (/lib64/libasan.so.2+0x9897a) >> #1 0x400703 in main (/tmp/a.out+0x400703) >> #2 0x7fc52f33757f in __libc_start_main (/lib64/libc.so.6+0x2057f) >> >> And it runs significantly faster than valgrind. > > Does it have a noticeable impact on the layout of the heap compared to > non-instrumented runs? Eh, yes. It's a totally different allocator. So I can see why mtrace is attractive. Thanks, Florian