From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 1477 invoked by alias); 16 Sep 2012 09:44:17 -0000 Received: (qmail 1467 invoked by uid 22791); 16 Sep 2012 09:44:16 -0000 X-SWARE-Spam-Status: No, hits=-3.8 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00,KHOP_THREADED X-Spam-Check-By: sourceware.org Received: from localhost (HELO sourceware.org) (127.0.0.1) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Sun, 16 Sep 2012 09:44:03 +0000 From: "dev at parallels dot com" To: glibc-bugs@sources.redhat.com Subject: [Bug malloc/14581] glibc leaks memory and do not reuse after free (leading to unlimited RSS growth) Date: Sun, 16 Sep 2012 09:44:00 -0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: glibc X-Bugzilla-Component: malloc X-Bugzilla-Keywords: X-Bugzilla-Severity: normal X-Bugzilla-Who: dev at parallels dot com X-Bugzilla-Status: NEW X-Bugzilla-Priority: P2 X-Bugzilla-Assigned-To: unassigned at sourceware dot org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: X-Bugzilla-URL: http://sourceware.org/bugzilla/ Auto-Submitted: auto-generated Content-Type: text/plain; charset="UTF-8" MIME-Version: 1.0 Mailing-List: contact glibc-bugs-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: glibc-bugs-owner@sourceware.org X-SW-Source: 2012-09/txt/msg00134.txt.bz2 http://sourceware.org/bugzilla/show_bug.cgi?id=14581 --- Comment #7 from Kirill Korotaev 2012-09-16 09:44:02 UTC --- (In reply to comment #6) > Could you explain what you mean by "if you print virtual addresses of allocated > objects and kernel VMAs, i.e. you will find a huge unused memory extents which > are never reused by glibc"? I'm not aware of any way the kernel VMA data could > inform you about heap utilization, which is entirely under userspace control. It's very simple. There is RSS and VSZ in /proc/pid/status. RSS tells you how much physical memory was really allocated by kernel. If you add memset() of objects after being allocated you will find that it's really 700MB which corresponds to VSZ as well. i.e. this memory is committed. > I did a simulation of your test loop with much smaller sizes using pen and > paper, and > > With SSIZE=2, ALIGN=8, LSIZE=5, NS=[many], NL=4: > > 3z: SS LLLLLSS LLLLLSS LLLLLSS LLLLL > 4a: SS SS LLLLLSS LLLLLSS LLLLL > 4z: SSSS SS LLLLLSS LLLLLSS LLLLLLLLLL > 5a: SSSS SS SS LLLLLSS LLLLLLLLLL > 5z: SSSSSS SS LLLLLSS LLLLLSS LLLLLLLLLL > 6a: SSSSSS SS LLLLLSS SS LLLLLLLLLL > 6z: SSSSSSSSSS LLLLLSS LLLLLSS LLLLLLLLLL > 7a: SSSSSSSSSS LLLLLSS LLLLLSS LLLLL > 7z: SSSSSSSSSS LLLLLSS LLLLLSSSS LLLLLLLLLL > ... > > where 3z means "at the end of iteration 3" and 4a means "after the free steps > of iteration 4", etc. I might have gotten some details wrong, but it seems this > pattern necessarily enforces fragmentation by destroying the alignment > potential of each LSIZE-sized free range. First 500 iterations are not interesting that much, cause they do not free any previously allocated objects. Have you noticed that array index wraps after NL and NS iterations passed and then most interesting begins? > Obviously there are some allocation strategies that avoid the issue, but I > don't see it being avoidable in dlmalloc-type schemes in general. If you have > an idea for how to avoid it without destroying the good properties of the > allocator strategy, please share. Looks like I start to realize what you mean... Actually, theoretically any allocator should not ever allocate physical RAM more then 2*allocated_size due to fragmentation on pattern like this, right? (it's simple: if you allocated more then 2x times, this means you have unused holes bigger then any single object and could allocate from it...). In our case we see about 10x times ratio... And there are many which behave like that: TCMalloc, buddy etc. What is not natural in this test is that memalign replaced with malloc() fixes the problem... -- Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.