From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 801 invoked by alias); 8 Apr 2011 13:56:06 -0000 Received: (qmail 788 invoked by uid 22791); 8 Apr 2011 13:56:05 -0000 X-SWARE-Spam-Status: No, hits=-6.5 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_HI,SPF_HELO_PASS,T_RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Fri, 08 Apr 2011 13:55:58 +0000 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id p38Dtumf004411 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 8 Apr 2011 09:55:56 -0400 Received: from zebedee.pink (ovpn-113-109.phx2.redhat.com [10.3.113.109]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id p38Dtsrr018514; Fri, 8 Apr 2011 09:55:55 -0400 Message-ID: <4D9F13E9.7000106@redhat.com> Date: Fri, 08 Apr 2011 13:56:00 -0000 From: Andrew Haley User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.15) Gecko/20110307 Fedora/3.1.9-0.39.b3pre.fc14 Thunderbird/3.1.9 MIME-Version: 1.0 To: Erik Groeneveld CC: java@gcc.gnu.org, "Boehm, Hans" Subject: Re: GC leaks debugging References: <4D95909E.4060309@redhat.com> <4D959C24.8030408@redhat.com> <238A96A773B3934685A7269CC8A8D04272EFEFD5C3@GVW0436EXB.americas.hpqcorp.net> <4D997D8C.9060903@redhat.com> <4D9993D4.9040704@redhat.com> <4D9ADA89.8000307@redhat.com> <4D9B112B.6000408@redhat.com> <4D9CB200.2000103@redhat.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-IsSubscribed: yes Mailing-List: contact java-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: java-owner@gcc.gnu.org X-SW-Source: 2011-04/txt/msg00037.txt.bz2 On 04/07/2011 06:42 PM, Erik Groeneveld wrote: > Thanks for trying. > > The last good dump I have from the test after 12 million cycles (it > then got killed) has nothing like File stuff at all. A also saw other > suspicious objects, but they all disappeared later on. The collecter > really works wel! > > See dump extract below (full dump attached). > > What can you suggest from this? > What does (Java) mean? I'm not exactly sure. This will take a bit of digging. > *** Memory Usage Sorted by Total Size *** > > Total Size Count Size Description > -------------- ----- -------- ----------------------------------- > 17% 3,958,024 = 70,679 * 56 - (Java) > 15% 3,426,048 = 71,376 * 48 - GC_PTRFREE > 9% 2,097,152 = 1 * 2,097,152 - GC_NORMAL > 9% 2,085,160 = 7 * 297,880 - [I > 8% 1,908,240 = 79,510 * 24 - (Java) > 6% 1,376,928 = 42 * 32,784 - [C > 5% 1,279,104 = 79,944 * 16 - (Java) > 4% 1,048,592 = 1 * 1,048,592 - [I > 4% 954,480 = 19,885 * 48 - GC_NORMAL > 4% 917,952 = 28 * 32,784 - [B > 2% 642,896 = 2 * 321,448 - [I > 2% 622,896 = 19 * 32,784 - [I > 1% 355,840 = 8,896 * 40 - (Java) > >> At a mad guess, someone is not closing their files but >> hoping that finalization will do it instead. > > It crossed my mind also, but I see no traces of that. > > Next hypothesis: > From analyzing graphs from the logs and comparing them to those of the > OpenJDK, I get the feeling that the collector looses control by not > collecting often enough. > > The heap is quite unused/free, and remains so during the process. It > seems that at some point, the heap fills up very quickly, and then the > collector decides to expand the heap instead of collecting (the > algorithm for deciding this seems rather complicated). However, a > larger heap also causes the collector to collect less frequently. So > the next time the heap fills up rapidly, it again decides to expand > the heap, again causing less frequent collections. And so on. I´ll > post the graph data in a separate post if you want it. That makes sense as an explanation. It looks, then, as though there isn't a leak at all. The collector does what it's supposed to do. There is always the risk of this with any non-compacting dynamic memory allocator. > And the next hypothesis: > Perhaps the program allocates many different (possibly large) sizes, > which remain on the free list, but cannot be used because the next > objects requested are slightly bigger. I have to study this somewhat > more. I wonder. > Just two questions: > 1. What is a reasonable number of heap sections? I have 131 here. > 2. What is a reasonable number of free lists? I have 60, which have > 13,000+ entries. Paging Hans Boehm. can you suggest ways to get the system to GC more frequently? Would doing so avoid this scenario? Andrew.