From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 478 invoked by alias); 19 Jan 2010 10:34:20 -0000 Received: (qmail 468 invoked by uid 22791); 19 Jan 2010 10:34:18 -0000 X-SWARE-Spam-Status: No, hits=-1.9 required=5.0 tests=AWL,BAYES_00,SARE_MSGID_LONG40,SPF_PASS X-Spam-Check-By: sourceware.org Received: from mail-iw0-f188.google.com (HELO mail-iw0-f188.google.com) (209.85.223.188) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Tue, 19 Jan 2010 10:34:07 +0000 Received: by iwn26 with SMTP id 26so2678149iwn.14 for ; Tue, 19 Jan 2010 02:34:05 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.154.77 with SMTP id n13mr441922ibw.11.1263897245782; Tue, 19 Jan 2010 02:34:05 -0800 (PST) In-Reply-To: <4B54A0CB.3090301@caviumnetworks.com> References: <4B542EE9.7030503@redhat.com> <4B54A0CB.3090301@caviumnetworks.com> Date: Tue, 19 Jan 2010 10:34:00 -0000 Message-ID: Subject: Re: How to minimize (unshareable) memory usage? From: Justin Santa Barbara To: "java@gcc.gnu.org" Cc: Hans Boehm , Andrew Haley , David Daney Content-Type: text/plain; charset=ISO-8859-1 Mailing-List: contact java-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: java-owner@gcc.gnu.org X-SW-Source: 2010-01/txt/msg00026.txt.bz2 Thanks to everyone for the great suggestions. I've tracked down the mysterious 8MB segments: they are the stacks for my threads (by putting a breakpoint on mmap) Though reserved in virtual memory space, pages shouldn't be allocated unless they're actually touched, so this is good news for physical memory. I verified this by dumping /proc//smaps; the output of one of my 8MB segments shows that only 16KB or so is really used. (Output attached at the end) So the next biggest culprit is my heap memory usage ... I tried using GCInfo.setOOMDump, and did get a dump. But all is not entirely well: my GC_MAXIMUM_HEAP_SIZE was set to 8,000,000 bytes, which results in an out of memory dump, but gc-analyze reports only 5.5MB usage, and of that half is listed under the 'free' column in the "Used Blocks" section of the dump. *** Used Blocks *** Size Kind Blocks Used Free Wasted ------- ------------- ------- ---------- ---------- ------- <...snipped...> ------- ------------- ------- ---------- ---------- ------- 1,352 2,700,672 2,758,624 78,496 Total bytes = 5,537,792 The percentages elsewhere also support the idea that about 3MB of data is actually in use, so I'm not sure exactly why we're failing memory requests when we don't seem to be close to the limit. As for the 6MB of writeable data that shows against libgcj, I believe this is the 'normal' data segment of the shared library. Cutting this down would probably be difficult. The easiest approach for me would be modularizing gcj so I don't have to pay for swing or awt. Again, suggestions are very welcome, but I think that this particular segment will be much more complicated than figuring out why the heap is not being used as efficiently as I hope it could be. Thanks Justin --- The smap output: 8MB of stack, but only 16KB actually used: b4b05000-b5305000 rw-p 00000000 00:00 0 Size: 8192 kB Rss: 16 kB Pss: 16 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 0 kB Private_Dirty: 16 kB Referenced: 16 kB Swap: 0 kB KernelPageSize: 4 kB MMUPageSize: 4 kB The heap segment (with a 10,000,000 byte limit to avoid crashing, and captured on a different machine from the other two): 7f43a12b0000-7f43a1bc0000 rw-p 7f43a12b0000 00:00 0 Size: 9280 kB Rss: 9280 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 0 kB Private_Dirty: 9280 kB Referenced: 9280 kB The data segment of libgcj (?), mostly dirty: b7122000-b7739000 rw-p 01ae0000 08:01 28534016 /usr/local/lib/libgcj.so.10.0.0 Size: 6236 kB Rss: 6236 kB Pss: 6236 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 1644 kB Private_Dirty: 4592 kB Referenced: 6236 kB Swap: 0 kB KernelPageSize: 4 kB MMUPageSize: 4 kB