Re: GC leaks debugging

[Erik Groeneveld <erik@cq2.nl>]

>> The last good dump I have from the test after 12 million cycles (it
>> then got killed) has nothing like File stuff at all.  A also saw other
>> suspicious objects, but they all disappeared later on.  The collecter
>> really works wel!
>>
>> Next hypothesis:
>> From analyzing graphs from the logs and comparing them to those of the
>> OpenJDK, I get the feeling that the collector looses control by not
>> collecting often enough.
>>
>> The heap is quite unused/free, and remains so during the process.  It
>> seems that at some point, the heap fills up very quickly, and then the
>> collector decides to expand the heap instead of collecting (the
>> algorithm for deciding this seems rather complicated).  However, a
>> larger heap also causes the collector to collect less frequently.  So
>> the next time the heap fills up rapidly, it again decides to expand
>> the heap, again causing less frequent collections.  And so on.  I´ll
>> post the graph data in a separate post if you want it.
>
> That makes sense as an explanation.
>
> It looks, then, as though there isn't a leak at all.  The collector
> does what it's supposed to do.  There is always the risk of this with
> any non-compacting dynamic memory allocator.

Yes, a non-moving collector has its limits, but I have still hope for
improvement.

>> And the next hypothesis:
>> Perhaps the program allocates many different (possibly large) sizes,
>> which remain on the free list, but cannot be used because the next
>> objects requested are slightly bigger.  I have to study this somewhat
>> more.
>
> I wonder.
>
>> Just two questions:
>> 1. What is a reasonable number of heap sections?  I have 131 here.
>> 2. What is a reasonable number of free lists?  I have 60, which have
>> 13,000+ entries.

I found from the code that 60 is the upper bound.  And 13,000 seems to
be a lot to me.  Altogether, the free space is about 700 Mb, of a 900
Mb heap.

Hans, is there a limit on the length of the free lists?
Adjacent blocks of the same size are coalesced; is there also
coalescing of blocks of different sizes at some point?

I tried a build with with USE_MUNMAP, hoping that would help the
collector clean up free-list, but I am still testing.

> Paging Hans Boehm.  can you suggest ways to get the system to GC more
> frequently?  Would doing so avoid this scenario?
>
> Andrew.
>