* [Bug libstdc++/53169] Memory leak in std::vector<std::vector*>
2012-04-30 14:06 [Bug libstdc++/53169] New: Memory leak in std::vector<std::vector*> antoinep92 at gmail dot com
@ 2012-04-30 14:28 ` redi at gcc dot gnu.org
2012-04-30 14:59 ` redi at gcc dot gnu.org
` (5 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: redi at gcc dot gnu.org @ 2012-04-30 14:28 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53169
--- Comment #1 from Jonathan Wakely <redi at gcc dot gnu.org> 2012-04-30 14:28:11 UTC ---
(In reply to comment #0)
> The attached source is a minimal test case, implementing a sparse array of
> std::vectors in class Collection, and test() demonstrates its use in a memory
> neutral way (all allocated objects are freed).
Unless you call Collection::at(n) twice for the same n, in which case you leak
raw[n]
> When compiled on x86-64 linux with gcc 4.6.1, gcc 4.7.1 and clang 3.0 (using
> GNU libstdc++), tools such as top show that memory increases when running
> test(), but does not not decrease after the function exits:
That's how GNU/Linux works. A process' memory will not decrease, but that
doesn't mean it's leaked.
> 500Mb are lost in
> this test case. Just increase to loop count and make that 4Gb if you wish: the
> amount of leaked memory don't seem to be bounded.
> `valgrind --leak-check=full ./a.out` reports there is not a single byte leaked,
> which I double checked with the heap profiler from google perf tools.
Indeed. No memory is leaked. The memory is returned to the heap and will be
available for further allocations.
> The memory is reserved by libstdc++ and unavailable to other processes or
It's virtual memory, if you delete it then it's available to other processes,
even if your process' memory usage doesn't appear to have decreased according
to top.
> subsequent malloc/frees within the same program. Subsequent C++ STL allocations
> (e.g. resizing a big vector) on the other hand don't register on process memory
> and seem to ruse the reserved buffers; sometimes they even trigger deallocation
> of the "leaked" memory.
In the default configuration libstdc++ just uses new/delete which uses malloc,
i.e. there's no memory caching in libstdc++ allocators
You can see this for yourself in include/ext/new_allocator.h
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Bug libstdc++/53169] Memory leak in std::vector<std::vector*>
2012-04-30 14:06 [Bug libstdc++/53169] New: Memory leak in std::vector<std::vector*> antoinep92 at gmail dot com
2012-04-30 14:28 ` [Bug libstdc++/53169] " redi at gcc dot gnu.org
@ 2012-04-30 14:59 ` redi at gcc dot gnu.org
2012-04-30 15:13 ` redi at gcc dot gnu.org
` (4 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: redi at gcc dot gnu.org @ 2012-04-30 14:59 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53169
--- Comment #2 from Jonathan Wakely <redi at gcc dot gnu.org> 2012-04-30 14:59:05 UTC ---
By changing your main to:
int main() {
test();
sleep(10);
char* p = (char*)malloc(1024 * 127);
for (int i=0; i < 100; ++i)
p[1024 * i] = 'a' + (i%26);
sleep(10);
free(p);
sleep(10);
return 0;
}
I see that freeing the small malloc'd memory region (which is below glibc's
MMAP_THRESHOLD value) does actually trigger the earlier new'd memory to be
returned to the system too.
So it's possible to get the memory libstdc+ allocates to be returned to the
system, but it's under the control of glibc, nothing to do with std::vector or
libstdc++
If your memory usage patterns don't result in memory being returned then there
are several posibilities, including not freeing memory when you think you are,
or fragmenting the heap so that later allocations cannot re-use memory returned
to freelists and must allocate new memory using mmap.
Not a GCC bug though.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Bug libstdc++/53169] Memory leak in std::vector<std::vector*>
2012-04-30 14:06 [Bug libstdc++/53169] New: Memory leak in std::vector<std::vector*> antoinep92 at gmail dot com
2012-04-30 14:28 ` [Bug libstdc++/53169] " redi at gcc dot gnu.org
2012-04-30 14:59 ` redi at gcc dot gnu.org
@ 2012-04-30 15:13 ` redi at gcc dot gnu.org
2012-04-30 15:16 ` redi at gcc dot gnu.org
` (3 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: redi at gcc dot gnu.org @ 2012-04-30 15:13 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53169
--- Comment #3 from Jonathan Wakely <redi at gcc dot gnu.org> 2012-04-30 15:13:07 UTC ---
Similarly, calling malloc_trim(0) after test() causes glibc to immediately
return the memory to the system (requiring a sbrk system call next time memory
is needed)
You should probably use sbrk and/or experiment with
http://www.gnu.org/software/libc/manual/html_node/Statistics-of-Malloc.html to
see what's actually happening in your program, and maybe using mallopt() to
adjust things.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Bug libstdc++/53169] Memory leak in std::vector<std::vector*>
2012-04-30 14:06 [Bug libstdc++/53169] New: Memory leak in std::vector<std::vector*> antoinep92 at gmail dot com
` (2 preceding siblings ...)
2012-04-30 15:13 ` redi at gcc dot gnu.org
@ 2012-04-30 15:16 ` redi at gcc dot gnu.org
2012-04-30 15:27 ` antoinep92 at gmail dot com
` (2 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: redi at gcc dot gnu.org @ 2012-04-30 15:16 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53169
--- Comment #4 from Jonathan Wakely <redi at gcc dot gnu.org> 2012-04-30 15:16:14 UTC ---
(In reply to comment #3)
> You should probably use sbrk and/or experiment with
Sorry, I meant use strace, *not* sbrk!
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Bug libstdc++/53169] Memory leak in std::vector<std::vector*>
2012-04-30 14:06 [Bug libstdc++/53169] New: Memory leak in std::vector<std::vector*> antoinep92 at gmail dot com
` (3 preceding siblings ...)
2012-04-30 15:16 ` redi at gcc dot gnu.org
@ 2012-04-30 15:27 ` antoinep92 at gmail dot com
2012-04-30 15:31 ` antoinep92 at gmail dot com
2012-04-30 15:52 ` redi at gcc dot gnu.org
6 siblings, 0 replies; 8+ messages in thread
From: antoinep92 at gmail dot com @ 2012-04-30 15:27 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53169
--- Comment #5 from Antoine Poliakov <antoinep92 at gmail dot com> 2012-04-30 15:27:16 UTC ---
(In reply to comment #2)
> So it's possible to get the memory libstdc+ allocates to be returned to the
> system, but it's under the control of glibc, nothing to do with std::vector or
> libstdc++
>
> If your memory usage patterns don't result in memory being returned then there
> are several posibilities, including not freeing memory when you think you are,
> or fragmenting the heap so that later allocations cannot re-use memory returned
> to freelists and must allocate new memory using mmap.
>
> Not a GCC bug though.
Thanks a lot for explaining this, and sorry for the invalid bug report.
There is probably a lot of memory fragmentation going on, but what's bothering
me is that if you launch several processes with { test(); sleep(30); }, waiting
each time for test() to return you will eventually begin to swap and go out of
memory. So the freed memory is not exactly available to other processes: that's
not just a counter problem.
Thanks for your feedback, and the <MMAP_THRESHOLD trick. I'm probably too harsh
on the system allocator and I will probably use a custom allocator using a
memory pool, which will allow me to minimize fragmentation and free the memory
in a predictable way.
I only see unconfirmed and resolved status, not invalid. Can you mark the bug
as invalid, please.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Bug libstdc++/53169] Memory leak in std::vector<std::vector*>
2012-04-30 14:06 [Bug libstdc++/53169] New: Memory leak in std::vector<std::vector*> antoinep92 at gmail dot com
` (4 preceding siblings ...)
2012-04-30 15:27 ` antoinep92 at gmail dot com
@ 2012-04-30 15:31 ` antoinep92 at gmail dot com
2012-04-30 15:52 ` redi at gcc dot gnu.org
6 siblings, 0 replies; 8+ messages in thread
From: antoinep92 at gmail dot com @ 2012-04-30 15:31 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53169
--- Comment #6 from Antoine Poliakov <antoinep92 at gmail dot com> 2012-04-30 15:31:17 UTC ---
(In reply to comment #4)
> (In reply to comment #3)
> > You should probably use sbrk and/or experiment with
>
> Sorry, I meant use strace, *not* sbrk!
malloc_trim(0) saved the day!
Thanks again
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Bug libstdc++/53169] Memory leak in std::vector<std::vector*>
2012-04-30 14:06 [Bug libstdc++/53169] New: Memory leak in std::vector<std::vector*> antoinep92 at gmail dot com
` (5 preceding siblings ...)
2012-04-30 15:31 ` antoinep92 at gmail dot com
@ 2012-04-30 15:52 ` redi at gcc dot gnu.org
6 siblings, 0 replies; 8+ messages in thread
From: redi at gcc dot gnu.org @ 2012-04-30 15:52 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53169
Jonathan Wakely <redi at gcc dot gnu.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |RESOLVED
Resolution| |INVALID
--- Comment #7 from Jonathan Wakely <redi at gcc dot gnu.org> 2012-04-30 15:51:57 UTC ---
(In reply to comment #5)
> I only see unconfirmed and resolved status, not invalid. Can you mark the bug
> as invalid, please.
As the reporter you could have done so yourself. Done now though.
^ permalink raw reply [flat|nested] 8+ messages in thread