From: liudongyun <liu.dongyun@h3c.com>
To: <libc-alpha@sourceware.org>
Cc: <libc-maintainers@gnu.org>, <libc-locales@sourceware.org>,
<deng.hongjie@h3c.com>, <liu.dongyun@h3c.com>,
<dai.xianjun@h3c.com>
Subject: [PATCH] release all idle heaps in the non-main-arena
Date: Fri, 11 Jun 2021 12:08:52 +0800 [thread overview]
Message-ID: <20210611040852.21745-1-liu.dongyun@h3c.com> (raw)
liu.dongyun@h3c.com report this question in follow:
The system has a total of 32G memory and 12 core cpu.
After frequent malloc and free memory by multiple threads, we found that many
physical memory that should have been released have not been released.We observe that
the maximum free memory can reach 15G.
deng.hongjie@h3c.com recommend reducing the heap size of the non-main-arena, but it
doesn't work. liu.dongyun@h3c.com found When the last heap in the user process into
the non-main-area has memory in use, the previous heap will not be released, even if the
previous heap is already free. This mechanism resulting in the memory cache reaching 15G.
liu.dongyun@h3c.com tried to solve this problem by checking to release the heap that the
current chunk belongs to and unmmaped if it is all free. Then we adjust the heap size of
the non-main-arena to 512KB. The method has achieved good results. Submit patch only to
modify one of them.
The test after modification has the following results.
With business:
--------------------------------------------------------------------------------
Memory statistics are measured in KB:
Slot 0:
Total Used Free Shared Buffers Cached FreeRatio
Mem: 32598612 7501484 25097128 0 1812 2121380 77.0%
--------------------------------------------------------------------------------
A few minutes after deleted business:
Memory statistics are measured in KB:
Slot 0:
Total Used Free Shared Buffers Cached FreeRatio
Mem: 32598612 9763528 22835084 0 32 2118208 70.0%
--------------------------------------------------------------------------------
Bugzilla: https://sourceware.org/bugzilla/show_bug.cgi?id=27976
Reported-by: liudongyun <liu.dongyun@h3c.com> and
Reported-by: denghongjie <deng.hongjie@h3c.com>
---
malloc/arena.c | 30 +++++++++++++++++++++++++++++-
malloc/malloc.c | 2 +-
2 files changed, 30 insertions(+), 2 deletions(-)
mode change 100644 => 100755 malloc/arena.c
mode change 100644 => 100755 malloc/malloc.c
diff --git a/malloc/arena.c b/malloc/arena.c
old mode 100644
new mode 100755
index 7eb110445e..7812997886
--- a/malloc/arena.c
+++ b/malloc/arena.c
@@ -606,14 +606,42 @@ shrink_heap (heap_info *h, long diff)
} while (0)
static int
-heap_trim (heap_info *heap, size_t pad)
+heap_trim(heap_info *heap, heap_info *free_heap, size_t pad)
{
mstate ar_ptr = heap->ar_ptr;
unsigned long pagesz = GLRO (dl_pagesize);
mchunkptr top_chunk = top (ar_ptr), p;
heap_info *prev_heap;
long new_size, top_size, top_area, extra, prev_size, misalign;
+ heap_info *last_heap;
+ /*release part of the cache memory*/
+ last_heap = heap_for_ptr(top_chunk);
+ if ((NULL != free_heap->prev) && (last_heap != free_heap))
+ {
+ p = chunk_at_offset(free_heap, sizeof(*heap));
+ if (!inuse(p))
+ {
+ if (chunksize(p) + sizeof(*free_heap) + MINSIZE == free_heap->size)
+ {
+ while(last_heap)
+ {
+ if (last_heap->prev == free_heap)
+ {
+ last_heap->prev = free_heap->prev;
+ break;
+ }
+ last_heap = last_heap->prev;
+ }
+ ar_ptr->system_mem -= free_heap->size;
+ arena_mem -= free_heap->size;
+ unlink_chunk(ar_ptr, p);
+ delete_heap(free_heap);
+ return 1;
+ }
+ }
+ }
+
/* Can this heap go away completely? */
while (top_chunk == chunk_at_offset (heap, sizeof (*heap)))
{
diff --git a/malloc/malloc.c b/malloc/malloc.c
old mode 100644
new mode 100755
index 0e2e1747e0..40f0a8e458
--- a/malloc/malloc.c
+++ b/malloc/malloc.c
@@ -4665,7 +4665,7 @@ _int_free (mstate av, mchunkptr p, int have_lock)
heap_info *heap = heap_for_ptr(top(av));
assert(heap->ar_ptr == av);
- heap_trim(heap, mp_.top_pad);
+ heap_trim(heap, heap_for_ptr(p), mp_.top_pad);
}
}
--
2.17.1
next reply other threads:[~2021-06-11 4:10 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-11 4:08 liudongyun [this message]
2021-06-11 6:41 ` Siddhesh Poyarekar
2021-06-18 8:02 ` 答复: " Liudongyun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210611040852.21745-1-liu.dongyun@h3c.com \
--to=liu.dongyun@h3c.com \
--cc=dai.xianjun@h3c.com \
--cc=deng.hongjie@h3c.com \
--cc=libc-alpha@sourceware.org \
--cc=libc-locales@sourceware.org \
--cc=libc-maintainers@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).