public inbox for glibc-bugs@sourceware.org help / color / mirror / Atom feed
From: "dimahabr at gmail dot com" <sourceware-bugzilla@sourceware.org> To: glibc-bugs@sourceware.org Subject: [Bug malloc/26969] A common malloc pattern can make memory not given back to OS Date: Fri, 29 Jan 2021 16:08:53 +0000 [thread overview] Message-ID: <bug-26969-131-twtEatOYEm@http.sourceware.org/bugzilla/> (raw) In-Reply-To: <bug-26969-131@http.sourceware.org/bugzilla/> https://sourceware.org/bugzilla/show_bug.cgi?id=26969 Dmitry <dimahabr at gmail dot com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dimahabr at gmail dot com --- Comment #3 from Dmitry <dimahabr at gmail dot com> --- This is a really common problem especially for long-running multithreaded processes with a lot of arenas. Calling malloc_trim - not always an option, if a service using C bindings and is written on high level languages. Also, malloc_trim would cause unnecessary overhead, since it will be trimming and locking all arenas. Additional confusion comes from man page for malloc_trim which says it's called sometimes during free. I was looking at the code and have the following suggestion for improvement: the main idea is to call mtrim for arena in _int_free. To amortize performance overhead: call it only if free is called for a chunk with size > than FASTBIN_CONSOLIDATION_THRESHOLD, and we have a chance to free more memory than let's say 3*TRIM_THRESHOLD. To understand how much we can return to OS we can add 1 bit flag to chunk, set it to 0 after returning to OS, set to 1 otherwise. This would not give 100% accurate result, but should give a good estimate. I'd be glad to work on this functionality, if you think it makes sense. If you have other suggestions also happy to discuss this. The main point - mtrim should be called during free some times, otherwise current malloc is unfortunately hard to use for long-running multithreaded workload, since it's using 2-3x more RSS in comparison with jemalloc or tcmalloc. -- You are receiving this mail because: You are on the CC list for the bug.
next prev parent reply other threads:[~2021-01-29 16:08 UTC|newest] Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-11-28 13:58 [Bug malloc/26969] New: " keyid.w at qq dot com 2020-11-28 13:59 ` [Bug malloc/26969] " keyid.w at qq dot com 2020-11-29 3:24 ` keyid.w at qq dot com 2020-12-01 1:03 ` uwydoc at gmail dot com 2020-12-01 2:51 ` carlos at redhat dot com 2020-12-01 8:43 ` keyid.w at qq dot com 2021-01-29 16:08 ` dimahabr at gmail dot com [this message] 2021-02-01 8:52 ` keyid.w at qq dot com 2022-06-29 16:34 ` romash at rbbn dot com
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=bug-26969-131-twtEatOYEm@http.sourceware.org/bugzilla/ \ --to=sourceware-bugzilla@sourceware.org \ --cc=glibc-bugs@sourceware.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).