From: Siddhesh Poyarekar <siddhesh@sourceware.org>
To: libc-stable@sourceware.org
Cc: Wilco Dijkstra <wdijkstr@arm.com>
Subject: [PATCH 07/10] Add single-threaded path to _int_free
Date: Sun, 01 Jan 2017 00:00:00 -0000 [thread overview]
Message-ID: <1511878186-31499-8-git-send-email-siddhesh@sourceware.org> (raw)
In-Reply-To: <1511878186-31499-1-git-send-email-siddhesh@sourceware.org>
From: Wilco Dijkstra <wdijkstr@arm.com>
This patch adds single-threaded fast paths to _int_free.
Bypass the explicit locking for larger allocations.
* malloc/malloc.c (_int_free): Add SINGLE_THREAD_P fast paths.
(cherry-picked from a15d53e2de4c7d83bda251469d92a3c7b49a90db)
---
ChangeLog | 4 ++++
malloc/malloc.c | 41 ++++++++++++++++++++++++++++-------------
2 files changed, 32 insertions(+), 13 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 49b720f..30e6f50 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,7 @@
+2017-10-20 Wilco Dijkstra <wdijkstr@arm.com>
+
+ * malloc/malloc.c (_int_free): Add SINGLE_THREAD_P fast paths.
+
2017-10-19 Wilco Dijkstra <wdijkstr@arm.com>
* malloc/malloc.c (_int_free): Fix deadlock bug in consistency check.
diff --git a/malloc/malloc.c b/malloc/malloc.c
index 44996e0..78676a6 100644
--- a/malloc/malloc.c
+++ b/malloc/malloc.c
@@ -4172,24 +4172,34 @@ _int_free (mstate av, mchunkptr p, int have_lock)
/* Atomically link P to its fastbin: P->FD = *FB; *FB = P; */
mchunkptr old = *fb, old2;
- unsigned int old_idx = ~0u;
- do
+
+ if (SINGLE_THREAD_P)
{
- /* Check that the top of the bin is not the record we are going to add
- (i.e., double free). */
+ /* Check that the top of the bin is not the record we are going to
+ add (i.e., double free). */
if (__builtin_expect (old == p, 0))
malloc_printerr ("double free or corruption (fasttop)");
- /* Check that size of fastbin chunk at the top is the same as
- size of the chunk that we are adding. We can dereference OLD
- only if we have the lock, otherwise it might have already been
- deallocated. See use of OLD_IDX below for the actual check. */
- if (have_lock && old != NULL)
- old_idx = fastbin_index(chunksize(old));
- p->fd = old2 = old;
+ p->fd = old;
+ *fb = p;
}
- while ((old = catomic_compare_and_exchange_val_rel (fb, p, old2)) != old2);
+ else
+ do
+ {
+ /* Check that the top of the bin is not the record we are going to
+ add (i.e., double free). */
+ if (__builtin_expect (old == p, 0))
+ malloc_printerr ("double free or corruption (fasttop)");
+ p->fd = old2 = old;
+ }
+ while ((old = catomic_compare_and_exchange_val_rel (fb, p, old2))
+ != old2);
- if (have_lock && old != NULL && __builtin_expect (old_idx != idx, 0))
+ /* Check that size of fastbin chunk at the top is the same as
+ size of the chunk that we are adding. We can dereference OLD
+ only if we have the lock, otherwise it might have already been
+ allocated again. */
+ if (have_lock && old != NULL
+ && __builtin_expect (fastbin_index (chunksize (old)) != idx, 0))
malloc_printerr ("invalid fastbin entry (free)");
}
@@ -4198,6 +4208,11 @@ _int_free (mstate av, mchunkptr p, int have_lock)
*/
else if (!chunk_is_mmapped(p)) {
+
+ /* If we're single-threaded, don't lock the arena. */
+ if (SINGLE_THREAD_P)
+ have_lock = true;
+
if (!have_lock)
__libc_lock_lock (av->mutex);
--
2.7.5
next prev parent reply other threads:[~2017-11-28 14:10 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-01-01 0:00 [PATCH 00/10][2.26] Malloc fixes and improvements Siddhesh Poyarekar
2017-01-01 0:00 ` [PATCH 01/10] malloc: Abort on heap corruption, without a backtrace [BZ #21754] Siddhesh Poyarekar
2017-01-01 0:00 ` [PATCH 02/10] malloc: Remove check_action variable " Siddhesh Poyarekar
2017-01-01 0:00 ` [PATCH 08/10] Fix build issue with SINGLE_THREAD_P Siddhesh Poyarekar
2017-01-01 0:00 ` [PATCH 05/10] malloc: Resolve compilation failure in NDEBUG mode Siddhesh Poyarekar
2017-01-01 0:00 ` [PATCH 06/10] Fix deadlock in _int_free consistency check Siddhesh Poyarekar
2017-01-01 0:00 ` Siddhesh Poyarekar [this message]
2017-01-01 0:00 ` [PATCH 03/10] malloc: Remove corrupt arena flag Siddhesh Poyarekar
2017-01-01 0:00 ` [PATCH 04/10] malloc: Change top_check return type to void Siddhesh Poyarekar
2017-01-01 0:00 ` [PATCH 09/10] Add single-threaded path to malloc/realloc/calloc/memalloc Siddhesh Poyarekar
2017-01-01 0:00 ` [PATCH 10/10] Add single-threaded path to _int_malloc Siddhesh Poyarekar
2017-01-01 0:00 ` [PATCH 00/10][2.26] Malloc fixes and improvements Siddhesh Poyarekar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1511878186-31499-8-git-send-email-siddhesh@sourceware.org \
--to=siddhesh@sourceware.org \
--cc=libc-stable@sourceware.org \
--cc=wdijkstr@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).