From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 1694 invoked by alias); 28 Nov 2017 14:10:22 -0000 Mailing-List: contact libc-stable-help@sourceware.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Subscribe: List-Archive: Sender: libc-stable-owner@sourceware.org Received: (qmail 1676 invoked by uid 89); 28 Nov 2017 14:10:22 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Checked: by ClamAV 0.99.2 on sourceware.org X-Virus-Found: No X-Spam-SWARE-Status: No, score=-25.9 required=5.0 tests=BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,KB_WAM_FROM_NAME_SINGLEWORD,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL autolearn=ham version=3.3.2 spammy= X-Spam-Status: No, score=-25.9 required=5.0 tests=BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,KB_WAM_FROM_NAME_SINGLEWORD,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on sourceware.org X-Spam-Level: X-HELO: hapkido.dreamhost.com Received: from hapkido.dreamhost.com (HELO hapkido.dreamhost.com) (66.33.216.122) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 28 Nov 2017 14:10:16 +0000 Received: from homiemail-a52.g.dreamhost.com (sub5.mail.dreamhost.com [208.113.200.129]) by hapkido.dreamhost.com (Postfix) with ESMTP id DE0E38D623 for ; Tue, 28 Nov 2017 06:10:14 -0800 (PST) Received: from homiemail-a52.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a52.g.dreamhost.com (Postfix) with ESMTP id A6CD56000638; Tue, 28 Nov 2017 06:10:14 -0800 (PST) Received: from devel.in.reserved-bit.com (unknown [202.189.238.75]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: siddhesh@gotplt.org) by homiemail-a52.g.dreamhost.com (Postfix) with ESMTPSA id 5D00A6000630; Tue, 28 Nov 2017 06:10:13 -0800 (PST) From: Siddhesh Poyarekar To: libc-stable@sourceware.org Cc: Wilco Dijkstra Subject: [PATCH 07/10] Add single-threaded path to _int_free Date: Sun, 01 Jan 2017 00:00:00 -0000 Message-Id: <1511878186-31499-8-git-send-email-siddhesh@sourceware.org> X-Mailer: git-send-email 2.7.5 In-Reply-To: <1511878186-31499-1-git-send-email-siddhesh@sourceware.org> References: <1511878186-31499-1-git-send-email-siddhesh@sourceware.org> X-SW-Source: 2017-11/txt/msg00035.txt.bz2 From: Wilco Dijkstra This patch adds single-threaded fast paths to _int_free. Bypass the explicit locking for larger allocations. * malloc/malloc.c (_int_free): Add SINGLE_THREAD_P fast paths. (cherry-picked from a15d53e2de4c7d83bda251469d92a3c7b49a90db) --- ChangeLog | 4 ++++ malloc/malloc.c | 41 ++++++++++++++++++++++++++++------------- 2 files changed, 32 insertions(+), 13 deletions(-) diff --git a/ChangeLog b/ChangeLog index 49b720f..30e6f50 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,7 @@ +2017-10-20 Wilco Dijkstra + + * malloc/malloc.c (_int_free): Add SINGLE_THREAD_P fast paths. + 2017-10-19 Wilco Dijkstra * malloc/malloc.c (_int_free): Fix deadlock bug in consistency check. diff --git a/malloc/malloc.c b/malloc/malloc.c index 44996e0..78676a6 100644 --- a/malloc/malloc.c +++ b/malloc/malloc.c @@ -4172,24 +4172,34 @@ _int_free (mstate av, mchunkptr p, int have_lock) /* Atomically link P to its fastbin: P->FD = *FB; *FB = P; */ mchunkptr old = *fb, old2; - unsigned int old_idx = ~0u; - do + + if (SINGLE_THREAD_P) { - /* Check that the top of the bin is not the record we are going to add - (i.e., double free). */ + /* Check that the top of the bin is not the record we are going to + add (i.e., double free). */ if (__builtin_expect (old == p, 0)) malloc_printerr ("double free or corruption (fasttop)"); - /* Check that size of fastbin chunk at the top is the same as - size of the chunk that we are adding. We can dereference OLD - only if we have the lock, otherwise it might have already been - deallocated. See use of OLD_IDX below for the actual check. */ - if (have_lock && old != NULL) - old_idx = fastbin_index(chunksize(old)); - p->fd = old2 = old; + p->fd = old; + *fb = p; } - while ((old = catomic_compare_and_exchange_val_rel (fb, p, old2)) != old2); + else + do + { + /* Check that the top of the bin is not the record we are going to + add (i.e., double free). */ + if (__builtin_expect (old == p, 0)) + malloc_printerr ("double free or corruption (fasttop)"); + p->fd = old2 = old; + } + while ((old = catomic_compare_and_exchange_val_rel (fb, p, old2)) + != old2); - if (have_lock && old != NULL && __builtin_expect (old_idx != idx, 0)) + /* Check that size of fastbin chunk at the top is the same as + size of the chunk that we are adding. We can dereference OLD + only if we have the lock, otherwise it might have already been + allocated again. */ + if (have_lock && old != NULL + && __builtin_expect (fastbin_index (chunksize (old)) != idx, 0)) malloc_printerr ("invalid fastbin entry (free)"); } @@ -4198,6 +4208,11 @@ _int_free (mstate av, mchunkptr p, int have_lock) */ else if (!chunk_is_mmapped(p)) { + + /* If we're single-threaded, don't lock the arena. */ + if (SINGLE_THREAD_P) + have_lock = true; + if (!have_lock) __libc_lock_lock (av->mutex); -- 2.7.5