From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 112386 invoked by alias); 30 Oct 2019 12:26:51 -0000 Mailing-List: contact libc-stable-help@sourceware.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Subscribe: List-Archive: Sender: libc-stable-owner@sourceware.org Received: (qmail 110841 invoked by uid 89); 30 Oct 2019 12:26:48 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Checked: by ClamAV 0.100.3 on sourceware.org X-Virus-Found: No X-Spam-SWARE-Status: No, score=-26.9 required=5.0 tests=BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,SPF_PASS autolearn=ham version=3.3.1 spammy=Increase X-Spam-Status: No, score=-26.9 required=5.0 tests=BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,SPF_PASS autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on sourceware.org X-Spam-Level: X-HELO: aloka.lostca.se Received: from aloka.lostca.se (HELO aloka.lostca.se) (178.63.46.202) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 30 Oct 2019 12:26:47 +0000 Received: from aloka.lostca.se (aloka [127.0.0.1]) by aloka.lostca.se (Postfix) with ESMTP id 2DFB811036 for ; Wed, 30 Oct 2019 12:26:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=lostca.se; h=date:from:to :subject:message-id:mime-version:content-type; s=howrah; bh=ZvIb +gJh4I0U7JOBumXFR9gm01s=; b=z2mbqzdCzQQauCnoPK9wQPVbIS8YjRNP8d06 +p7UkZS7zF7Rfaef4QyTOiO0r4ak8MO1wS83tUJCAs4KUdSS5am86aKObYYm8wwp BbVL6ss54B2o4AFjvSMGlgriAJYpa0DdRCuT5+S36l1IAFjqJtZic8FQowHnK1Nl IGjOMzg= Received: from localhost (unknown [IPv6:2a01:4f8:120:624c::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aloka.lostca.se (Postfix) with ESMTPSA id CAB3411035 for ; Wed, 30 Oct 2019 12:26:44 +0000 (UTC) Date: Tue, 01 Jan 2019 00:00:00 -0000 From: Arjun Shankar To: libc-stable@sourceware.org Subject: [2.28 COMMITTED] Small tcache improvements Message-ID: <20191030122643.GB9332@aloka.lostca.se> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.10.1 (2018-07-13) X-IsSubscribed: yes X-SW-Source: 2019-10/txt/msg00002.txt.bz2 Change the tcache->counts[] entries to uint16_t - this removes the limit set by char and allows a larger tcache. Remove a few redundant asserts. bench-malloc-thread with 4 threads is ~15% faster on Cortex-A72. Reviewed-by: DJ Delorie * malloc/malloc.c (MAX_TCACHE_COUNT): Increase to UINT16_MAX. (tcache_put): Remove redundant assert. (tcache_get): Remove redundant asserts. (__libc_malloc): Check tcache count is not zero. * manual/tunables.texi (glibc.malloc.tcache_count): Update maximum. (cherry picked from commit 1f50f2ad854c84ead522bfc7331b46dbe6057d53) --- ChangeLog | 8 ++++++++ malloc/malloc.c | 14 ++++++-------- manual/tunables.texi | 2 +- 3 files changed, 15 insertions(+), 9 deletions(-) diff --git a/ChangeLog b/ChangeLog index 668ae78ae7..8c5c162f56 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,11 @@ +2019-05-17 Wilco Dijkstra + + * malloc/malloc.c (MAX_TCACHE_COUNT): Increase to UINT16_MAX. + (tcache_put): Remove redundant assert. + (tcache_get): Remove redundant asserts. + (__libc_malloc): Check tcache count is not zero. + * manual/tunables.texi (glibc.malloc.tcache_count): Update maximum. + 2019-02-04 Joseph Myers * malloc/malloc.c (tcache_get): Compare tcache->counts[tc_idx] diff --git a/malloc/malloc.c b/malloc/malloc.c index 78edbfc8db..22f1049056 100644 --- a/malloc/malloc.c +++ b/malloc/malloc.c @@ -321,6 +321,10 @@ __malloc_assert (const char *assertion, const char *file, unsigned int line, /* This is another arbitrary limit, which tunables can change. Each tcache bin will hold at most this number of chunks. */ # define TCACHE_FILL_COUNT 7 + +/* Maximum chunks in tcache bins for tunables. This value must fit the range + of tcache->counts[] entries, else they may overflow. */ +# define MAX_TCACHE_COUNT UINT16_MAX #endif @@ -2908,12 +2912,10 @@ typedef struct tcache_entry time), this is for performance reasons. */ typedef struct tcache_perthread_struct { - char counts[TCACHE_MAX_BINS]; + uint16_t counts[TCACHE_MAX_BINS]; tcache_entry *entries[TCACHE_MAX_BINS]; } tcache_perthread_struct; -#define MAX_TCACHE_COUNT 127 /* Maximum value of counts[] entries. */ - static __thread bool tcache_shutting_down = false; static __thread tcache_perthread_struct *tcache = NULL; @@ -2923,7 +2925,6 @@ static __always_inline void tcache_put (mchunkptr chunk, size_t tc_idx) { tcache_entry *e = (tcache_entry *) chunk2mem (chunk); - assert (tc_idx < TCACHE_MAX_BINS); /* Mark this chunk as "in the tcache" so the test in _int_free will detect a double free. */ @@ -2940,8 +2941,6 @@ static __always_inline void * tcache_get (size_t tc_idx) { tcache_entry *e = tcache->entries[tc_idx]; - assert (tc_idx < TCACHE_MAX_BINS); - assert (tcache->counts[tc_idx] > 0); tcache->entries[tc_idx] = e->next; --(tcache->counts[tc_idx]); e->key = NULL; @@ -3046,9 +3045,8 @@ __libc_malloc (size_t bytes) DIAG_PUSH_NEEDS_COMMENT; if (tc_idx < mp_.tcache_bins - /*&& tc_idx < TCACHE_MAX_BINS*/ /* to appease gcc */ && tcache - && tcache->entries[tc_idx] != NULL) + && tcache->counts[tc_idx] > 0) { return tcache_get (tc_idx); } diff --git a/manual/tunables.texi b/manual/tunables.texi index d8c22dd29f..74ae253ebf 100644 --- a/manual/tunables.texi +++ b/manual/tunables.texi @@ -188,7 +188,7 @@ per-thread cache. The default (and maximum) value is 1032 bytes on @deftp Tunable glibc.malloc.tcache_count The maximum number of chunks of each size to cache. The default is 7. -The upper limit is 127. If set to zero, the per-thread cache is effectively +The upper limit is 65535. If set to zero, the per-thread cache is effectively disabled. The approximate maximum overhead of the per-thread cache is thus equal -- 2.21.0