From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 79537 invoked by alias); 18 Nov 2019 18:01:33 -0000 Mailing-List: contact libc-stable-help@sourceware.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Subscribe: List-Archive: Sender: libc-stable-owner@sourceware.org Received: (qmail 79527 invoked by uid 89); 18 Nov 2019 18:01:33 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Checked: by ClamAV 0.100.3 on sourceware.org X-Virus-Found: No X-Spam-SWARE-Status: No, score=-26.9 required=5.0 tests=BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,SPF_PASS autolearn=ham version=3.3.1 spammy= X-Spam-Status: No, score=-26.9 required=5.0 tests=BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,SPF_PASS autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on sourceware.org X-Spam-Level: X-HELO: aloka.lostca.se Received: from aloka.lostca.se (HELO aloka.lostca.se) (178.63.46.202) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Mon, 18 Nov 2019 18:01:31 +0000 Received: from aloka.lostca.se (aloka [127.0.0.1]) by aloka.lostca.se (Postfix) with ESMTP id 51AB315E71 for ; Mon, 18 Nov 2019 18:01:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=lostca.se; h=date:from:to :subject:message-id:mime-version:content-type; s=howrah; bh=hjH2 D+c7R9sVa+WYy7jgOS3BLp0=; b=TWYDjhsLce0+xUp5U33ldTOFVgcWu8LFgFVT 40AgXlA3otfPJgNwWduKBkm2SnoKAHN00SWe3UFAiTyG+q8tbvSLcYfjmvCrbpe+ UEt04KZTtYTqY/xOlfbo4eXTkFlpvp/FOAnx85g9VRwBfmnic+k9JiRsJ+LJ812J V7ZP4xY= Received: from localhost (unknown [IPv6:2a01:4f8:120:624c::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aloka.lostca.se (Postfix) with ESMTPSA id 166A515E70 for ; Mon, 18 Nov 2019 18:01:29 +0000 (UTC) Date: Tue, 01 Jan 2019 00:00:00 -0000 From: Arjun Shankar To: libc-stable@sourceware.org Subject: [2.29 COMMITTED] Small tcache improvements Message-ID: <20191118180128.GB73357@aloka.lostca.se> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.10.1 (2018-07-13) X-IsSubscribed: yes X-SW-Source: 2019-11/txt/msg00011.txt.bz2 Change the tcache->counts[] entries to uint16_t - this removes the limit set by char and allows a larger tcache. Remove a few redundant asserts. bench-malloc-thread with 4 threads is ~15% faster on Cortex-A72. Reviewed-by: DJ Delorie * malloc/malloc.c (MAX_TCACHE_COUNT): Increase to UINT16_MAX. (tcache_put): Remove redundant assert. (tcache_get): Remove redundant asserts. (__libc_malloc): Check tcache count is not zero. * manual/tunables.texi (glibc.malloc.tcache_count): Update maximum. (cherry picked from commit 1f50f2ad854c84ead522bfc7331b46dbe6057d53) --- ChangeLog | 8 ++++++++ malloc/malloc.c | 14 ++++++-------- manual/tunables.texi | 2 +- 3 files changed, 15 insertions(+), 9 deletions(-) diff --git a/ChangeLog b/ChangeLog index 0c7b7e9374..a58505952e 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,11 @@ +2019-05-17 Wilco Dijkstra + + * malloc/malloc.c (MAX_TCACHE_COUNT): Increase to UINT16_MAX. + (tcache_put): Remove redundant assert. + (tcache_get): Remove redundant asserts. + (__libc_malloc): Check tcache count is not zero. + * manual/tunables.texi (glibc.malloc.tcache_count): Update maximum. + 2019-02-04 Joseph Myers * malloc/malloc.c (tcache_get): Compare tcache->counts[tc_idx] diff --git a/malloc/malloc.c b/malloc/malloc.c index 59fa1a18a5..3163601492 100644 --- a/malloc/malloc.c +++ b/malloc/malloc.c @@ -321,6 +321,10 @@ __malloc_assert (const char *assertion, const char *file, unsigned int line, /* This is another arbitrary limit, which tunables can change. Each tcache bin will hold at most this number of chunks. */ # define TCACHE_FILL_COUNT 7 + +/* Maximum chunks in tcache bins for tunables. This value must fit the range + of tcache->counts[] entries, else they may overflow. */ +# define MAX_TCACHE_COUNT UINT16_MAX #endif @@ -2915,12 +2919,10 @@ typedef struct tcache_entry time), this is for performance reasons. */ typedef struct tcache_perthread_struct { - char counts[TCACHE_MAX_BINS]; + uint16_t counts[TCACHE_MAX_BINS]; tcache_entry *entries[TCACHE_MAX_BINS]; } tcache_perthread_struct; -#define MAX_TCACHE_COUNT 127 /* Maximum value of counts[] entries. */ - static __thread bool tcache_shutting_down = false; static __thread tcache_perthread_struct *tcache = NULL; @@ -2930,7 +2932,6 @@ static __always_inline void tcache_put (mchunkptr chunk, size_t tc_idx) { tcache_entry *e = (tcache_entry *) chunk2mem (chunk); - assert (tc_idx < TCACHE_MAX_BINS); /* Mark this chunk as "in the tcache" so the test in _int_free will detect a double free. */ @@ -2947,8 +2948,6 @@ static __always_inline void * tcache_get (size_t tc_idx) { tcache_entry *e = tcache->entries[tc_idx]; - assert (tc_idx < TCACHE_MAX_BINS); - assert (tcache->counts[tc_idx] > 0); tcache->entries[tc_idx] = e->next; --(tcache->counts[tc_idx]); e->key = NULL; @@ -3053,9 +3052,8 @@ __libc_malloc (size_t bytes) DIAG_PUSH_NEEDS_COMMENT; if (tc_idx < mp_.tcache_bins - /*&& tc_idx < TCACHE_MAX_BINS*/ /* to appease gcc */ && tcache - && tcache->entries[tc_idx] != NULL) + && tcache->counts[tc_idx] > 0) { return tcache_get (tc_idx); } diff --git a/manual/tunables.texi b/manual/tunables.texi index 8edfea4edd..17de5bcc6b 100644 --- a/manual/tunables.texi +++ b/manual/tunables.texi @@ -189,7 +189,7 @@ per-thread cache. The default (and maximum) value is 1032 bytes on @deftp Tunable glibc.malloc.tcache_count The maximum number of chunks of each size to cache. The default is 7. -The upper limit is 127. If set to zero, the per-thread cache is effectively +The upper limit is 65535. If set to zero, the per-thread cache is effectively disabled. The approximate maximum overhead of the per-thread cache is thus equal -- 2.21.0