From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dragonfly.birch.relay.mailchannels.net (dragonfly.birch.relay.mailchannels.net [23.83.209.51]) by sourceware.org (Postfix) with ESMTPS id E305E3844044; Fri, 11 Jun 2021 06:41:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org E305E3844044 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=gotplt.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gotplt.org X-Sender-Id: dreamhost|x-authsender|siddhesh@gotplt.org Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id EC945121A49; Fri, 11 Jun 2021 06:41:56 +0000 (UTC) Received: from pdx1-sub0-mail-a51.g.dreamhost.com (100-96-133-111.trex.outbound.svc.cluster.local [100.96.133.111]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 60FD012199A; Fri, 11 Jun 2021 06:41:56 +0000 (UTC) X-Sender-Id: dreamhost|x-authsender|siddhesh@gotplt.org Received: from pdx1-sub0-mail-a51.g.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384) by 100.96.133.111 (trex/6.3.1); Fri, 11 Jun 2021 06:41:56 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|siddhesh@gotplt.org X-MailChannels-Auth-Id: dreamhost X-Eight-Harbor: 27eaccfa5b37f904_1623393716727_2466470692 X-MC-Loop-Signature: 1623393716726:32045501 X-MC-Ingress-Time: 1623393716726 Received: from pdx1-sub0-mail-a51.g.dreamhost.com (localhost [127.0.0.1]) by pdx1-sub0-mail-a51.g.dreamhost.com (Postfix) with ESMTP id 1CC6B7F53E; Thu, 10 Jun 2021 23:41:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gotplt.org; h=subject:to :cc:references:from:message-id:date:mime-version:in-reply-to :content-type:content-transfer-encoding; s=gotplt.org; bh=4THzMC 2fgwgIC8iC/i9PDBzTnyo=; b=lQpYpRB2K4F3KAlXTXKWQ6QM59SEI+y/tN291u aAO+iPbmFtyjIrhHjJEqcMLc8mfQnw0AsZzWWK01xDiHr4bJ0Rh5C8GRO3gVfIkK OCAJnHgaixhNn+ZF2HABoNDs56sm3NwxjO7qfhA17XDpT4KUvsL40wLpwTCz7iM3 UM2dY= Received: from [192.168.1.126] (unknown [1.186.101.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: siddhesh@gotplt.org) by pdx1-sub0-mail-a51.g.dreamhost.com (Postfix) with ESMTPSA id 5BDB87F535; Thu, 10 Jun 2021 23:41:50 -0700 (PDT) Subject: Re: [PATCH] release all idle heaps in the non-main-arena To: liudongyun , libc-alpha@sourceware.org Cc: libc-locales@sourceware.org, libc-maintainers@gnu.org, deng.hongjie@h3c.com, dai.xianjun@h3c.com References: <20210611040852.21745-1-liu.dongyun@h3c.com> X-DH-BACKEND: pdx1-sub0-mail-a51 From: Siddhesh Poyarekar Message-ID: <06d6723d-3209-c957-b855-8cc9b0bdbbe4@gotplt.org> Date: Fri, 11 Jun 2021 12:11:45 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 In-Reply-To: <20210611040852.21745-1-liu.dongyun@h3c.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-3029.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-locales@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-locales mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jun 2021 06:42:00 -0000 On 6/11/21 9:38 AM, liudongyun wrote: > liu.dongyun@h3c.com report this question in follow: > The system has a total of 32G memory and 12 core cpu. > After frequent malloc and free memory by multiple threads, we found that many > physical memory that should have been released have not been released.We observe that > the maximum free memory can reach 15G. > > deng.hongjie@h3c.com recommend reducing the heap size of the non-main-arena, but it > doesn't work. liu.dongyun@h3c.com found When the last heap in the user process into > the non-main-area has memory in use, the previous heap will not be released, even if the > previous heap is already free. This mechanism resulting in the memory cache reaching 15G. Does that cause any actual performance degradation or any other issues though? heap_shrink() calls madvise (MADV_DONTNEED) on the free parts of the heap to tell the kernel that the blocks are not needed, so a fully freed heap ought to not have any performance impact. The RSS footprint will depend on when the kernel reclaims those pages, which may not be immediate since madvise is just a hint. Alternatively, if you disable memory overcommit by setting /proc/sys/vm/overcommit_memory to 2, heap_shrink() will drop all permissions on the heap with mprotect(PROT_NONE) instead of doing madvise. This has a more immediate effect on RSS usage unlike madvise. Siddhesh