From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 77682 invoked by alias); 19 Oct 2016 06:53:36 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Received: (qmail 77659 invoked by uid 89); 19 Oct 2016 06:53:35 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.4 required=5.0 tests=AWL,BAYES_00,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,RCVD_IN_SORBS_SPAM,SPF_PASS autolearn=no version=3.3.2 spammy=2.10, ber, cre, calculated X-Spam-User: qpsmtpd, 2 recipients X-HELO: mail-wm0-f67.google.com X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=Xj5rKwb8ineGsQu9yNOsC6yBdltSXh+FJdO8VEyjb0o=; b=BDTFQ+Brs0maLJkVkcI8lGzDMeIhTArSjIplnY8x4kpujwxyLNqq4KcgUT5/XKtaM0 4IMoILs5bvPTLBdPi4eDa9ZxDh4bivMsbl4nzAN725hOhgBF5QucAonBbfr9zo2BRSaK qqS6aLSd37FcyAylvAl+o72tFpY9xrVz+pJ6XA8XElhlfZHe3LDgxxuFGRWcne3bjb4r Jb2NHPcAtrybCa5z4wHP5vq2nYvWq5+JcY0Jaav8B5vZXJnDH952EHxN0Mzd1DmKwwHw R9gdY1dy1UJGJ/XSfNPkJg07lAyXtpV8koVpSVheF61nu7i5RXD3JA0Xf6beMz6v5d2z 1Yag== X-Gm-Message-State: AA6/9Rl3+jkCnUcIqs98ntHq+3eq9CbUuhiNwtW+1dSqSuJkWjxh+V3xQ3S6w4dVHDyAiw== X-Received: by 10.28.209.142 with SMTP id i136mr1460998wmg.1.1476860002942; Tue, 18 Oct 2016 23:53:22 -0700 (PDT) Subject: Re: [PATCH 2/2] Document the M_ARENA_* mallopt parameters To: siddhesh@sourceware.org, Carlos O'Donell References: <1476120388-26662-1-git-send-email-siddhesh@sourceware.org> <1476120388-26662-2-git-send-email-siddhesh@sourceware.org> <8183dc36-7d6a-bbfc-b6ff-568a2de8af28@redhat.com> <33f30947-9f11-363e-3f35-6c593632f000@sourceware.org> <968422de-27f2-3359-6a76-61c152c253dd@sourceware.org> Cc: mtk.manpages@gmail.com, libc-alpha From: "Michael Kerrisk (man-pages)" Message-ID: <44b3adfe-e986-48f0-3b91-efafc4b95632@gmail.com> Date: Wed, 19 Oct 2016 06:53:00 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <968422de-27f2-3359-6a76-61c152c253dd@sourceware.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-SW-Source: 2016-10/txt/msg00322.txt.bz2 Hi Siddhesh, On 10/18/2016 06:46 PM, Siddhesh Poyarekar wrote: > On Tuesday 18 October 2016 09:33 PM, Michael Kerrisk (man-pages) wrote: >>>> I don't think you're correct here. 'arena_max' is a field in a static >>>> structure that is not otherwise initialized, AFAICT. So, it has the >>>> value zero. (Some dirty hacking with a program that uses >>>> malloc_get_state() and inspects the internal data structure seems >>>> to confirm this.) >>>> >>>> And then in malloc/arena.c we have >>>> >>>> if (mp_.arena_max != 0) >>>> narenas_limit = mp_.arena_max; >>>> else if (narenas > mp_.arena_test) >>>> { >>>> int n = __get_nprocs (); >>>> >>>> if (n >= 1) >>>> narenas_limit = NARENAS_FROM_NCORES (n); >>>> else >>>> /* We have no information about the system. Assume two >>>> cores. */ >>>> narenas_limit = NARENAS_FROM_NCORES (2); >>>> >>>> So, I believe my original statement about M_ARENA_MAX is correct. >>>> Have I missed something? >>> >>> You're right in that the variable arena_max is initialized to 0. >> >> Okay. >> >>> However you also concluded that there is no limit to the number of >>> arenas that can be created when arena_max is 0, which is incorrect. As >>> the code snippet you pasted above shows that if arena_max is 0, once we >>> cross arena_test arenas, the narenas_limit static variable is set to a >>> default upper limit based on the number of cores. That acts as the >>> upper limit to the number of arenas that can be created when arena_max is 0. >> >> D'oh! Yes, of course. Thanks for that. Not sure how I managed to >> misread that code :-}. So a better formulation would be something like: >> >> The default value of this parameter is 0, meaning that the limit on >> the number of arenas is determined according to the setting of >> M_ARENA_TEST. > > No, it is not the value of M_ARENA_TEST :) I agree. But it's also not what I said ;-). I said "determined according to the value of..." Currently, the M_ARENA_TEST text in mallopt(3) says: M_ARENA_TEST This parameter specifies a value, in number of arenas cre‐ ated, at which point the system configuration will be exam‐ ined to determine a hard limit on the number of created arenas. (See M_ARENA_MAX for the definition of an arena.) The computation of the arena hard limit is implementation- defined and is usually calculated as a multiple of the num‐ ber of available CPUs. Once the hard limit is computed, the result is final and constrains the total number of are‐ nas. The default value for the M_ARENA_TEST parameter is 2 on systems where sizeof(long) is 4; otherwise the default value is 8. This parameter has been available since glibc 2.10 via --enable-experimental-malloc, and since glibc 2.15 by default. The value of M_ARENA_TEST is not used when M_ARENA_MAX has a nonzero value. > M_ARENA_TEST is set by default to NARENAS_FROM_CORES(1). The limit on > the number of arenas is decided after M_ARENA_TEST number of arenas have > already created, but it is actually set to NARENAS_FROM_CORES(n), where > n is the number of cores. If we can't find the number of cores for some > reason, then we set the limit to NARENAS_FROM_CORES(2). > > So something like this is more accurate: > > The default value of this parameter is 0, meaning that the limit on > the number of arenas is determined by the number of CPU cores > online and the size of the '''long''' datatype. For 32-bit systems > the limit of on the number of arenas is 2 * '''number of CPU cores > online''' while 64-bit systems, the limit on the number of arenas > is 8 * '''number of CPU cores online'''. If information on CPU > cores is not available, it is assumed that there are 2 CPU cores > online. Your details following are a little more precise. But I believe Carlos, when he wrote the text about how the limit was calculated, deliberately chose to be more vague. I'm okay with including this level of detail in the man page, but, if I recall correctly, Carlos wanted to avoid it. Cheers, Michael > I have implicitly stated here that 32-bit long == 32-bit pointers since > I don't think there are Unix systems that have 32-bit long and 64-bit > pointers and also because the choice of multipliers for cores (2 and 8) > correlates better to the size of the address space than to size of long. > > Siddhesh > -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/