From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 44D583858404 for ; Wed, 10 Nov 2021 00:36:13 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 44D583858404 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-288-3MEzoigHMVyF0r7O18Ytyw-1; Tue, 09 Nov 2021 19:36:09 -0500 X-MC-Unique: 3MEzoigHMVyF0r7O18Ytyw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DE67818D6A25; Wed, 10 Nov 2021 00:36:08 +0000 (UTC) Received: from greed.delorie.com (ovpn-112-76.phx2.redhat.com [10.3.112.76]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B149667841; Wed, 10 Nov 2021 00:36:08 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.15.2/8.15.2) with ESMTP id 1AA0a7Oi2811794; Tue, 9 Nov 2021 19:36:07 -0500 From: DJ Delorie To: Patrick McGehearty Cc: libc-alpha@sourceware.org Subject: Re: [PATCH v2] Remove upper limit on tunable MALLOC_MMAP_THRESHOLD In-Reply-To: <03e480f8-c6c5-3247-5726-8982b02123af@oracle.com> Date: Tue, 09 Nov 2021 19:36:07 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-6.6 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Nov 2021 00:36:15 -0000 Patrick McGehearty writes: > If a chunk smaller than the mmap_threshold is requested, > then MORECORE [typically sbrk()] is called and HEAP_MAX > is not considered by the malloc code.=C2=A0 Heaps are only used > for mmap()ed allocations, not sbrk()'ed allocations, > so far as I can tell in reading the code. There are two types of arenas: the sbrk-based arena (limited in size by ulimit), and zero or more mmap-based arenas (limited by HEAP_MAX). The sbrk-based one is used when the program is single threaded; the mmap-based ones are used when the program is multi-threaded. Your original email made it sound like you were concerned with the multi-threaded case, where the mmap-based heaps are used. In either case, a malloc() request may be satisfied by pulling a free chunk out of either type of arena (possibly growing the arena if needed and possible), or by calling mmap() directly to satisfy that one request. I would think mmap_threshold should still apply if you're using the mmap'd heaps, so you can reserve the heaps for smaller chunks, but that is meaningless if mmap_threshold is larger than the heap size. I could not find an obvious place in the code where mmap_threshold is used to bypass the mmap'd heaps, though. So while I have no problems with allowing larger mmap_threshold settings for the sbrk-based arena, I still wonder what happens to requests that go through an mmap-based arena that are larger than HEAP_MAX but still under the mmap_threshold. Of course, I've spent more time typing this response than it would take to write a test program and see what happens ;-) > It might be desirable to also allow HEAP_MAX to be set by > the user before the first call to malloc, but I see that > as a separate task. Our current implementation requires that the heap size be a compile-time constant, but... yeah.