From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by sourceware.org (Postfix) with ESMTPS id 6A69B3858D37 for ; Thu, 13 Apr 2023 16:13:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 6A69B3858D37 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 33DCxUxT005269; Thu, 13 Apr 2023 16:13:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=references : from : to : cc : subject : in-reply-to : date : message-id : content-type : mime-version; s=corp-2023-03-30; bh=fyAtjHXeyoTlnAqGaeKYbadCqCQe6I0bZnyGBNQ1Pz8=; b=2/Y2EBvn6ne0tyrK7hf7oYmSEn2sT5EsKfNAeJ7BWWPA2ZJoqPNMBGBfCq/N1BCCrRXJ 2RzVlvTDRzU6vVTUDDFYjXdZa0iJtX20IpXVC/CSHm2XfDprGKrDPQhAvh1RgC84D/tB 9mHDRdvzMSarTTKSHznG7fTzkWUE24Gkqn0Lvl5tOI1N9CeHFz0IYgrD27hNsvOqXTFb 5XQsC9pG/qurzBo7erwpmLMF54K/JI7BDK8FM2V0iinj1Q9/ALnB+WTlVoPdqmcauZk9 IjMKNdE1NMDBxAJmRncYCId5sE6cZKg3p4jt4TGSltDY8+Ye+nVYOcmm7ls27gYiMTDW sA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3pu0bw3mx0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 13 Apr 2023 16:13:48 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 33DFlUSb039737; Thu, 13 Apr 2023 16:13:47 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2044.outbound.protection.outlook.com [104.47.66.44]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3puwbryk38-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 13 Apr 2023 16:13:47 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ae0J+q3ZW5Gi0NJetzXGvRQ0TsbA2juS975JZc2Y5vyZgLLRqjHQNXC/dBa5xAxCYS4ush2mKCaKyS2xBU7U5bpN7aUonqhrwaAK+IYWeo2cVOKyJGys2iRrctx3Vro9ANxsKZIOyiGgllGZugnhtC3a0MuoUnbJ8Zc5Hbq+Uq+PrQj6iFm7A4VZDBCAN6Ju7WfwX7/41g/79Kv1lqKrcj9QyvIIaGXL62fmay7FFQh0danu5Go1nOLEXozbYAGHM21RER75uCdj2a/JoqbyUDajjyrXRT0k2UAfOmQ/uBqwZrURTQgSbzvoFix37tTc7e/FdKcCWS2c1rn6StFNxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fyAtjHXeyoTlnAqGaeKYbadCqCQe6I0bZnyGBNQ1Pz8=; b=Gq1HvITxQqzH2X2PCoYxVdvQB9UxwdtKSg37gzmsMKCkYNz30HXdnPAEie/8gj/3mLrQGrGs5FMrmXPb6gTRQE6qgg8tUp+kusKs4p4cCHcd0qNnDqEswDjWP78+PnXfwDm1a6p1A83Pklu9pz/KsqKrsRLbfBLQMKkXv4V/VFQvw75cZVo8m6vDQgmChSsWTBj+rl7CdIMAmSsoYoJQO5D6slPeAUW0clVFtaWRvK04vrkAW3Q4rs+DIwTQ1tuTbwFF+dz2YZWmTSkgjcAWYqyM28SkIyKApO/KtbrqsDuJPbIKaaBi3zTD5bZErV3mAM+qI6nSuE1FPfmVkXlX+Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fyAtjHXeyoTlnAqGaeKYbadCqCQe6I0bZnyGBNQ1Pz8=; b=MANkveuWmtCwMj1TolTymVLXENmguvwm7TKG+sgzajj1mJcWgJJQfUpJm9S53Hx+f4BpQjRw2uKJpIWRTzQsaXxrT7h6F2GUDI+hsRq1FBXXFgoA31aTGOiWyiDEhjArfw8rn8oLbMtbAU19ZZLEXMLAZXnRsugWbrLgNn+rwg0= Received: from BN6PR1001MB2340.namprd10.prod.outlook.com (2603:10b6:405:30::36) by CH3PR10MB7502.namprd10.prod.outlook.com (2603:10b6:610:163::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.30; Thu, 13 Apr 2023 16:13:45 +0000 Received: from BN6PR1001MB2340.namprd10.prod.outlook.com ([fe80::4998:96fd:86b1:a388]) by BN6PR1001MB2340.namprd10.prod.outlook.com ([fe80::4998:96fd:86b1:a388%6]) with mapi id 15.20.6277.036; Thu, 13 Apr 2023 16:13:45 +0000 References: <20230328152258.54844-1-cupertino.miranda@oracle.com> <20230328152258.54844-2-cupertino.miranda@oracle.com> <8f313a5d-f16a-d682-1d78-f216c446099f@linaro.org> <873555cwh5.fsf@oracle.com> <645838ea-afc0-0289-233e-50fa22f126c1@linaro.org> User-agent: mu4e 1.4.15; emacs 28.1 From: Cupertino Miranda To: Adhemerval Zanella Netto Cc: libc-alpha@sourceware.org, Florian Weimer , jose.marchesi@oracle.com, elena.zannoni@oracle.com Subject: Re: [PATCH v5 1/1] Created tunable to force small pages on stack allocation. In-reply-to: <645838ea-afc0-0289-233e-50fa22f126c1@linaro.org> Date: Thu, 13 Apr 2023 17:13:39 +0100 Message-ID: <87pm87daks.fsf@oracle.com> Content-Type: text/plain X-ClientProxiedBy: LO4P123CA0164.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18a::7) To BN6PR1001MB2340.namprd10.prod.outlook.com (2603:10b6:405:30::36) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN6PR1001MB2340:EE_|CH3PR10MB7502:EE_ X-MS-Office365-Filtering-Correlation-Id: 66f49efc-32c2-41b1-8a90-08db3c3a0e01 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6qLlNQ+EwUzOanV4B7zv78bCCTu6D/ZVHY7GC8Yw2HwzPAszEW9Zn1NylpHt6WHXOH+QS4jtGZ43gn4DT0GpKbovM6h8qjTj0S+ryGY6MrpRLjx8LHZ3qZQxNCwY2xUnJnZ3Q0UslsuzH0/HIhaqqBoQTw1JI22v+eg2R+xAA2v75PvslyhX3Bl8eTHIoWASlAm8EJnghiDBHQwUyDomcOY4na9mwliJgkt6lWT4RE6d/3BQV3d90chtUKATM8io1e/0uSjXb++WDIsbg1178rSMcxt6ZSXERLZLya7h24wTT+69oBDk3IGD1V7nED5VbVRXpxAQvmM/KTEr584K2JuQTv6j/Ab6i4/IOAkCKiCL1h+whkCbRFPqOSM8CB5exoeb1Qd+n84vNzYAqPBUmMt+hvRIX5BLmnjwIzAIuvFPljKQRKvipxPoumQlfp6lJNI2ARJWbi2F4JcLqGge42RlsITxDQivl+KNPGaYF1pZf8SEwgLQNfYX1+p+3E03U21P3AePFZ7Gxj41eJu4KwwXCXoCpPFQ7zWU0gW/zJ0oBssaSJUkIAuJ2A2e/CfEwzJEc0JI/ntKkLeollmUQg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN6PR1001MB2340.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(376002)(39860400002)(136003)(346002)(396003)(366004)(451199021)(316002)(41300700001)(53546011)(6506007)(6512007)(6486002)(966005)(186003)(86362001)(6666004)(38100700002)(107886003)(2616005)(66556008)(83380400001)(66476007)(66946007)(6916009)(4326008)(36756003)(8676002)(8936002)(5660300002)(2906002)(44832011)(478600001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?uXA1aypNpqvlJ5dsuAcu4llb1KSh4FFXABDc9V7FkYVZcxRc6RdySxfFxvZ4?= =?us-ascii?Q?mEDy6w8fciArZIEvpsSAXSZNW4BTuALdXgz90wB9Y+Do8S3CQpoNRo0hxwVk?= =?us-ascii?Q?w7QVXvX9vtS3hGVX/P/dlzEe4OuPsnJe5LRFw3hPmSTOvozxYPY3cPDm0KHV?= =?us-ascii?Q?T36SHxsxa9fUyFCgQaiu9oksY59vwhV5gqtaqkALv75BW0IQ6n17Wi+xwFTK?= =?us-ascii?Q?qn1lWd4PKVGOpdEBUCiE7UmaGJok/bWCcs+ubd9VvbiRHE72lGeBC0k+G8zG?= =?us-ascii?Q?wSkAFcfeeKr8GtDhLW38jKhBZp0Pai8dyvAbljAEFnV5/XSlS5V6djwmcdfJ?= =?us-ascii?Q?jYSY0gsb3zJUabaPojWLImyKnufQPGOmVt89vu3+Q1jiHppXSbCPOiOu3Wh5?= =?us-ascii?Q?nTzJSuX9XnzfANig1Z+z74bRBIsytZHLKGx+O1Rm84NHIBX2DlNCRdJ+jUS5?= =?us-ascii?Q?i8H2MOvl4Uegkhu4wRGhcZhyhCeD1XIwZB8wL+LSJTBBn8VJAOMGui+5Esvd?= =?us-ascii?Q?OHLtLdMSy7/LClWbovZHFgWalDf6qFw7OGtYPnj6EGswbVIQ2RdLFq7+M8dt?= =?us-ascii?Q?z7pet1GiBtrThv3LTV+JBPtqZAM1hn7V0XilMycA4aunMaGL5oVVnKc46n2b?= =?us-ascii?Q?sAv9BfmmPqULuUGvFHbeB5fijY8rI+v8VFXBnH2Y6+XtaMcwra+KJRne102w?= =?us-ascii?Q?GalO7l4vC+7VcPg35AcTHQFEaK4FE5/jIS5KRlUMvB/4SX/naSPj8TXPKFGO?= =?us-ascii?Q?aLLK1uUsjbXfnrHtnYVPxIT3ULxl6ViNBCghQqk49zNrHP9OFWOvhczrmOBM?= =?us-ascii?Q?HIqc+tUCtBudR1ROevcMEF5/ogLvlSH3FGxYokrx6Oik2mNTB4OIdYxJZzUq?= =?us-ascii?Q?zSZBxsEVgF4FHPIouOAa2pO10OSgzvjF4DtK6jb+nmx3FsA7CCSOv4Hw7k52?= =?us-ascii?Q?Ouei4/ssn6Kb+46b2RPMt+mCLZJjj6t4bqy9UV7xeLkFcRXW9+dSDHjFYbny?= =?us-ascii?Q?c+TpeCMhv8to+LJjToxDz/i6T8DC2122thKfPIgwi8RDky9ZXfZKKkmP/zM8?= =?us-ascii?Q?5g/ABd1o3j4B/Rbi0938pFN2EXMXcPmBgLcZF941tyi4L85oqeznqWL3Eh6m?= =?us-ascii?Q?HKb+fSEMPWvFVnVqHMzbgAGXANijZHvGEw973w3GtndDGEH4tinpqLjD5IHV?= =?us-ascii?Q?D4Mn5qqVoZ3aUCFeIil9L1HvmIDozSrwVhpy5+5Lhc5gsPg9h/yDaFqXWjIL?= =?us-ascii?Q?k7c88Tl1OfHiqSP6wmvbnAxnKH1zg4pE2qKATvcVfsE3SrB7iKY2SN7ayL2A?= =?us-ascii?Q?9JKz4R8jftcmkbrzetD8PBJadHpz6eKsI0SlYn4aOnAmOhAcNp2TGhFKUyHp?= =?us-ascii?Q?wTLLvFk51UOrvjFfkjoghb7hIdh8DCxAB+vkY1vkqC0qYCDQAMZoRaNIMoWD?= =?us-ascii?Q?h54WQjfxceoCYwZQtWIhAioVyejG65pc8W3AeIkufIMV1oZS6rRh0P3AMRS6?= =?us-ascii?Q?l3FVaniyP6bT3Y4L/PJI7gKmpOfXIf4xERtVXqwy1xJ657wkBhZOvxKvRtOl?= =?us-ascii?Q?5tOUn/D3W/nDeboG901Q1ns9+0i2d2KRRyN8bjarGJTgz5FhzV93D7BRaBib?= =?us-ascii?Q?YVAF4dN7EnbvH0Ivwo6B0ag=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 8qqwVdMCjUq1Z+z46opuOO0rFMH07WRMc/7K3OcDVPT6vga5MSMXsWj1IEDhAD4Ec4LghxHESXAOzGCunl8Fr1WHnNv2Ep2WKIGIDXEOG7F06Qy4G06FmN8bxe4K9gAXpPdptqZLXcIJwu1MMHtXvAIRTy37M71WShvyktdnWPNJJXJfuSo0d9sURCsN5G0qy/IAJXJ5HJp8syDyCh15a7pkbLg1FwpX1wM+a3bsF/aFF8KhdCc8kXNQNKndvoe1QI096f3QkY7Q1sTocfGcrHpGwsO9rYkX3O1kf0m6V93+ZcY7d5VhARiKcGtpo3P+/dCQ7N/RLNYe8sov89H4v0AtiNekJ1bTq8gxFKNS7WEFnQLzF9h1B0mQVi/sKkpumv8lSO3nBcunRJvawiC/Zq3NLQRDx0JSft3f342MgxUYIkMNvj/PuYnfjdAppvr5vLTHA0WAWAJdTX5swDo9+A40V/t+STjHVa1oCHOPjIHs0Wgdxd31ojgQl9+alKMzTkVMuODRJ7EYY13unEUZKog6lNCHBXH0LnojcCIYpRWDNCEjaWLhkPn/T2PggawJjZJVg3xoYVhdTiajAu8RPf1nLLuq+rD3RoPJ8t+h8lguIZhjwzXSIEuJNFGlRZcumRxL0fH81lHh95/J6UnOiHvAYEan3IrVCYo4QfaqJSEOy1MN7OLij5X5SAWWEKxtNuPArP/L5J2z1sEBxnUHQvvkcyHjrBeegQxQW9AXQu4LQBwPr8yI67Vk3jegvz5h/tcHm0kK9MHiyXroSBrXMDwx+Ef1ncHK31CzHEamDiKWOmAJvtWzguMO6V/ufkQO X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 66f49efc-32c2-41b1-8a90-08db3c3a0e01 X-MS-Exchange-CrossTenant-AuthSource: BN6PR1001MB2340.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2023 16:13:44.6843 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hEbx0H0DZYhGFdQuTFpM945CK5I7Bk3YRFmt2fhIJ7nBjWPiiFHRfLWOMNb6ucJ/8ewWBYD2TPYr4LMI8aVy/6EF5vCnsXPzy/1dbtGZmmM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR10MB7502 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-13_11,2023-04-13_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 malwarescore=0 suspectscore=0 bulkscore=0 spamscore=0 phishscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304130145 X-Proofpoint-GUID: uBy5325Mar-kRLXReufrJAanAaaefkJP X-Proofpoint-ORIG-GUID: uBy5325Mar-kRLXReufrJAanAaaefkJP X-Spam-Status: No, score=-11.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hi Adhemerval, Sorry for taking a while to reply to you. I compiled your patch with the particular example that lead me in the tunable propose, it does not seem to optimize in that particular case. Looking at the code, I realized that the condition would most likely never allow for it to be madvised unless you would have a guardsize == thpsize, which appears irrealistic. + if ((uintptr_t) mem % thpsize != 0 (is not aligned) + || size % thpsize != 0 (1) + || (size - guardsize) % thpsize != 0) (2) + return 0; If (1) is not true then (2) is likely to be. I aggree that for the scenarios where the hugepages would never be used, we should advise the kernel not to bloat it. In any case I am not fully sure you can always predict that. Lastly, I would say that the tunable would still be usable. For example, when the application creates sort lived threads but allocates hugepage size stacks to control allignment. Regards, Cupertino Adhemerval Zanella Netto writes: > On 12/04/23 05:53, Cupertino Miranda wrote: >>> >>> So this patch is LGTM, and I will install this shortly. >>> >>> I also discussed on the same call if it would be better to make the m >>> advise the *default* behavior if the pthread stack usage will always ended >>> up requiring the kernel to split up to use default pages, i.e: >>> >>> 1. THP (/sys/kernel/mm/transparent_hugepage/enabled) is set to >>> 'always'. >>> >>> 2. The stack size is multiple of THP size >>> (/sys/kernel/mm/transparent_hugepage/hpage_pmd_size). >>> >>> 3. And if stack size minus guard pages is still multiple of THP >>> size ((stack_size - guard_size) % thp_size == 0). >>> >>> It does not mean that the stack will automatically backup by THP, but >>> it also means that depending of the process VMA it might generate some >>> RSS waste once kernel decides to use THP for the stack. And it should >>> also make the tunables not required. >>> >>> [1] https://sourceware.org/glibc/wiki/PatchworkReviewMeetings >>> [2] https://bugs.openjdk.org/browse/JDK-8303215?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&showAll=true >>> [3] https://lore.kernel.org/linux-mm/278ec047-4c5d-ab71-de36-094dbed4067c@redhat.com/T/ > > I implemented my idea above, which should cover the issue you brought without > the need of the extra tunable. It seems that if the kernel can not keep track > of the touch 'subpages' once THP is used on the stack allocation, it should be > always an improvement to madvise (MADV_NOHUGEPAGE). > > What do you think? > > --- > > [PATCH] nptl: Disable THP on thread stack if it incurs in large RSS usage > > If the Transparent Huge Page (THP) is set as 'always', the resulting > address and the stack size are multiple of THP size, kernel may use > THP for the thread stack. However, if the guard page size is not > multiple of THP, once it is mprotect the allocate range could no > longer be served with THP and then kernel will revert back using > default page sizes. > > However, the kernel might also not keep track of the offsets within > the THP that has been touched and need to reside on the memory. It > will then keep all the small pages, thus using much more memory than > required. In this scenario, it is better to just madvise that not > use huge pages and avoid the memory bloat. > > The __malloc_default_thp_pagesize and __malloc_thp_mode now cache > the obtained value to avoid require read and parse the kernel > information on each thread creation (if system change its setting, > the process will not be able to adjust it). > > Checked on x86_64-linux-gnu. > --- > nptl/allocatestack.c | 32 +++++++++++++++ > sysdeps/generic/malloc-hugepages.h | 1 + > sysdeps/unix/sysv/linux/malloc-hugepages.c | 46 ++++++++++++++++++---- > 3 files changed, 72 insertions(+), 7 deletions(-) > > diff --git a/nptl/allocatestack.c b/nptl/allocatestack.c > index c7adbccd6f..d197edf2e9 100644 > --- a/nptl/allocatestack.c > +++ b/nptl/allocatestack.c > @@ -33,6 +33,7 @@ > #include > #include > #include > +#include > > /* Default alignment of stack. */ > #ifndef STACK_ALIGN > @@ -206,6 +207,33 @@ advise_stack_range (void *mem, size_t size, uintptr_t pd, size_t guardsize) > #endif > } > > +/* If the Transparent Huge Page (THP) is set as 'always', the resulting > + address and the stack size are multiple of THP size, kernel may use THP for > + the thread stack. However, if the guard page size is not multiple of THP, > + once it is mprotect the allocate range could no longer be served with THP > + and then kernel will revert back using default page sizes. > + > + However, the kernel might also not keep track of the offsets within the THP > + that has been touched and need to reside on the memory. It will then keep > + all the small pages, thus using much more memory than required. In this > + scenario, it is better to just madvise that not use huge pages and avoid > + the memory bloat. */ > +static __always_inline int > +advise_thp (void *mem, size_t size, size_t guardsize) > +{ > + enum malloc_thp_mode_t thpmode = __malloc_thp_mode (); > + if (thpmode != malloc_thp_mode_always) > + return 0; > + > + unsigned long int thpsize = __malloc_default_thp_pagesize (); > + if ((uintptr_t) mem % thpsize != 0 > + || size % thpsize != 0 > + || (size - guardsize) % thpsize != 0) > + return 0; > + > + return __madvise (mem, size, MADV_NOHUGEPAGE); > +} > + > /* Returns a usable stack for a new thread either by allocating a > new stack or reusing a cached stack of sufficient size. > ATTR must be non-NULL and point to a valid pthread_attr. > @@ -373,6 +401,10 @@ allocate_stack (const struct pthread_attr *attr, struct pthread **pdp, > So we can never get a null pointer back from mmap. */ > assert (mem != NULL); > > + int r = advise_thp (mem, size, guardsize); > + if (r != 0) > + return r; > + > /* Place the thread descriptor at the end of the stack. */ > #if TLS_TCB_AT_TP > pd = (struct pthread *) ((((uintptr_t) mem + size) > diff --git a/sysdeps/generic/malloc-hugepages.h b/sysdeps/generic/malloc-hugepages.h > index d68b85630c..21d4844bc4 100644 > --- a/sysdeps/generic/malloc-hugepages.h > +++ b/sysdeps/generic/malloc-hugepages.h > @@ -26,6 +26,7 @@ unsigned long int __malloc_default_thp_pagesize (void) attribute_hidden; > > enum malloc_thp_mode_t > { > + malloc_thp_mode_unknown, > malloc_thp_mode_always, > malloc_thp_mode_madvise, > malloc_thp_mode_never, > diff --git a/sysdeps/unix/sysv/linux/malloc-hugepages.c b/sysdeps/unix/sysv/linux/malloc-hugepages.c > index 683d68c327..5954dd13f6 100644 > --- a/sysdeps/unix/sysv/linux/malloc-hugepages.c > +++ b/sysdeps/unix/sysv/linux/malloc-hugepages.c > @@ -22,19 +22,33 @@ > #include > #include > > +/* The __malloc_thp_mode is called only in single-thread mode, either in > + malloc initialization or pthread creation. */ > +static unsigned long int thp_pagesize = -1; > + > unsigned long int > __malloc_default_thp_pagesize (void) > { > + unsigned long int size = atomic_load_relaxed (&thp_pagesize); > + if (size != -1) > + return size; > + > int fd = __open64_nocancel ( > "/sys/kernel/mm/transparent_hugepage/hpage_pmd_size", O_RDONLY); > if (fd == -1) > - return 0; > + { > + atomic_store_relaxed (&thp_pagesize, 0); > + return 0; > + } > > char str[INT_BUFSIZE_BOUND (unsigned long int)]; > ssize_t s = __read_nocancel (fd, str, sizeof (str)); > __close_nocancel (fd); > if (s < 0) > - return 0; > + { > + atomic_store_relaxed (&thp_pagesize, 0); > + return 0; > + } > > unsigned long int r = 0; > for (ssize_t i = 0; i < s; i++) > @@ -44,16 +58,28 @@ __malloc_default_thp_pagesize (void) > r *= 10; > r += str[i] - '0'; > } > + atomic_store_relaxed (&thp_pagesize, r); > return r; > } > > +/* The __malloc_thp_mode is called only in single-thread mode, either in > + malloc initialization or pthread creation. */ > +static enum malloc_thp_mode_t thp_mode = malloc_thp_mode_unknown; > + > enum malloc_thp_mode_t > __malloc_thp_mode (void) > { > + enum malloc_thp_mode_t mode = atomic_load_relaxed (&thp_mode); > + if (mode != malloc_thp_mode_unknown) > + return mode; > + > int fd = __open64_nocancel ("/sys/kernel/mm/transparent_hugepage/enabled", > O_RDONLY); > if (fd == -1) > - return malloc_thp_mode_not_supported; > + { > + atomic_store_relaxed (&thp_mode, malloc_thp_mode_not_supported); > + return malloc_thp_mode_not_supported; > + } > > static const char mode_always[] = "[always] madvise never\n"; > static const char mode_madvise[] = "always [madvise] never\n"; > @@ -66,13 +92,19 @@ __malloc_thp_mode (void) > if (s == sizeof (mode_always) - 1) > { > if (strcmp (str, mode_always) == 0) > - return malloc_thp_mode_always; > + mode = malloc_thp_mode_always; > else if (strcmp (str, mode_madvise) == 0) > - return malloc_thp_mode_madvise; > + mode = malloc_thp_mode_madvise; > else if (strcmp (str, mode_never) == 0) > - return malloc_thp_mode_never; > + mode = malloc_thp_mode_never; > + else > + mode = malloc_thp_mode_not_supported; > } > - return malloc_thp_mode_not_supported; > + else > + mode = malloc_thp_mode_not_supported; > + > + atomic_store_relaxed (&thp_mode, mode); > + return mode; > } > > static size_t