From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2059.outbound.protection.outlook.com [40.107.94.59]) by sourceware.org (Postfix) with ESMTPS id 298F93858C54 for ; Wed, 8 Mar 2023 10:24:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 298F93858C54 Authentication-Results: sourceware.org; dmarc=pass (p=quarantine dis=none) header.from=azul.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=azul.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=S2KgX10dbw+BK4bYCx9KQNK/R8Om4jJrzj4kUjMJ8E8lBz5ODUB7kY5AlZ940VD2mlBhqitv5Go4PGrchnZjtc+pOLgkz7a3rIKIyQKrBHY39ww+GTGF0aOvxrkMMbAiGuANNbMt0PaWBINpXUzbsWc4NX7oF8EgxK/eDgk9lA5K9uMKEx6GUfHBUAqBVjhDGXDPzZjf9R47Bk6X5zW2Q2UqB3tpRQgxkSB2e0tqzvtiE0XXTnGgbAtpkHuhWofvNygGCcUow4sw68KUJmQaIvohDEKP+A5gESdBs2VrOKnUDkhzrArnxsbGU7FxntI2NzecZXx2wuRMRy4OH1xE6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=X0EgsOftLWP4vEItzIRgWReanIaqsNa24BEyy1dgaZQ=; b=lgyF1QZ5jDgbYWWOnGnTBQUOqdBTtgr1BCYCOuKDNM7RgVGguYh68VUSMaqHJVSS3STtmmwNexV0PeaL/W29pWq/VNq9c4yEYW4N0+QPLaJkccmZ9Bexf/ZK4dpDPvnm6dXd8Ay7hGWinPDypiEO/TxKC8ZorfxwTergQ8AiLAAhtDJR5NtL7RgiKNYqkQKS+vH1/9uM0n0S2DA4Q8sdnRW48KaakFPT5QQVt+1ZFC7z8azlgMx9RH8+0WYsSiBOl7EfN653jRJwvbx7H1ab3i7OfOM+/KEDW+qe1Hx+vHWd77e7nTRibXPUKMLcKCfEpglrEBIlX4XHiwWDj7mk2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=azul.com; dmarc=pass action=none header.from=azul.com; dkim=pass header.d=azul.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=azul.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=X0EgsOftLWP4vEItzIRgWReanIaqsNa24BEyy1dgaZQ=; b=aAbjEwIpWdgWWQ9tZcpGKNSuRGQrYkPB+Wsw6cRb/Hw53vY7ObB5Ta35zvqm05P1Q93VTyYauqLGtVnGd7y1SfRt7xEOELbHUlfxiv+oce/AH0QdyGLYN/Z5zh8PxOg7aoaanntMQb6JmxC44VJIcd8tOcCCEjSJb4xzg8qtnof0GsrYK+PZqIXTOBtyQTHGzOruNBWztpVbN7hIQOtn7UTfqO8OCqxq5CKdOgRzT3polF3GqXVDN+KppCYXqTFyMjWddH1yY/X2vfIlljmzmz2llarWw43LX0f3vsoxorlHp0K59xb+VV06TnpFTU1mIOgIvLVyW+r0XWMClhY/GQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=azul.com; Received: from DM6PR11MB4073.namprd11.prod.outlook.com (2603:10b6:5:19f::22) by SJ1PR11MB6274.namprd11.prod.outlook.com (2603:10b6:a03:457::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.17; Wed, 8 Mar 2023 10:24:02 +0000 Received: from DM6PR11MB4073.namprd11.prod.outlook.com ([fe80::8476:1cf6:afeb:c285]) by DM6PR11MB4073.namprd11.prod.outlook.com ([fe80::8476:1cf6:afeb:c285%3]) with mapi id 15.20.6156.028; Wed, 8 Mar 2023 10:24:02 +0000 Date: Wed, 8 Mar 2023 18:23:55 +0800 From: Jan Kratochvil To: Florian Weimer Cc: libc-alpha@sourceware.org, Anton Kozlov , Adhemerval Zanella Netto Subject: Re: [PATCH] RFC: Provide a function to reset IFUNC PLTs Message-ID: References: <87v8jdq7ht.fsf@oldenburg.str.redhat.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87v8jdq7ht.fsf@oldenburg.str.redhat.com> User-Agent: Mutt/2.2.7 (2022-08-07) X-ClientProxiedBy: VI1PR0601CA0025.eurprd06.prod.outlook.com (2603:10a6:800:1e::35) To DM6PR11MB4073.namprd11.prod.outlook.com (2603:10b6:5:19f::22) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR11MB4073:EE_|SJ1PR11MB6274:EE_ X-MS-Office365-Filtering-Correlation-Id: 04bebb63-47c2-46b2-9231-08db1fbf3cc8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JtzEZxotXYtLTp/v+gghfrxO88mWyuIG8WIjE9XczYQGsQIhhwu405oY0auFoo09cgch5Hv7e4PPSDMrBb8ElVaqMTQdzw8LKHuHj2Mp/rfJp9CJoliqItDFT3+1ee214Bz3T0tNKuW26M5JHbYS9z/sO5W0bvWUfYdwyrh67Lb21sUKJjQChL5eMcudgvc4YmgAi8KtoekrAMapCHwOOvZe/kd7mjH19gvq3efZktU5fDVNf4ChCGlatBjPM3Sdz0eeuOo5uXvI1mgxcdbcsLEP/MRL7SljAWSYPsqrQPJ5/CZ1GAvbYmJxYO1xPAHM4FtW65REi4wry3tlPYywRFA3VMu/+HLepH6ZA+JJ2HFF+DeRT7J+sFpIkD2Gcna7I4yizrWDC+1hJDRAqcN2rQVkyeZCfOpyId0m/BG36wYDPEhpehXU3iJDL/froF5rLE9pAMxYOq2RwejpHr1C9dZpsiBfSZeCx5KNj2AMnm3V87tM6jrhaEG8dq0Wa07LqDeyTpDfrsWrL/50hgJaV6qYbw/KT4tZ7tvJOwP+gbYRVS0aUcuRv2aFxWPirwvTYDinmPLlVygljQR8+Vl47pUouu9WNR7HMS/I8o01bS1OINuf4GAtfMAOcH+YXGoWVC56IYNY9fReDfkcp/N1ebYk9DPjJPRo3Dr4qVEa8kRtOqXQo7bXB2C8ci/a9lt8RBuFufQ4SbgrdjNyp21Ab/vMgB7vvxYtr1c4OSzTPAo= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR11MB4073.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(376002)(39850400004)(366004)(136003)(396003)(346002)(451199018)(8936002)(5660300002)(6506007)(9686003)(6512007)(6666004)(38100700002)(83380400001)(186003)(54906003)(86362001)(316002)(41300700001)(66556008)(6916009)(8676002)(66476007)(66946007)(6486002)(966005)(478600001)(2906002)(4326008)(67856001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?5OWneDq3tRts8xpse3TwS3jf5ZbXT4DyNu20pvaGOAd2MSIL2t6T3wfjVxM9?= =?us-ascii?Q?tr1ich65A5nXUSGDMptuqOW3sdAiHpF22jne+oyJsXebIA+6FQjO88RczX27?= =?us-ascii?Q?YWWHwhotVh4HEhT7vCOISJ3irxiFIWQg/bXGiYKq7ZzC3DvFvCPOFFhcFVmW?= =?us-ascii?Q?yee90K+3/C7wSDvD9Mfo4ojKozSWkMdGuKtYdQIdjwUOhmcMLwT2Z1CePAxA?= =?us-ascii?Q?5tJdBYmck0tLeG/73zLAmJRotADca4U+khB27RR+9nfJZBRV82IW2kX82mnY?= =?us-ascii?Q?st0bEclgRjCXRDZNs3hmqOl5wx+VbicyXosUjz2JTac8eHi0l2ZXFEjp9O1T?= =?us-ascii?Q?9K23YTWmssXZzIILn4PQPl8nc4yVTfXwbxj+6WQHiVL7XMd7vadj4AcwNHuM?= =?us-ascii?Q?NnpJYC7ZkNJ36r2GBsOy1SXaHTHG+HTcHck+IryZhe/ea1FbZY4/Rfg8ZoCj?= =?us-ascii?Q?KHS/vev1kAYiOW94vuUekZajUlG2Ggych2MnckRHhDvvzao85NesSLdx3uFg?= =?us-ascii?Q?f5ahIJAu4LwuntkWOh6RA2Dx7wbFHB+muTwt+rGx/IfuOY6E1gGjM5PRqe4l?= =?us-ascii?Q?XZrLeC9F228lsQn2F/dSNgtSsN7I4fqzTQgYQTqgQk6KpAtwejMqN/2CSo45?= =?us-ascii?Q?d9jttCWvAvzINOYKbIshChWGdQ96UJLVV45EDnowh9Lea6peu0HOscvWd/Lq?= =?us-ascii?Q?q0j+zg8OETqmBkmGWsVUdybfK6yx9uvRJ2fK9GP0WbdYzRoO4DeUEEb9Ks3X?= =?us-ascii?Q?/z4lNSQzc4PsD8ilt28/Vvd7NFy2D9/DO7OSwrefnGBp9WLJdtiLysGkwHOr?= =?us-ascii?Q?S9LvTxdsdo7DWUFyfz79ucGEUONV9dawxeuT1aYvJSamU9LQLprl8e2olrF4?= =?us-ascii?Q?HHu9EsUnXHzY3xcqiA4hovwbO6rDiAJVlRzDYXSRBGouiYSHYvNq0KhJPfds?= =?us-ascii?Q?RVGVJR9+w5W7Civs7wWBB6NqYFHRLJ8F2BmK+mBHVpvupqGPK+1LrbZGaHOa?= =?us-ascii?Q?+ZnYb0C0b2N6pp527sb2ZfzRUFXd63Gyl24wgoOYh2ted0IvzHdSf3LaVLDa?= =?us-ascii?Q?AJpM+uKXPWTR4VnZ2OnBgEgwnTDN4rVGRLu0ftEoCS+ff6xTwWMYGnbFACMi?= =?us-ascii?Q?J7RfZxF62EZpUriO35X+oXbG7OuTxBLBUN+ldTf12pGvHMc1tvcGD9jF3Neo?= =?us-ascii?Q?dKSM2KQ0w+YP7Qi0HGyA4IEEt4glsUPJqIGn1q6B6iHR1YdfPnYkg/EQIZjf?= =?us-ascii?Q?hnB0ptsD/82b94cbYzMphDbktbxDBxq4ABkx52GohUDanncYx6utjkscHZxO?= =?us-ascii?Q?iIlUrTLN5RJW8KBzLTO1sg3AMhEQ7FP26xbZ1pBIzeb2Sz1+NFAK61U3Ei31?= =?us-ascii?Q?X4oGIe8iKoCCL5hHVqnv7ZVhZOfTxCXUPzhWIoeU9ky7ZxpjLE3UzpsXZpET?= =?us-ascii?Q?4rTetGSYUxYN4zj4bGbSzVyq9Z4itt+Tppo8bjCqBmX6VcqVklBjGZ1ZwehI?= =?us-ascii?Q?HFTLXZg69o+Ox64KdnJSjMYU0sCN3z+x4eHJiYpvCBdJMXevqXzHNVNsvWVs?= =?us-ascii?Q?xd3vEjypIHqzm2NXS7bonY+Mn7otLKxK6570c6AAG6FWi8D3JCvsGzgSw5IL?= =?us-ascii?Q?FA=3D=3D?= X-OriginatorOrg: azul.com X-MS-Exchange-CrossTenant-Network-Message-Id: 04bebb63-47c2-46b2-9231-08db1fbf3cc8 X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4073.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Mar 2023 10:24:02.4705 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: c480eb31-2b17-43d7-b4c7-9bcb20cc4bf2 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: c7Xc2lZ4gNYTg7WIzSqgsnl/1TpgP86KteaM2PNU1haMNlS+IGGN+ybxOhUYaI3pEmqs0w4nWUwufG8uCnn/8g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR11MB6274 X-Spam-Status: No, score=-5.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Tue, 07 Mar 2023 16:40:46 +0800, Florian Weimer wrote: > Do you have a high-level overview how this is supposed to work? Is CRIU > expected to call the new _dl_reset_ifunc symbol after restore? Yes, it should be called by CRIU. The current implementation was calling it from CRaC restore (which uses CRIU underneath) but I agree it is not the proper placement: https://github.com/openjdk/crac/pull/41/files#diff-aeec57d804d56002f26a85359fc4ac8b48cfc249d57c656a30a63fc6bf3457adR6274 The function is called differently there as the IFUNC reset implementation is outside of glibc there. > Use of is somewhat rare. Not even GCC uses it > AFAIK. It has its own cached CPU data used for target clones and > similar features. Re-running those IFUNC resolvers will just give the > same results. I do not use anywhere, why do you mention it? I am doing there memset((volatile uint8_t *)cpu_features, 0, sizeof(*cpu_features)); ... _dl_x86_init_cpu_features(); to prevent the cache of CPU data. And it does work. > Distributions have started to deploy alternate builds in > glibc-hwcaps/x86-64-v3 directories: > > openSUSE Tumbleweed gains optional x86-64-v3 optimization > > > This might be a common situation fairly soon. This is not IFUNC-based > at all, so any IFUNC-based approach is not going to help with that. What about some "dlreopen" to replace the variant of glibc? > Practically speaking, I think cluster heterogeneity needs to be hidden > at the cluster level. The question is how to implement it. I have found https://github.com/ddcc/libcpuidoverride but it is not compatible with latest glibc (maybe its loader code needs to implement ELF relocations, it would be definitely fixable). Another option is GLIBC_TUNABLES=glibc.cpu.hwcaps=... although then one has to translate cpuid feature bits into the glibc names like SSE4_2 etc. Also re-exec() after setenv("GLIBC_TUNABLES") is not too nice. There are multiple projects solving the same problem such as: https://github.com/eclipse-openj9/openj9/issues/14253 So I was thinking some more standardized solution may be already appropriate. Thanks, Jan