From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2080.outbound.protection.outlook.com [40.107.20.80]) by sourceware.org (Postfix) with ESMTPS id CA7BA3858CD1 for ; Tue, 14 Nov 2023 11:40:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org CA7BA3858CD1 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org CA7BA3858CD1 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.20.80 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1699962005; cv=pass; b=L4RGWUwcrL8dtu2y25+R4GhZdgz+0R4+sgSPpPNpUaDzfyUj6OUutpTcRwC++V8tpP+r24H0ubSGR8alPFTokqfCn9g70Hi/bVmNOgsqHYb/aj6lp5EG2itdpzZi8vFbyzEsZACc+oy58ZG8bnBFCoAd1l07mCxGUwOgD9JTeEg= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1699962005; c=relaxed/simple; bh=wxbj3jEEdRkWzPDwwt7/HrOC5XGIASIYVAyEmIrevlA=; h=DKIM-Signature:DKIM-Signature:Date:From:To:Subject:Message-ID: MIME-Version; b=cNK2f/suBTaFSc7RsV1nuwmAOAsR3iw6Kfk5NvX7nux3von++4IbwX9seNFZgi6i7RgkGmKm40xrw518i8IcE5EISFwBXSVd35iTFm13vwmbEIiImhp28giCOf23GTl+aRABKaJjHPlO9ffGk5oUozW9TdawYPmDuzbAw8ED8Uo= ARC-Authentication-Results: i=3; server2.sourceware.org ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=hJlvcskRk613ZlB5/A7i173PsWFcobtBbGnOR9q+UmSukij6miYQVgo/A9tZVhv1GbyS6PsAochm7+1CX3/dZPjipVT9NVUvy5nahDum0vc5907ZZiwvUyLEW29sghGWNvFkC8bWYQDBuJ3crhtntgwVb13oqgpgFNgmTqmbrRQzs1nVJ5RY7phPymPGp0hOQHcB9yJ6MvCotL3Irxd5tGzO2JDQaPjrAq2r33sDmc06KftjcvSSNNIj26JbODt32QiCcmWGNfR2+6h+C0Ld6BBwkQrFvtwo7PREO/Hc9EiHg5dF8mBy2I+u/wZzf/5jH1vl4y/soIujoAP1cKoldA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lRy39OGR6K4Au27sg8L59y9XgrDmnyeCwzKwLwBmRC0=; b=gxb95KHi54DdpJizLnGzJblhNq5cGewkyvwSkCI0cC5sv39UjpmyTs9CGL5f8JaNrcjEadSUtVcXC6efnMUmKv0caVL6YgsvW5qQVOk/sZOpym4dtSUGCnAj+cDpnzuiNPIAg+zZyAJhANseyPP6VkMJ6BXBG0vUjAfeivBEQGQKvBnYR6ewNMo/wQzZFFmtWRfHHOxEEmVdOekGclNFJt7S670bjaj+u7xzEUtlS4Nh+BB5DrwOTPGyDeP79E70mqp1THoqsDoBbXLBKLz5mc48uAt1+MX7L07RGrquAFje+sTV1TFM4Ix5J0iwW/gtV9wlF560ZSnYlFlYGQQ70w== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lRy39OGR6K4Au27sg8L59y9XgrDmnyeCwzKwLwBmRC0=; b=EhHnmY7GU4/sVCjS57n9n62uhJB+lxmd3kEbt6sM78fT8FvbCMtZm0+2oMTB472itWuaNrpiSdvAam0lodlCJHNgPCw+rCfX8yEvAym+UKaFHcPKEgVsBkFyfIhrOCEMiBFAylFpWTw9/709FIU8UpFDQs+qruGJfIpbLeYjSDA= Received: from AS9PR06CA0616.eurprd06.prod.outlook.com (2603:10a6:20b:46e::25) by AS8PR08MB9502.eurprd08.prod.outlook.com (2603:10a6:20b:61e::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6977.29; Tue, 14 Nov 2023 11:40:01 +0000 Received: from AM2PEPF0001C70B.eurprd05.prod.outlook.com (2603:10a6:20b:46e:cafe::d4) by AS9PR06CA0616.outlook.office365.com (2603:10a6:20b:46e::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6977.31 via Frontend Transport; Tue, 14 Nov 2023 11:40:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM2PEPF0001C70B.mail.protection.outlook.com (10.167.16.199) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7002.14 via Frontend Transport; Tue, 14 Nov 2023 11:40:01 +0000 Received: ("Tessian outbound 26ee1d40577c:v228"); Tue, 14 Nov 2023 11:40:00 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 378289854a9e99db X-CR-MTA-TID: 64aa7808 Received: from 1392ffb2e755.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 27828173-4DA6-4F2E-8EE7-1E752B663951.1; Tue, 14 Nov 2023 11:39:54 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 1392ffb2e755.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 14 Nov 2023 11:39:54 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AtofxRoI309dBR2dnoaao0t3rCgaMJjOTw1L/OX/5nV9fYBoaz4YkuQXmXGyC//+0rr0/mpLgszaDktEFPWnXxW4SlOVDpVPhpShAMTDj5l/0jy9sJ4X9+g+JhNZNMZu26YA1adsF1+1SNCI9pQ9c7yqFwPeBwBoTB0eGM7jew41WtjAkveIP3y8NkD87LT9sGn8uYgq9KrSYeNujg1p68qOhdUBO1BUqjzrLId5mhyk5BxYb2kaLLp9UfM5SJ3UiJmNgscd6117YzNbD+LMuJxOaqvHMIu2a8cbsnQEjRTI/Q5gBa7ZJ9yy2YtMMcxYhCb/keO/YkGrc+infhqSbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lRy39OGR6K4Au27sg8L59y9XgrDmnyeCwzKwLwBmRC0=; b=flWx4U2hcEFgqde6Jo3yRBcdKU2HvkZ1bbMlyxebNoOo8gojBvGKM3Vk8ijYBuHxJYtG3P7dgbcb/GwbSmBKKtHnWpQUnztJCAsAO0UMk9daQtbPDSxql1h5EtMMtDgnN0YXigPCnvdVHAYr5YpwjhNOzhaDn0D9sYzMs6XVcL/Rsd/6H3S+F+fKblqVr6O75UWX1PEvJc9mGsVitdJF6HMBN+3PjzfO0bsCjMwQ8evNND9dZG2Y/9BGoFxfpTS8RpzdGTmYzDEoUhrrUsC2s4Winw7bbf3L7O9+1f52BFTHa3HtI372AZN3HBuyCe6V2HkdLue56PRV3KuHRyWl7Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lRy39OGR6K4Au27sg8L59y9XgrDmnyeCwzKwLwBmRC0=; b=EhHnmY7GU4/sVCjS57n9n62uhJB+lxmd3kEbt6sM78fT8FvbCMtZm0+2oMTB472itWuaNrpiSdvAam0lodlCJHNgPCw+rCfX8yEvAym+UKaFHcPKEgVsBkFyfIhrOCEMiBFAylFpWTw9/709FIU8UpFDQs+qruGJfIpbLeYjSDA= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from DB9PR08MB7179.eurprd08.prod.outlook.com (2603:10a6:10:2cc::19) by AS8PR08MB10028.eurprd08.prod.outlook.com (2603:10a6:20b:63a::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6977.31; Tue, 14 Nov 2023 11:39:53 +0000 Received: from DB9PR08MB7179.eurprd08.prod.outlook.com ([fe80::a991:ac53:e218:e554]) by DB9PR08MB7179.eurprd08.prod.outlook.com ([fe80::a991:ac53:e218:e554%3]) with mapi id 15.20.6977.029; Tue, 14 Nov 2023 11:39:53 +0000 Date: Tue, 14 Nov 2023 11:39:37 +0000 From: Szabolcs Nagy To: Kozlukov Sergei , "libc-help@sourceware.org" Subject: Re: A public interface for library search Message-ID: References: Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: LO2P123CA0073.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:138::6) To DB9PR08MB7179.eurprd08.prod.outlook.com (2603:10a6:10:2cc::19) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: DB9PR08MB7179:EE_|AS8PR08MB10028:EE_|AM2PEPF0001C70B:EE_|AS8PR08MB9502:EE_ X-MS-Office365-Filtering-Correlation-Id: 8129198c-745a-4f15-c517-08dbe5066fca x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: G3joSwzl7fqril5QMYVtiJgdFCM1ftKaTegJMVT5mdfMHQYMItnfiSvNGU75WxmckIc0ZLiBQabIu/ptNdZiAOC28comgjAgYt3bZIjiL3T70iCt0Vz8eh01ikbHm8jieLdDMHBrtMyf1054bBSxE0rkJoLsEJHRiX5QUrsuzHJp+g6zAk5zr2oP73KUDJQeD+rRfydgcfDQIIw1jaIqHlMCCbTgMWEzVQulJS5LoHLmDL5n4PKT1xpqScX+6COn0ekJpIXkswWa+pyrgYIsQYaQ/NhQPA4JVbmmYYyCPZFd2MFlas+Wm5n+1E7VIqWV634wjCnXP6JPRL7mjRE5Duzrca0DIAg5ArDrWeL9CAGwKmoMcFxNozAFAJskXFWlkeSZO8uGSLW/5K6XXUSTL1WQJ8GsrB722zoaGyJewnBeIDNZGbylH28Ifk+l31ZZ5Qr4UCP8bKMlMNCUr0QD/DQ7TI5/EufMo5ck+YRb6Spz17glTgKdDuMFRJ+iOxZhJE6k5KJ6vSBLOMdaMoFdxAYnqrgmgmX93YPoHEcnEcnB6+P7lm/D/j1miqoBULHQKmL3xOc6N6yRJ8pujTr1vg== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB7179.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(396003)(366004)(39860400002)(346002)(136003)(376002)(230922051799003)(186009)(1800799009)(451199024)(64100799003)(83380400001)(6512007)(26005)(6666004)(6506007)(2616005)(316002)(66556008)(66476007)(66946007)(110136005)(38100700002)(8676002)(36756003)(8936002)(2906002)(41300700001)(44832011)(5660300002)(86362001)(478600001)(966005)(6486002);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB10028 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM2PEPF0001C70B.eurprd05.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 2683fb27-e69e-4c76-abc9-08dbe5066ac7 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Hc9uy9aUaMtFPMZtkO5gCeCkOT0H7+HMX9NBYR9uY6HrJRM8N2jsnX1nhmWee3xM+BNu7AF91oo5f2Fz+mIj2VMge/NIflwpZqgFycri0ZnlxQf9y3lx9Ul8+O3dCAYbNYTqv7qUhPRbNF7BSoa6AZ3aUURUdzvygzMUzMzVTz+4nhNh/FZgbU+0T/ySW3Iqo3vJ63qYhzBypFDXV0eVp8aGUNQbfGxD/H39yM/8MGfXvuiXKKmo5ybr2dlHeiiXdXbiQ39kzQzEjDExTsXDeThv80GuOKmXG1E51zEy6f/GJ0MAwdXUcFyHCwtOBI3ssEJfsdyr9Z8J+9hEWLNhZaHLxSZBXkOiY4Ebug088SeItNAKMyYxXY9UI9LSkqa7FBWvH3AQGBT3P7fFSUwwmaAjGUyamc8+u1vlqT7keaHGAcnCsjNzWq6faFYoXFu6rf98HKfw9TZSTKO2JqX1eN1akyeO4C5eBFocS6q2+fBMaPV1RHIjAmkK73HzZZ2/hvfUz70xYqLaoSSBmSlYEdfjjArsggOpSJY2vqCmC81wk8a2dwKISBySlDABhr2aQWT34+XdIcSX5FahatVXDDouIPIGWWzVvX2Iqq7TLQ73MplkE2CUGxLnSE1FMPkNlcbm9pNTGrugmaaIPztWq9nbQ/CVlkYgYliYh236n8KZuTNwdctQiNHpLWxrF4XOTYxO7vwFL7rK+eVvEm0MGqFsg+n/FfOgtorlCe7dgnvFz6CGaokWkMjHdZUrByOB94qmuQFqo8iHdvTwFYkBcw== X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(396003)(346002)(136003)(376002)(230922051799003)(451199024)(186009)(64100799003)(1800799009)(82310400011)(40470700004)(46966006)(36840700001)(40480700001)(40460700003)(70206006)(70586007)(110136005)(86362001)(356005)(82740400003)(81166007)(36756003)(6512007)(83380400001)(336012)(26005)(6506007)(6666004)(2616005)(36860700001)(966005)(2906002)(478600001)(41300700001)(6486002)(316002)(8676002)(47076005)(8936002)(5660300002)(44832011);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Nov 2023 11:40:01.0556 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8129198c-745a-4f15-c517-08dbe5066fca X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM2PEPF0001C70B.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB9502 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,FORGED_SPF_HELO,KAM_DMARC_NONE,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: The 11/14/2023 00:11, Kozlukov Sergei via Libc-help wrote: > Hi! I wasn't sure if this was worth a ticket on BugZilla yet, I hope libc-help will be an appropriate venue. > > I'm trying to assess if there's such a thing as an agreeable interface for locating the host platform's shared libraries. Specifically, if there's a "conventional" way of predicting, given a soname, what specific file the `dlopen()` would have loaded, but without actually loading it. i think posix allows names without / to be treated specially (e.g. ld.so could recognize 'libc.so.6' and not load anything from the filesystem). the path lookup can change at runtime (e.g. file system can change) if you are fine with a static view then ld.so --list exe can give you the mappings. (won't cover dlopened modules unless you create an exe that directly depends on those, but that's not guaranteed to be possible given the runtime dependent behaviour). > This task is, I think, precisely what certain container-related projects try to accomplish by peeking into the /etc/ld.so.cache, with the purpose of locating the userspace drivers that need to be mounted into the container (think mesa or, most realistically, libcuda.so): > > - https://github.com/NVIDIA/libnvidia-container/blob/1eb5a30a6ad0415550a9df632ac8832bf7e2bbba/src/nvc_ldcache.c#L370 > - https://github.com/NVIDIA/nvidia-container-toolkit/blob/da0755769fee3ab071aaa3b9fac6fe085da7071d/internal/ldcache/ldcache.go#L36 > > These tools are then used by many downstream projects, e.g. apptainer/sylabs-singularity, docker, etc. > > As far as I can tell, /etc/ld.so.cache is glibc internals and isn't meant to be used as a public interface. In particular, NixOS does not populate that file, and the nixpkgs simply patch all software that refers to it. > > I think this behaviour has to change upstream, in those projects, and for that we need an interface that could be considered "conventional", "maintstream", and "supported". > > The combination of `dlopen` and `RTLD_DI_ORIGIN` are almost perfect for the task: they respect LD_LIBRARY_PATH, LD_PRELOAD, and even the inquiring process's RUNPATH. Unfortunately, that comes with the side-effect of executing the loaded libraries' constructors. I understand that this shouldn't be a security concern, at least in the specific case of nvidia's container-toolkit and from glibc's perspective. However, there's no good reason to actually "load" the libraries being searched for, so I expect that this solution would be, generally, hard to sell. this assumes that the container somehow has the same setup as the host outside, which is not necessarily the case (same glibc, same config files, same env, same paths,..) note that the libc can dlopen internal components you don't know about so i don't see how this can ever be robust. (not to mention portability to bionic, musl, etc systems) so while an api can be designed for the path mapping it seems it won't fully solve the problem.