From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-AM7-obe.outbound.protection.outlook.com (mail-am7eur03on2068.outbound.protection.outlook.com [40.107.105.68]) by sourceware.org (Postfix) with ESMTPS id 07B533858D37 for ; Mon, 3 Apr 2023 12:02:00 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 07B533858D37 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tM3drf4Z+aMd1xWpn+PnF1wCSh7U+B8WbKUMQADScTk=; b=5tcITy354aMh6zbtYZEZ9smiwK9J3pXwqvfnrIc+tLfQEmEk9MgwoE02cJr7RiW9yM+wSO/3zDCTjw0jkpfkFqxbynD7LdLC3drz4k50qMIDlQs4TPhlIikalKMOuQxgoaMbLU6fJ65Y3gi0Q9Vz5KF4Rafq5I3Td7Vd3h5DLis= Received: from AS9PR06CA0667.eurprd06.prod.outlook.com (2603:10a6:20b:49c::12) by PA4PR08MB6174.eurprd08.prod.outlook.com (2603:10a6:102:e6::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.33; Mon, 3 Apr 2023 12:01:53 +0000 Received: from AM7EUR03FT031.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:49c:cafe::e0) by AS9PR06CA0667.outlook.office365.com (2603:10a6:20b:49c::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.22 via Frontend Transport; Mon, 3 Apr 2023 12:01:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT031.mail.protection.outlook.com (100.127.140.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6222.22 via Frontend Transport; Mon, 3 Apr 2023 12:01:53 +0000 Received: ("Tessian outbound 8b05220b4215:v136"); Mon, 03 Apr 2023 12:01:52 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: e76015ee6fa80b49 X-CR-MTA-TID: 64aa7808 Received: from cf3f317cd309.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id C9A8EA9C-0FF2-4CDA-9190-F4A6CF82B093.1; Mon, 03 Apr 2023 12:01:45 +0000 Received: from EUR04-DB3-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id cf3f317cd309.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 03 Apr 2023 12:01:45 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PlUmM0DggJ+5oMUTvtwk1Cd1Sfnkmouw/N6x9GNKsMDfx63J6MjbwOOHAFyTeyCPzvn8OpUOoZnzozlEGizh2Xoyx+nY1VKdJ484I718snksxZNA7E4tmRnkAQSRK0LhL/slZyyRxLcrR40TrowFNY5msNkM4ERhfkTbI1nCHhuCdXejzLxmDV/wU6ZIBudy48ipBYGg12+0H66KurVleTnSNsJJBYiuXZ/bd0st/JPf1sh+Q6U4kbSAX8h+HgAaVc3ng0Zr+oHqfeE0g2qWabA1mm+ze/MmBCAXtFEKKte3vm5kMa4whmDY7mjZFehIQIRUhd8OpluIueZJavawTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tM3drf4Z+aMd1xWpn+PnF1wCSh7U+B8WbKUMQADScTk=; b=X8UXCEX+hiQh+Gl3PDK+3fJ1O+pBG/M5bIlVAAjf2JjbeuharEjAiguN5fpCB3r5v5jfLWTuB790LZO2Nzz2rU6h2ZuI5t0CYyasc8V8WFhYnP74Mjmp9brRC2C6DDtniyKuOV/3H8XAr59WiAJ5QjS10HKQ2mpNstARdpL1RykKkei+fwJqRzvwjPi/pnE5HsTRz+22VPpxtRJANRhj6rD6xW/hxRVgvNtWUxGIhtHdLbBkm5a9x8oF7+Q7mV7erG9komH5TIEUPnSBkXeyqmGM9zt4+ysn+f64bUD5yirfPz+yEnwP8QkiZlFeU7StPffgUy4vhYnWdzZ6GogW2g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tM3drf4Z+aMd1xWpn+PnF1wCSh7U+B8WbKUMQADScTk=; b=5tcITy354aMh6zbtYZEZ9smiwK9J3pXwqvfnrIc+tLfQEmEk9MgwoE02cJr7RiW9yM+wSO/3zDCTjw0jkpfkFqxbynD7LdLC3drz4k50qMIDlQs4TPhlIikalKMOuQxgoaMbLU6fJ65Y3gi0Q9Vz5KF4Rafq5I3Td7Vd3h5DLis= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from DB9PR08MB7179.eurprd08.prod.outlook.com (2603:10a6:10:2cc::19) by PR3PR08MB5866.eurprd08.prod.outlook.com (2603:10a6:102:85::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.33; Mon, 3 Apr 2023 12:01:42 +0000 Received: from DB9PR08MB7179.eurprd08.prod.outlook.com ([fe80::e3d1:5a4:db0c:43cc]) by DB9PR08MB7179.eurprd08.prod.outlook.com ([fe80::e3d1:5a4:db0c:43cc%3]) with mapi id 15.20.6254.033; Mon, 3 Apr 2023 12:01:42 +0000 Date: Mon, 3 Apr 2023 13:01:27 +0100 From: Szabolcs Nagy To: stsp , Adhemerval Zanella Netto , libc-alpha@sourceware.org, janderson@rice.edu, Carlos O'Donell , Rich Felker Subject: Re: [PATCH v9 0/13] implement dlmem() function Message-ID: References: <20230318165110.3672749-1-stsp2@yandex.ru> <2f3a10fa-4f79-7f9a-6407-d227dbf31935@yandex.ru> <298b04a6-3055-b89b-59c1-4cfbe955848e@yandex.ru> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <298b04a6-3055-b89b-59c1-4cfbe955848e@yandex.ru> X-ClientProxiedBy: LO4P123CA0697.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:37b::12) To DB9PR08MB7179.eurprd08.prod.outlook.com (2603:10a6:10:2cc::19) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: DB9PR08MB7179:EE_|PR3PR08MB5866:EE_|AM7EUR03FT031:EE_|PA4PR08MB6174:EE_ X-MS-Office365-Filtering-Correlation-Id: 133a8214-1a23-4109-54b5-08db343b36db x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: Mt1PpLUYB0rQS+tXtj7C+N+9g5VIOACoyNc2UTYLGej+YQrn5t1cXPVf5MZTHLONlP4HGhapVEQcu5ephteHr7cyoqIjklOE/1G+vVRrnBaLPC2zakC3/T4n4keTC8hqwuiTmFWwddsQNm+DCePX+IJ3EPmuWbv+4/DNUOAGkcP97MkvV+Ri7isCBlTanI/py8c+69FouViGnx+cqEpuNwYl5Kn+xCHg7BQ+yRxASPlBGhzs+gqtxLV5ghbwCxb9zUw6AaQ6bLMX7uagSPbbt+uTDVC6UVchhE8dw0pOHuQvGFD6Anjn2q+eVAcaxnOMF3v488exPQM26MHdQqoopN35PBTj0DrlODWSm19S6esBXWoI0KHihfJ2ZYObTpj3P2FixGAv6cbxAjzn/4fBMq31ekTkmGxJO8AxjgQxmCu3B1ycgmWXbJv4FZ6TCRJsaVCOWU2rgPKQD4Nt5hneghEu6elb2EmHdfZW+e2E+WzCkDyVB9usVgJP/B6NtPgCkwHdthJeKk81LLpzQsdDnCLiS3BbQHhe7wmZApNWhV0Dagrjh2ml2U+Uk5RyQpvL X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB7179.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(366004)(136003)(39860400002)(396003)(376002)(346002)(451199021)(186003)(6512007)(6506007)(26005)(44832011)(6666004)(110136005)(478600001)(36756003)(38100700002)(66556008)(86362001)(66946007)(66476007)(83380400001)(8676002)(316002)(2906002)(41300700001)(5660300002)(6486002)(2616005)(8936002);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5866 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT031.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: e5163482-8ea8-40bc-7ade-08db343b302e X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HhFxUoQeWg2Xzo5KVVLIlf4oRVc2EnRyUnwSjCRBXiDW6HKIxzRizNh+/xhYp2AjlHF5pJUlUZpoTSeaPGZBbjmDaLbStMVEIfuc4MGVQB5YL7C20LnF4hnwghy2UTK5Kt809imrGSLPWbO7qMD30QzuvL+Yth2XaJ9n1Jk1g9xPUD0O0UMCehzBhNTccdWzPxPJYqeW69roX8Xf+6XLbl2uMM9kd6NgqJLyKydEeNmbWoqMfGKTtsjDaMTk0SUx5B8v2l3+6Hqq3iYR2f+xY/ed8Ji1WY3GcYL+8tWepGPcI8h6EA1K2Vqk1GGtBNZS5JJNwoA7uI8C95GQFxcbwr3RTf9zn7bQCPvQY6qfysAEefXkouUkX2m4BhToge7AXeVyYiUjDrgbNXhherntmRUluq3WKTlREIUewhO72dno23Zzn1WXZW1fYb8xKjoYG9e6Ww6Yl+2Vva5yYUbgU0nXbKJf2MKrQhNqpb1fZDDugiLMO9hj2lywVmvW9NlxiZYod6t3PHWnIK1pS6r9ZoaXc92W6jZbDpW/+T5gr10azGQIWxJn7FXG7TA3vLqf4ncl087dOEzuX/THbjLkDpcEYNfWRGl8LbJKiE9riQif7w0hYTuHYISiGW+lgStbGCM8lIbaXYK622ZlKf37R8KoMKNRhYxTXU507MnsXe10+8XW+9DtLvvtAvT/mC3n8RfXfM24qDQbhjEZj/Jy3w== X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230028)(4636009)(376002)(346002)(136003)(39860400002)(396003)(451199021)(40470700004)(36840700001)(46966006)(40480700001)(40460700003)(8676002)(70206006)(70586007)(36860700001)(478600001)(316002)(110136005)(8936002)(44832011)(356005)(82740400003)(41300700001)(81166007)(5660300002)(47076005)(186003)(83380400001)(2616005)(336012)(6486002)(6666004)(26005)(6506007)(6512007)(86362001)(36756003)(82310400005)(2906002);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2023 12:01:53.0334 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 133a8214-1a23-4109-54b5-08db343b36db X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT031.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6174 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,BODY_8BITS,DKIM_SIGNED,DKIM_VALID,FORGED_SPF_HELO,KAM_DMARC_NONE,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,TXREP,UNPARSEABLE_RELAY,URIBL_BLACK autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: The 04/03/2023 15:43, stsp via Libc-alpha wrote: > 03.04.2023 15:04, Szabolcs Nagy пишет: > > the segments of an elf file can be mmapped in many ways, > > there is no such thing as initial mmap (there need not > > be a mapping that covers all segments and there is no > > ordering requirement for the mappings or which mapping > > is file backed etc) your callback exposes a particular > > behaviour. > > I think this is a misunderstanding. > Even before my patch there is this code in dl-load.c: these are all *internals*, there is nothing in the public api (or even elf spec) that says this is how this should be done. > So I am working on an existing frame-work only. no, you are exposing implementation internal details. > > and libc apis are exposed to not use locks that have > > lock ordering requirement wrt the dynamic linker > > internal lock. (e.g. we could add such a lock in mmap > > or shm_open since they are not required to be as-safe. > > so your callback can deadlock in a future glibc). > > Well you probably can't add such a lock into mmap(), > as the loader calls mmap() anyway. Premap callback > just does the same thing. no, the loader does not call the public mmap libc api. it calls internal __mmap that may behave differently. the current implementation does not impose any requirement on the public mmap symbol. > Do you really think some lock can be added to > ftruncate()? we have to reason about the constraints not second guess what others think, we have plenty syscalls emulated in the libc for posix conformance reasons doing non-trivial things and various libc apis are interposed by user code (see asan) and then they can do whatever (e.g. asan interposes mmap, but cannot interpose the internal __mmap). > > and it's not just locks but any unusual libc internal > > state that becomes observable via the callback. > > (the callback runs when the dynamic linker may be in > > inconsistent internal state. why do you assume that > > shm_open or other libc apis work in that case? this > > is a new requirement that has to be documented.) > > Mainly mmap(), and the loader does mmap()s by > himself, so I quite assume they work from the > callback. again, this is wrong. > > it does not matter if it's 99% or 99.999%, what matters > > is that if we expose an api then that has to work > > *forever* under all potential usage and we have to > > keep *supporting* it with all the api&abi constraints. > > Would it be acceptable to settle on an agreement > to not add a lock to ftruncate() that would prevent > its use from a premap callback? If its not possible, > of course there is still an option to document the > premap callback in a way so it will have to do the > syscalls directly. Which doesn't look like a very > good choice, but possible. a user callback must not run in a context where libc internals are exposed (e.g. under internal locks). if you do that you need very strong justification and very careful specification of the api. (moving the callback outside the dynlinker locks or passing down flags instead of a callback would solve this particular issue.) > > > Unfortunately my setup is much more > > > complicated than that: I need to force > > > the mapping under 4Gb AND I need to > > > force it into the shared buffer which I > > > then mmap() to another address... > > you didnt explain this in the rationale of the patches. > > I mentioned such a use-case actually in the > rationale, here's the quote: > > More so, if DLMEM_DONTREPLACE flag is used, then the mapping > established by the premap callback, will not be replaced with the > file-backed mapping. In that case dlmem() have to use memcpy(), which > is likely even faster than mmaps() but doesn't end up with the proper > /proc/self/map_files or /proc/self/maps entries. So for example if the > premap callback uses MAP_SHARED, then with the use of the DLMEM_DONTREPLACE > flag you can get your solib relocated into a shared memory buffer. there are so many unexplained details here.. we have to guess what you have in mind. (should tls also end up in shared memory? what if the libc wants to generate executable code per module for instrumentation? e.g. for PLT hooking, should that go to the shared mapping too? what if the malloc implementation reuses bits of memory from shared libs they waste due to page alignment is that legal? we can answer these questions if you either specify the interface contracts or the point of all of this, but right now it's anybody's guess.) libc will likely not include an api to load libs into shared memory. (e.g. aarch64 has features that work on anon or file backed memory but not on shared memory and some systems may prevent it for security reasons, so this has at least portability problems, but it also clearly constrains the implementation.) > > but it seems what you want is not suitable for libc. > > you have to write your own loader to do this. > I very much wish to do that! > But so far I failed with a few naive attempts, > like building the patched glibc statically, or > loading custom libc.so into a separate name-space... you can load a shared lib / generate code that does not use normal libc shared library abi, but can call into libc (or some other library) or can be called from normal libraries. this is kind of what jits do. > I don't know what interface should be exposed > by glibc to let my own loader to create a new > link-map and bind symbols to it. Maybe even > none, maybe I can do that via r_debug for example. > So I very much wish to work in that direction, if you > think it is a more appropriate solution that > can be accepted into glibc (if any changes are > at all needed). > > However, dlmem() can be used for Android's > dlopen_with_offset(), which was also rejected > from glibc. With dlmem() its just a dozen of > lines that never need to be in glibc itself. So > for example the problem with dlopen_with_offset() > could be easily settled with my dlmem(). there is probably a reason why that api got rejected and likely the same reason applies to dlmem.