From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04on0629.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe0c::629]) by sourceware.org (Postfix) with ESMTPS id 2F5393858C5F for ; Mon, 3 Apr 2023 10:05:09 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2F5393858C5F Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HudbslKMHrnq4Yvxxn3VLfXXyiAf7Idq6Q3r7BAp8qU=; b=h63oGjAbZicIOtPgLoQcReHhZi2r88eWzXkKOtuIXfA+Yktg+OiyFlFmXMUE+y6/43pQcVNEKNcW6qoRqtorKnhQcxRijeSITNPe+59yAMCwrskExMci9oeYB+/1G1tbg8V6jFWT5L+i0wemNS0ivzng9qFCMJRluK/OD/kIyvg= Received: from AS9PR04CA0063.eurprd04.prod.outlook.com (2603:10a6:20b:48b::9) by DU0PR08MB8280.eurprd08.prod.outlook.com (2603:10a6:10:40c::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.29; Mon, 3 Apr 2023 10:05:04 +0000 Received: from AM7EUR03FT033.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:48b:cafe::94) by AS9PR04CA0063.outlook.office365.com (2603:10a6:20b:48b::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.22 via Frontend Transport; Mon, 3 Apr 2023 10:05:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT033.mail.protection.outlook.com (100.127.140.129) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6222.22 via Frontend Transport; Mon, 3 Apr 2023 10:05:03 +0000 Received: ("Tessian outbound 5bb4c51d5a1f:v136"); Mon, 03 Apr 2023 10:05:03 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 1ec5cbd3833b0d8d X-CR-MTA-TID: 64aa7808 Received: from 74f15d9e1fb1.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 731B1957-3C50-43D2-B719-1EA6D98FCA54.1; Mon, 03 Apr 2023 10:04:56 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 74f15d9e1fb1.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 03 Apr 2023 10:04:56 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=L8pNa+ATgPXN31IVQ7Vd2I84M5GNDRySoOMqQxdsk7Dn23XJOYOLt+PBm3uFpix+8YsXUBv++UP5abJJETlShFR8aOsXPFNLmyACZ+euY09HWf7OSMXbCGfZlXJapqUPicFQb/cQuI0gUx87zl7sK19XDPtk6ER69Qxg3wQ3udBVFuPkjvSDEDCczZgtYc+wkAI7WrxKzE3jKiGHxHAh3E9fJ0DKCwhFU4tvYEkJscHPvVcGaiooVPDLgT9H59KLrDF496CRM/c7+ICeTP/EtuXZQwbMomOp9pOGmZqS4Zxc583ytGVHs27B4gtvF8rSk2tiuLt03ZT2U/SpATJyOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HudbslKMHrnq4Yvxxn3VLfXXyiAf7Idq6Q3r7BAp8qU=; b=Yt8mKLNBe3R0tgtnNA27RCaz4vIs4a36dEI1aObP1ttAzqQ95mjzv9s+3D15LFjGtxG8X2YLveG4+ibZh7BkJO8YXpSNzoEliBMNOQPmnfL6SVbXiehGUmpnASkSCz3V5vCoO+pdr4vu4DlCWqv3sewyWmKOJ0yMtvY78FrFTiuSPmUzy+EtitA8iNWlc+RFpVicdoaAHWfqLxMkugDp7Z2+lnB+CbBqpyV2kFIt/9vpIq+9hNfZU7CDLrYfpPqvtv9+2sqjC+41H+eEpsD5lO7Bimty54y72tFwG2X36Fl1n4hdCRY6ey9OkeLbiNgORxaCpFs9lPagKQhwoD8H2A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HudbslKMHrnq4Yvxxn3VLfXXyiAf7Idq6Q3r7BAp8qU=; b=h63oGjAbZicIOtPgLoQcReHhZi2r88eWzXkKOtuIXfA+Yktg+OiyFlFmXMUE+y6/43pQcVNEKNcW6qoRqtorKnhQcxRijeSITNPe+59yAMCwrskExMci9oeYB+/1G1tbg8V6jFWT5L+i0wemNS0ivzng9qFCMJRluK/OD/kIyvg= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from DB9PR08MB7179.eurprd08.prod.outlook.com (2603:10a6:10:2cc::19) by DU2PR08MB10086.eurprd08.prod.outlook.com (2603:10a6:10:46e::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.23; Mon, 3 Apr 2023 10:04:54 +0000 Received: from DB9PR08MB7179.eurprd08.prod.outlook.com ([fe80::e3d1:5a4:db0c:43cc]) by DB9PR08MB7179.eurprd08.prod.outlook.com ([fe80::e3d1:5a4:db0c:43cc%3]) with mapi id 15.20.6254.033; Mon, 3 Apr 2023 10:04:53 +0000 Date: Mon, 3 Apr 2023 11:04:38 +0100 From: Szabolcs Nagy To: stsp , Adhemerval Zanella Netto , libc-alpha@sourceware.org, janderson@rice.edu, Carlos O'Donell , Rich Felker Subject: Re: [PATCH v9 0/13] implement dlmem() function Message-ID: References: <20230318165110.3672749-1-stsp2@yandex.ru> <2f3a10fa-4f79-7f9a-6407-d227dbf31935@yandex.ru> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-ClientProxiedBy: LO4P123CA0223.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:1a6::12) To DB9PR08MB7179.eurprd08.prod.outlook.com (2603:10a6:10:2cc::19) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: DB9PR08MB7179:EE_|DU2PR08MB10086:EE_|AM7EUR03FT033:EE_|DU0PR08MB8280:EE_ X-MS-Office365-Filtering-Correlation-Id: 3ff0da31-4e08-49d2-8c30-08db342ae4f0 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: jI9TBOXF6ikU28xdRaj9Okh+L1KVCv1P1aqvR3M6cx/UMsurBiadp6vRLTXZxmZrm5GAHdx2TUKI72VkKll14KWvwx0KvNLoQGrzO9ybbfsi7gt/JvJSVzA4621UGwnyuv2Q3HjxW/3xE4D/PKlxl4HTTV1EelohxENWCeyeqnzOGCrscebHuQx8do9TpQxX350g9rYmQWWeELFwBfJN4T6EIIMS7bBdQYLjfmHbW7yqUIdweQLnFh9eEG3IuAhVTfXeNHXx30fnJlWV5MKj4HJ40XvC8r5Jq/5q5BF2cShWvQr1HjEv6+c9cIBJdOd+tuzQwOg3Fzk13VVpxkY251BWhKbWM9hZqvg0djtCNjO1z7czRvbAbaq4X9gi0win5wTiHP0PR90msW32bGn778woYQQ7YrAbXY2RyQjEDbR5F6sunU2sQCqCIc6z6uqO3WE+QLn/0b2yfSH6FwE6ER13BFxaZmbLcD7/GmmTKaz9bfxNXFvGBNrt+wF3u8ejbaU3zk1Z6mXbaeHMiDnh0d7nkFOLQFSCXhX98Dfv6mMk6sGI+jYsQZxgJTt7u+lo X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB7179.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(366004)(39860400002)(136003)(396003)(376002)(346002)(451199021)(36756003)(83380400001)(2616005)(6506007)(6512007)(6486002)(26005)(110136005)(478600001)(186003)(6666004)(66476007)(316002)(66946007)(66556008)(44832011)(8676002)(8936002)(41300700001)(5660300002)(38100700002)(86362001)(2906002);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR08MB10086 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT033.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 1352d0d5-bf80-4def-53b3-08db342adde1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KtDfSoztOiQfwYffJZFKqTkRv5oQl9XhO5fValvWSo2LRVdb4gHr79aBIYRstESlAjeClYuW3uCR1Dcrye6SmeLeVvxqGl9sZIvZHnpu5FC43TJ2/mh/hxZAmfR/tLrM2emgqxxY3Vlim0nO+imitlwtrsEenPYOEml1FJ3QnpfpLrp0CUxQtGiL+bkDk7Won2snXh+kLQKMRNzTpHd6TMnxwTOCXNlR//Wp/e9yQ77iQ22qzwpAY8tLBLfnDJp7EoaTPR61dXx/hl+9yH01brPApTfWfL6oS8qznHoB0x/+ezDJd57+d31RaKH70+d9mCSL3B/SlyOItxTtkWvUsRdNR5ssH2n263zzJ2g2ofUQCz/TVTCQnySsmQANHIXxwMd0CayM5lrljmr4+6Im3/9xSnTxqRQBI2RT7GY/e+3FxcVLjDS/Kj8wZVo846Mgah57Fdk3XMonKb343boyRN6InZixUWSno9kekOVbdp+WX2HSNZW5R/Np3HZjLwt7pZ5GpabY32RftiCBhIBCxW1pRTj7xizN2NuLRqSOMYS3LAqizDbEszgFKhGMgh0RX5SDyAQbuHqEMGB3Gh8N9ffKXTmznt3BUITt4MUJfc6i+lYP3zubY00DhAi/TnyNwfNvdcXOfoTEu8nJmuLa7p82BXDU8Rk0DKSojQGCQa8lTc34EOEyuerlxEe7jITvefdxnK7zjMbJWcrh4cgqAQ== X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230028)(4636009)(396003)(136003)(39860400002)(346002)(376002)(451199021)(46966006)(36840700001)(40470700004)(86362001)(186003)(336012)(44832011)(2906002)(81166007)(41300700001)(2616005)(8676002)(6666004)(82740400003)(6506007)(26005)(6512007)(40480700001)(356005)(82310400005)(83380400001)(47076005)(6486002)(478600001)(36860700001)(110136005)(316002)(40460700003)(8936002)(70206006)(70586007)(5660300002)(36756003);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2023 10:05:03.6672 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3ff0da31-4e08-49d2-8c30-08db342ae4f0 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT033.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB8280 X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,BODY_8BITS,DKIM_SIGNED,DKIM_VALID,FORGED_SPF_HELO,KAM_DMARC_NONE,SPF_HELO_PASS,SPF_NONE,TXREP,UNPARSEABLE_RELAY autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: The 03/31/2023 22:36, stsp wrote: > 31.03.2023 22:12, Szabolcs Nagy пишет: > > loading from special place and loading into special place > > are different things. > > These are related. > File-based mmaps leave you no chance > of preserving the destination mapping, "preserving" the "destination mapping" (?) a new requirement.. > > your dlmem design it passes down a user callback that runs > > under dynamic linker locks to map segments. the interface > > contract of this callback is not documented, so it can > > deadlock > > Good point. > But this is a documentation issue, right? > Not too much is needed: in the simple case > you only do anonymous mmap() in a callback. > In a more difficult case (my use-case) I do > shm_open(), ftruncate() and mmap(). > Of course I need to document that any libdl > functions are disallowed, is this correct? that is not correct. (see below) > > and exposes dynamic linker internals (makes > > implementation changes harder later). > > Could you please clarify that part? > typedef void * > (dlmem_premap_t) (void *mappref, size_t maplength, size_t mapalign, >               void *cookie); > > This only passes things needed for an initial > mmap(). The single mmap() that just defines > the mapping address. > What internals do you mean? the segments of an elf file can be mmapped in many ways, there is no such thing as initial mmap (there need not be a mapping that covers all segments and there is no ordering requirement for the mappings or which mapping is file backed etc) your callback exposes a particular behaviour. and libc apis are exposed to not use locks that have lock ordering requirement wrt the dynamic linker internal lock. (e.g. we could add such a lock in mmap or shm_open since they are not required to be as-safe. so your callback can deadlock in a future glibc). and it's not just locks but any unusual libc internal state that becomes observable via the callback. (the callback runs when the dynamic linker may be in inconsistent internal state. why do you assume that shm_open or other libc apis work in that case? this is a new requirement that has to be documented.) > Also please note that the callback is optional > and is not needed for 99% of the interesting > cases possible with dlmem(). it does not matter if it's 99% or 99.999%, what matters is that if we expose an api then that has to work *forever* under all potential usage and we have to keep *supporting* it with all the api&abi constraints. giving examples is not a *proof* that the api works. > > if all you want is force mapping below 4G then you should > > have asked for an RTLD_ flag (which may not be useful > > enough or may not be supportable on all relevant targets, > > but at least does not need significant loader changes). > There is already such flag. x86 specific flag. > Unfortunately my setup is much more > complicated than that: I need to force > the mapping under 4Gb AND I need to > force it into the shared buffer which I > then mmap() to another address... you didnt explain this in the rationale of the patches. but it seems what you want is not suitable for libc. you have to write your own loader to do this.