From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from forward502c.mail.yandex.net (forward502c.mail.yandex.net [IPv6:2a02:6b8:c03:500:1:45:d181:d502]) by sourceware.org (Postfix) with ESMTPS id 512D13858401 for ; Thu, 13 Apr 2023 15:59:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 512D13858401 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=yandex.ru Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=yandex.ru Received: from mail-nwsmtp-smtp-production-main-57.myt.yp-c.yandex.net (mail-nwsmtp-smtp-production-main-57.myt.yp-c.yandex.net [IPv6:2a02:6b8:c12:1488:0:640:3dc6:0]) by forward502c.mail.yandex.net (Yandex) with ESMTP id D3C645ED91; Thu, 13 Apr 2023 18:59:04 +0300 (MSK) Received: by mail-nwsmtp-smtp-production-main-57.myt.yp-c.yandex.net (smtp/Yandex) with ESMTPSA id 1xZL8T4WwOs0-fWnyJwgT; Thu, 13 Apr 2023 18:59:04 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1681401544; bh=uf/kPvGkOanuB6vbo0rnGv4pJ/omlhuN2tlcpK5xm2A=; h=From:In-Reply-To:Cc:Date:References:To:Subject:Message-ID; b=PaFzmJq2wLLoCoOgNHLKvCibSBgnN3IrVQ4ue3ySuHcWhz2ROV5pj9kal9SXDwBru chnYkDowG+ehfotdNR6l+SSa3JsiHAoRPcvQR6ikcxjwPahMHGwCXiP5EybId3ZWy+ gGjiSUEBzbbsg4l9wU14rxyG4HKVHrRDOH63d3yk= Authentication-Results: mail-nwsmtp-smtp-production-main-57.myt.yp-c.yandex.net; dkim=pass header.i=@yandex.ru Message-ID: <78b5b5dc-5657-4bf8-24c6-6c00afb1cc40@yandex.ru> Date: Thu, 13 Apr 2023 20:59:01 +0500 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: [PATCH v9 0/13] implement dlmem() function To: Szabolcs Nagy , Rich Felker Cc: Adhemerval Zanella Netto , libc-alpha@sourceware.org, janderson@rice.edu, Carlos O'Donell References: <298b04a6-3055-b89b-59c1-4cfbe955848e@yandex.ru> <81749d04-8cdb-de0b-b88e-24347ed535ba@yandex.ru> <729710b5-6dae-d5f2-99ee-6923be5e627d@yandex.ru> <20230412182043.GI3298@brightrain.aerifal.cx> <08d9ca95-112c-d85e-8e82-7a595ef4d051@yandex.ru> Content-Language: en-US From: stsp In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,BODY_8BITS,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hi, 13.04.2023 17:38, Szabolcs Nagy пишет: > it does not allow handling relocation failures sensibly, > does not specify what happens with dependencies and > exposes half loaded modules to users with unclear use-case > or semantics for them. Thanks, lets update the spec to address this: RTLD_NORELOC - do not relocate the object, allowing to perform the relocation later. Ctors are delayed, and are called immediately after the relocation is done. Relocation is performed upon the first operation on the returned handle, before the requested operation on a handle is performed, with the exception of dlset_object_baseaddr() which doesn't perform the relocation. If this flag is used, it is recommended to use dlrelocate() to explicitly perform the relocation and check the result of it. This flag doesn't delay the load of an object deps and the calls to their ctors. It also doesn't delay the LA_ACT_CONSISTENT audit event. dlset_object_baseaddr(handle, addr) - sets the base address of an unrelocated object. Returns error if the object is already relocated. The base address set by this function, will be used when relocation is performed. dlrelocate(handle) - perform the object relocation if the RTLD_NORELOC flag was used, return EINVAL if already relocated. This function may be omitted even if RTLD_NORELOC was used, in which case the relocation will be performed upon the first operation on a handle, but using that function allows to handle relocation errors and run ctors before using the object's handle. If the function returned success then ctors of an object were called by it. If it returned error other than EINVAL (EINVAL means object already relocated), then relocation error happened and the handle should be closed with dlclose(). It seems that spec covers all of your concerns. It does cover even the exposure of "half-loaded modules" by stating that the relocation may happen lazily so things would work as expected in any case. > if you fix an actual bug that's accepted. I believe this classifies as a fix: https://patchwork.ozlabs.org/project/glibc/patch/20230301150818.31815-2-stsp2@yandex.ru/ > you are adding a new interface without gathering consensus > that it's a good idea at all or how it should work. I can get such a consensus only if people, like you do (many thanks for that!), explain the problems to me and suggest an amendments. Unfortunately, its not the case. People are saying that my proposal is bad, and the only suggested amendment is to drop it immediately. :) But I am very thankful to those who at least did some side-activities for me, like discussing my use-case and reviewing part of the patches. These activities were very valuable, but the real problems were never discussed before, proposed API was not reviewed, and so on. And with your input I already advanced to the point where the API can be split into 2 parts, whereas before I thought of dlmem() as a thing in itself that can't be split. Now I see that RTLD_NORELOC can solve the biggest part of the puzzle alone. > when i said you should write your own loader i expected > you to create simple stubs in memory at the location > you want them to be and bind them to a library that > is loaded by libc (so you can connect your weird world > with the libc world). > > it looked like a simple solution to me, but i still dont > know exactly your requirements so in a sense we still > don't have a well understood use-case for any of your > proposed changes. so maye the use-case is where you > should start. You mean "restart". :) As I already explained it very thoroughly in the bugzilla, but you missed that and now its better to repeat, since the bugzilla entry had a very lengthy discussions afterwards. The use-case is a VM with 32bit code inside. It has a 4Gb window mapped somewhere in a 64bit space (call it VM_window_start). The task is very simple: first I find the suitable mapping address within a VM address space. Such address is within the 0-4Gb boundaries. I relocate the solib to that "low" address. Unfortunately this is still not visible to the VM, because VM's window is at VM_window_start, as we remember. So in order for VM-running 32bit code to access the solib's data (like .bss, .rodata) when it gets a 32bit pointer to such data, I also need to map the full duplicate of the solib object into the address that looks like: dupe_addr = VM_window_start + reloc_addr For that I invented the optional dlmem flag that doesn't destroy the destination mapping. But such flag doesn't survive your requirement to get rid of a call-back, so in the next prop I'll pass a shm fd to dlmem instead, and an optional flag will ask dlmem() to mmap that fd as a destination backing-store. So I am sure all problems can be solved! But of course I can't do that alone, if everyone just keeps saying "drop your patches immediately".