From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from forward502a.mail.yandex.net (forward502a.mail.yandex.net [178.154.239.82]) by sourceware.org (Postfix) with ESMTPS id 9540F3858D37 for ; Mon, 3 Apr 2023 13:07:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 9540F3858D37 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=yandex.ru Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=yandex.ru Received: from mail-nwsmtp-smtp-production-main-68.vla.yp-c.yandex.net (mail-nwsmtp-smtp-production-main-68.vla.yp-c.yandex.net [IPv6:2a02:6b8:c0f:4c8e:0:640:3460:0]) by forward502a.mail.yandex.net (Yandex) with ESMTP id 17BB35E928; Mon, 3 Apr 2023 16:07:54 +0300 (MSK) Received: by mail-nwsmtp-smtp-production-main-68.vla.yp-c.yandex.net (smtp/Yandex) with ESMTPSA id p7NEF9nDfKo0-MCfIfmA5; Mon, 03 Apr 2023 16:07:53 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1680527273; bh=TBZLrlECHce/nqkYKnHrOd7jlVG0FME1v6ocXr1WmRo=; h=In-Reply-To:From:Date:References:To:Subject:Message-ID; b=fWS30qGTfm30AMLB5B3Iw2qk/gf1hjp/CHXOu9pFL1N3yhdb+zYtbTdG1is773x8o 06HWhXMt9zNqbgtUCbTz4aQw1cDlLUh4IQBgGx7sW4oQlh2HYOdBaI1Oxl1gZd+zkh 8+0npPiBPsop1SjtAWy4mh/FeUEF9NKWoZqoREHU= Authentication-Results: mail-nwsmtp-smtp-production-main-68.vla.yp-c.yandex.net; dkim=pass header.i=@yandex.ru Message-ID: <35c52a39-09ca-66b9-c6b5-9bd50144d2b6@yandex.ru> Date: Mon, 3 Apr 2023 18:07:50 +0500 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: [PATCH v9 0/13] implement dlmem() function Content-Language: en-US To: Szabolcs Nagy , Adhemerval Zanella Netto , libc-alpha@sourceware.org, janderson@rice.edu, Carlos O'Donell , Rich Felker References: <20230318165110.3672749-1-stsp2@yandex.ru> <2f3a10fa-4f79-7f9a-6407-d227dbf31935@yandex.ru> <298b04a6-3055-b89b-59c1-4cfbe955848e@yandex.ru> From: stsp In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-3.0 required=5.0 tests=BAYES_00,BODY_8BITS,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hello, 03.04.2023 17:01, Szabolcs Nagy пишет: > these are all *internals*, there is nothing in the > public api (or even elf spec) that says this is how > this should be done. Yes, OTOH of course its reasonable to expect that the solib's mapping size's upper limit can be estimated before doing the actual mapping. So I rely on such estimation code which is already there. Maybe it would be a solution to add another function that will return such a size estimation? Well, I am sure this also have dozens of down-sides... >>> and libc apis are exposed to not use locks that have >>> lock ordering requirement wrt the dynamic linker >>> internal lock. (e.g. we could add such a lock in mmap >>> or shm_open since they are not required to be as-safe. >>> so your callback can deadlock in a future glibc). >> Well you probably can't add such a lock into mmap(), >> as the loader calls mmap() anyway. Premap callback >> just does the same thing. > no, the loader does not call the public mmap libc api. > it calls internal __mmap that may behave differently. > > the current implementation does not impose any > requirement on the public mmap symbol. An option may be to pass a premap callback with a pointers to such __mmap, __ftruncate. But of course not the best option again. :( > a user callback must not run in a context where libc > internals are exposed (e.g. under internal locks). > if you do that you need very strong justification > and very careful specification of the api. (moving > the callback outside the dynlinker locks or passing > down flags instead of a callback would solve this > particular issue.) I considered passing flags and fd. This makes things much less flexible, but the real problem here is to set the relocation address. premap callback just returns it after doing the mapping of a requested size. Without knowing the estimated size beforehand, you unfortunately can't easily go that route. :( The func to get the estimated size, could help. > there are so many unexplained details here.. we have > to guess what you have in mind. (should tls also end > up in shared memory? what if the libc wants to generate > executable code per module for instrumentation? e.g. > for PLT hooking, should that go to the shared mapping > too? what if the malloc implementation reuses bits of > memory from shared libs they waste due to page alignment > is that legal? we can answer these questions if you > either specify the interface contracts or the point of > all of this, but right now it's anybody's guess.) No, no, its much simpler! :) I just put _entire_ solib into a shared mapping. I do not care about tls or anything. All I do is create the initial large mapping, that remains unaffected by the loader. Even the inter-segment gaps you mentioned, or the anonymously mapped parts like .bss - just all go into that initial shared mapping. Which is why I haven't described such details. That are simply not important in that case. > libc will likely not include an api to load libs into > shared memory. (e.g. aarch64 has features that work on > anon or file backed memory but not on shared memory and > some systems may prevent it for security reasons, so > this has at least portability problems, but it also > clearly constrains the implementation.) Well, from the API stand-point its not mapping into a shared memory. Its mapping into where's the premap callback decided. Can we assume that the premap callback is responsible to not do stupid things on aarch64, where mapping to shared memory will not work as expected? > you can load a shared lib / generate code that does > not use normal libc shared library abi, but can > call into libc (or some other library) or can be > called from normal libraries. this is kind of what > jits do. I wanted to load a solib and get its DT_NEEDED deps all resolved. If I had some way of linking things with my patched, statically built libc, that would work. Maybe I can rip off the glibc loader and somehow build it separately, etc. So what you say, may work, but seems like a tremendous task of either writing a full loader or ripping off the one from glibc... And even that route will only allow me to use such libs in a separate NS, as in base NS it won't have a link-map visible by glibc. But having that in a separate NS is acceptable for my needs, so eventually I may go that route if all else fails. > there is probably a reason why that api got rejected > and likely the same reason applies to dlmem. Likely, but I just thought having an android extensions in, is not so bad for glibc (of course its not me to decide). So having a building block that supports dlopen_with_offset() being externally implemented, looked like a tasty thing for glibc, but perhaps it wasn't the case.