From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 0C94B3858D1E for ; Wed, 29 Mar 2023 18:13:26 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0C94B3858D1E Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680113605; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zwaOJ+xKASbE5kEgYeGcQiYK5rgeyrp9MZmBhVMoSG4=; b=NXxT4hNMbCjxvgpFVGMdESng5detFleiX87lB2C1LmHBU1K3I0nVqB8cb6yk139vI0iVw6 jQSEk7NPNzFJWllmuYO+/VMdoFptoVK2TC/L/BgkzAykOHC4HsIPxtlrJ/PbT47hgvuvzx JJ5ZYb3JveyXvbtmRJT/fCXQZOav7wc= Received: from mail-yw1-f200.google.com (mail-yw1-f200.google.com [209.85.128.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-596-csj09dBCOLCGYpktdfYZUg-1; Wed, 29 Mar 2023 14:13:18 -0400 X-MC-Unique: csj09dBCOLCGYpktdfYZUg-1 Received: by mail-yw1-f200.google.com with SMTP id 00721157ae682-541942bfdccso163981377b3.14 for ; Wed, 29 Mar 2023 11:13:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680113592; h=content-transfer-encoding:in-reply-to:organization:from:references :to:content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=zwaOJ+xKASbE5kEgYeGcQiYK5rgeyrp9MZmBhVMoSG4=; b=MvcMoLnOz+v/G0SXDboiHyJzZdsFNkJimlsQ+DTGFQ9/uqcPkGfmnzvsgwAGhvIvpe C01jaJIUsKs5UrlobN92G+a2Jx6qk3iG0W+8w1fyoL8Clwz++4BqK8fJguDV40p4iMU3 OTTpHhby+g9jrTem178wlCiiMB7s21DwEsNcLJ1nt9N1CtngF1njZEnmPI2ASSntR6A+ WDhdBipxLeuyih/eoh92MHi+eqUBEJn+v5mNBXrsctWmtKIk4gHrh+sEVSR9wDawyuBr w8z9E1pZq4OgMS6uMZSEbT9NLjZXgc1XPDJ1gpeC8Xwvc/cTVR366ytgoTVrj3MFWbtb 584Q== X-Gm-Message-State: AAQBX9fKasN7UVWtv3ltH6j71qLjHPknyMniFwV1t4Nqq/fZ7wdjBdg1 SLfNT/CTYFUw51jnvRMydAPMGZYbfkx0zZnFP6OBFwTHeFZaWqKY2UAWFRYs7gc/3dzDksk+M9m YxEGu3CxUNIuVHJ15e7wk0pG2zd++ X-Received: by 2002:a25:41c4:0:b0:b75:8b6e:88b5 with SMTP id o187-20020a2541c4000000b00b758b6e88b5mr19422410yba.51.1680113592709; Wed, 29 Mar 2023 11:13:12 -0700 (PDT) X-Google-Smtp-Source: AKy350b66iDFYyeRss0Kr24OgF5vF0RWVrJUztJv/3U7qgEVEa2QrA3kZ4sHKwSl/4xOL2q0lRgnhg== X-Received: by 2002:a25:41c4:0:b0:b75:8b6e:88b5 with SMTP id o187-20020a2541c4000000b00b758b6e88b5mr19422396yba.51.1680113592398; Wed, 29 Mar 2023 11:13:12 -0700 (PDT) Received: from [192.168.0.241] ([198.48.244.52]) by smtp.gmail.com with ESMTPSA id 22-20020a250316000000b00b7767ca7464sm3611683ybd.1.2023.03.29.11.13.11 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 29 Mar 2023 11:13:11 -0700 (PDT) Message-ID: <81d75bd3-147e-f85a-9955-0c7f0f2dfbeb@redhat.com> Date: Wed, 29 Mar 2023 14:13:11 -0400 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.9.0 Subject: Re: [PATCH v9 0/13] implement dlmem() function To: stsp , libc-alpha@sourceware.org References: <20230318165110.3672749-1-stsp2@yandex.ru> <481ef2c5-a59f-dea4-7f5f-2fcd229a4c25@redhat.com> <7fab954f-4a3d-0df0-8211-a9697f3966c1@yandex.ru> From: Carlos O'Donell Organization: Red Hat In-Reply-To: <7fab954f-4a3d-0df0-8211-a9697f3966c1@yandex.ru> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-5.3 required=5.0 tests=BAYES_00,BODY_8BITS,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 3/29/23 13:03, stsp wrote: > > 29.03.2023 18:17, Carlos O'Donell пишет: >> On 3/18/23 12:50, Stas Sergeev via Libc-alpha wrote: >> >> A cover letter needs to explain in detail what the series does and why glibc >> should include the series e.g. use cases, workloads. > Attaching a cover letter for the upcoming > v10. The patch-set is not yet completed so > please let me know if the cover is now > adequate or needs more descriptions. > > Also it would be good if we settle on this > "elf header" stuff before v10 is released, > so I am eagerly waiting your reply on whether > the proof was enough or not. > I can try to bring you any proof you want, > but I can't be asked in a way "its possible > to create an elf that can break your impl, > please prove the opposite". Proving that > something can't be created, usually requires > infinite amount of time. So any proof you > want, needs to be finite and well-defined. > From 46e5095ebfe63be4dcd813c4237d6a491a3f9768 Mon Sep 17 00:00:00 2001 > From: Stas Sergeev > Date: Mon, 13 Feb 2023 18:15:34 +0500 > Subject: [PATCH v10 0/12] implement dlmem() function The most important thing is the reasons for the change and that should come first. > Changes in v10: > - squashed patch 1 as suggested by Adhemerval Zanella > - fixed a few bugs in an elf relocation machinery after various hot discussions > - added a new test tst-dlmem-dloff that demo-implements dlopen_with_offset() > > Changes in v9: > - use "zero-copy" machinery instead of memcpy(). It works on linux 5.13 > and newer, falling back to memcpy() otherwise. Suggested by Florian Weimer. > - implement fdlopen() using the above functionality. It is in a new test > tst-dlmem-fdlopen. Suggested by Carlos O'Donell. > - add DLMEM_DONTREPLACE flag that doesn't replace the backing-store mapping. > It switches back to memcpy(). Test-case is called tst-dlmem-shm. > > Changes in v8: > - drop audit machinery and instead add an extra arg (optional pointer > to a struct) to dlmem() itself that allows to install a custom premap > callback or to specify nsid. Audit machinery was meant to allow > controling over the pre-existing APIs like dlopen(), but if someone > ever needs such extensions to dlopen(), he can trivially implement > dlopen() on top of dlmem(). > > Changes in v7: > - add _dl_audit_premap audit extension and its usage example > > Changes in v6: > - use __strdup("") for l_name as suggested by Andreas Schwab > > Changes in v5: > - added _dl_audit_premap_dlmem audit extension for dlmem > - added tst-auditmod-dlmem.c test-case that feeds shm fd to dlmem() > > Changes in v4: > - re-target to GLIBC_2.38 > - add tst-auditdlmem.c test-case to test auditing > - drop length page-aligning in tst-dlmem: mmap() aligns length on its own > - bugfix: in do_mmapcpy() allow mmaps past end of buffer > > Changes in v3: > - Changed prototype of dlmem() (and all the internal machinery) to > use "const unsigned char *buffer" instead of "const char *buffer". > > Changes in v2: > - use instead of "../test-skeleton.c" > - re-target to GLIBC_2.37 > - update all libc.abilist files > > This patch-set implements the dlmem() function that allows to load > the solib from page-aligned memory buffer. It has lots of optional > functionality for the fine-grained control over the loading process. > The API looks as below: This needs to explain the workload that requires the API. Why that workload is generic. Any core system library, like glibc, is not the place for *all* APIs, but the place for the most generic building blocks for ISO C, POSIX, BSD, GNU, and Linux APIs. > /* Callback for dlmem. */ > typedef void * > (dlmem_premap_t) (void *mappref, size_t maplength, size_t mapalign, > void *cookie); > > /* Do not replace mapping created by premap callback. > dlmem() will then use memcpy(). */ > #define DLMEM_DONTREPLACE 1 > > struct dlmem_args { > /* Optional name to associate with the loaded object. */ > const char *soname; > /* Namespace where to load the object. */ > Lmid_t nsid; > /* dlmem-specific flags. */ > unsigned flags; > /* Optional premap callback. */ > dlmem_premap_t *premap; > /* Optional argument for premap callback. */ > void *cookie; > }; > > /* Like `dlmopen', but loads shared object from memory buffer. */ > extern void *dlmem (const unsigned char *buffer, size_t size, int mode, > struct dlmem_args *dlm_args); > > > In most cases dlm_args should just be set to NULL. It provides the > advanced functionality, most of which is obvious (soname, nsid). > The premap callback allows to set the relocation address for the solib. > More so, if DLMEM_DONTREPLACE flag is used, then the mapping established > by the premap callback, will not be replaced with the file-backed mapping. > In that case dlmem() have to use memcpy(), which is likely even faster > than mmaps() but doesn't end up with the proper /proc/self/map_files > or /proc/self/maps entries. So for example if the premap callback uses > MAP_SHARED, then with the use of the DLMEM_DONTREPLACE flag you can get > your solib relocated into a shared memory buffer. Such functionality > may be interesting for virtualized environments where the relocation > address may have a special constraints, like eg MAP_32BIT. It is not that the workload you have is unimportant, since importance is relative, but the workload is not generic enough for glibc to add dlmem() into the library. You need to have a very strong argument for the question "Why should this go into the library?" > -- > 2.37.2 > -- Cheers, Carlos.