From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by sourceware.org (Postfix) with ESMTPS id 8833A385380E for ; Wed, 28 Jul 2021 18:09:06 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 8833A385380E Received: by mail-pj1-x1031.google.com with SMTP id ca5so6353267pjb.5 for ; Wed, 28 Jul 2021 11:09:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=aXVCo5Ap2jv7AFjrvG4AzYNnkifZkxoHvGyOnT62JGc=; b=d20yF34zPWp7eYB2ATbynYK+DFGh2hRSjLtj5Dr1VcvwrTxXjI9Z2RrMjK6niYMhFi JUqBk+hvN1mILD82Be0jwPjiq6tIV+y4SQ4RkW7JAfKHtDTP07XzCyhcqLMUunWHwwDN wxd/HI3gh9wD3vSkwEBrYpzUH3/wKxX9CcV1BBppRpA7oU0P7lPn5EfnKgG5OLv4hGk1 X03ZZLv8+dvuRk/WLsffD8zfZq34HmLb/Uv6VTrJVNLwzEu4AkWvC9eCIp1VitnQ624T HqcX3P1aUqJcwCvpUrZN0aq4mNaKZ2omAQiqRJoGj9dLX6CRXLwNU76AsTCvhWYWaAdp GFDQ== X-Gm-Message-State: AOAM532Guif57nsZx32C1fo3MneKsdKhBWFm37DNGh5gz1mvJzLrIume 4hpQ7hAdzUT287wGDgBIdtn16PvmL0JYqQ== X-Google-Smtp-Source: ABdhPJwZcLl2Wtj2UyKcTZufpi6xOgTbF3HYndwhDN1ec3T2NDML9l+1hAUMviFM72usJzg8GCckcw== X-Received: by 2002:a05:6a00:8c7:b029:3a1:119b:736 with SMTP id s7-20020a056a0008c7b02903a1119b0736mr918905pfu.52.1627495745373; Wed, 28 Jul 2021 11:09:05 -0700 (PDT) Received: from ?IPv6:2804:431:c7cb:43e2:b556:b9a4:7138:fcc1? ([2804:431:c7cb:43e2:b556:b9a4:7138:fcc1]) by smtp.gmail.com with ESMTPSA id ha22sm4818022pjb.32.2021.07.28.11.09.04 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 28 Jul 2021 11:09:05 -0700 (PDT) Subject: Re: Enable preloading in dlopen-ed shared libraries? To: Fengkai Sun Cc: libc-help@sourceware.org References: <81d3637d-8ea4-4048-98de-584e813ddeaf@linaro.org> From: Adhemerval Zanella Message-ID: Date: Wed, 28 Jul 2021 15:09:02 -0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, URIBL_BLACK autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-help@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-help mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jul 2021 18:09:08 -0000 On 28/07/2021 03:17, Fengkai Sun wrote: > Hi Adhemerval, > > This seems very specific to add such complexity and extra internal state >> on loader and I want to understand better the issue you are trying to solve >> and why you can't use the current tools instead. >> >> Thanks for your reply! I now realize that changing the interface of the > dynamic linker needs reason and user cases that are solid and general > enough. But originally I planned to ask if this is a good idea and if there > is an existing interface to achieve this. Thank you for the information > provided! > > I will explain the original problem I want to solve here: > > We have an acadamic project which proposes a framework, and it should load > and isolate some external libraries completely, making them self-contained > and does not need to refer to anything outside their own namespace. > Ideally, dlmopen would suffice, but I found that dlmopen can only support > up to 16 namespaces, but we might use way more libraries than that. Do you need each library to be completely isolated from each other? Can't you load all of them in a single namespace or at least work around specific sets up to the 16 namespace limits? > > So I came up with another solution to simulate multi-namespace inside one: > use dlopen and load the libraries all into the default namespace. To > achieve complete isolation, I recursively renamed the dependencies of the > external libraries to bypass the limitation of allowing only one instance > inside a namespace. This is done by ELF patching tools, and I used patchelf. > For example, let's say there is a dependency graph like this: > libexternal.so -> lib1.so; libexternal.so -> libc.so > lib1.so -> libc.so > libc.so -> ld-linux.so > > After the renaming, the filename and dependency names of each library is > changed, so that ld.so will load another instance even if some libraries > already exist in the default namespace. > The dependency graph after renaming is like this: > libexternal.so -> lib1-1.so; libexternal.so -> libc-1.so > lib1-1.so -> libc-1.so > libc-1.so -> ld-linux-1.so > > This is not a good solution because my understanding of glibc is quite > shallow. dlopen seems to have problems in initializing libc and libpthread > when the filename of them are changed. So I tried not to rename libc or > libpthread and made them shared(just like the RTLD_UNIQUE flag proposed by > Vivek). You seems to want unlimited namespace, which is different than what Vivek is trying to fix with RTLD_SHARED / RTLD_UNIQUE. This should be doable by adding an extra list on _rtld_global and handle all places that access the _dl_ns link_namespaces (dlopen, dlclose, etc.). This would require some work on how we will handle static TLS (since this take in consideration the maximum number of namespaces). And you are correct, glibc internally check if DT_SONAME is the one expected by the hard-coded LIBC_SO. Trying to mess with it might broke in some places and that's exactly that dlmopen intends to provide. > > And we also want to interpose the symbols inside the external libraries. So > I asked if there is a way to provide a different definition for symbols in > local scope, just like what LD_PRELOAD does to global scope. > Currently I use patchelf to prepend a dependency, let's call it > libpreload.so, in the dynamic section of libexternal.so, so that ld.so will > search the symbol for libexternal.so and its dependencies first in > libpreload.so. Things are going well so far, and I will be very grateful if > you and other people on the list have a look at this solution. > It seems that our use case is really rare, and a full-functioned rtld-audit > interface can solve the problem for it. The dlmopen should work to provide the symbol resolution for a specific set of libraries: you first load on the namespace and then load the set of libraries. But without dlmopen I don't have a easy solution for you. If you are not bounded to the system glibc, you might change the DL_NNS internally to expected number of namespace. It has some implication, specially in the static TLS requirement, but I think it should work. I am not really against the local scope preload environment variable, but this will be another *very* specific feature that we will need to support indefinitely for an specific usercase that might not even used extensively (as you put this is a research project). This will also have some semantic issues we will need to sort out also, like how to represent an specific namespace (since glibc can return any value and Lmid_t does not need to map to an specific number).