From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qv1-xf2e.google.com (mail-qv1-xf2e.google.com [IPv6:2607:f8b0:4864:20::f2e]) by sourceware.org (Postfix) with ESMTPS id 2BC23386FC1B for ; Tue, 8 Jun 2021 13:14:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 2BC23386FC1B Received: by mail-qv1-xf2e.google.com with SMTP id g12so10680574qvx.12 for ; Tue, 08 Jun 2021 06:14:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=b7T+SOo4lQNPuMsPnbKRbC63p3du6TRR4U1hZeiHxfQ=; b=P/EkcjCm4fVlAPnHuhcp0P4d6cD+LLvHFBQbtwUpyPYyOWjXrv1r/rmcYURMbKDCeh mjTyKywzDWE7aa/eY9vuiVbQwDZBdibMOsettBnMlKENmOfeCAldyGakjN2bDwfC34nK Ghet0JXSBnhsjuBU/DwwZLdtoSjbMToPR/JSp5nC9ieUmpnxwNrphKpIq0VouEZv3RVV d2Awj1kn3UAQkyzvATvgNPxY6C4hyi+fHA9LfoD9Vgt0ukRfWwEqhs5MiCeZW8lKyVYh 2FgYDIK29+O62cEFWeQ5xF/DLKbvZi0NkLVHrHnRtdwFCn3oqYqswGmEZ9W8PiRV4NcJ oljw== X-Gm-Message-State: AOAM531+tNOT7jDNUflsx8JviQkHvpnXbbQJeR9QaUYjh0pJZ6s8VwUN 9NjzJEdSYb6TiRnoMqoMBZvrVMxaMYwFzQ== X-Google-Smtp-Source: ABdhPJzfaGxbxhmsnyzyhQit/aMYKOnN4fkGRKAl9LnFoaORZQgFgYw1EJobCZDQmh/RHO9Rstu9Dg== X-Received: by 2002:ad4:57c4:: with SMTP id y4mr23464760qvx.12.1623158055483; Tue, 08 Jun 2021 06:14:15 -0700 (PDT) Received: from [192.168.1.4] ([177.194.59.218]) by smtp.gmail.com with ESMTPSA id 42sm10638369qtf.37.2021.06.08.06.14.14 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 08 Jun 2021 06:14:15 -0700 (PDT) Subject: Re: inode-based dlopen caching To: "Soni L." , Libc-help References: <62ae59a3-cada-f56c-5f34-5e0cc26ff4d4@linaro.org> From: Adhemerval Zanella Message-ID: <5eec9b26-e1cb-99ad-aab9-f57482b08987@linaro.org> Date: Tue, 8 Jun 2021 10:14:12 -0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-6.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-help@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-help mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jun 2021 13:14:20 -0000 On 07/06/2021 19:50, Soni L. wrote: > > > On 2021-06-07 6:53 p.m., Adhemerval Zanella wrote: >> >> >> On 05/06/2021 10:59, Soni L. via Libc-help wrote: >>> Currently dlopen caching is based on filenames, it'd be nice if it was >>> based on inodes to support better "re"loading (aka loading a new module >>> with the same name because unloading modules with threads is never a >>> good idea). This is good for stuff that deals with plugins. >>> >> >> What do you mean by 'caching' in this scenario? glibc does not maintain >> a cache of loaded libraries, different than other implementation it >> does try to unload the library on dlclose (there are cases where it >> is not readily possible due dependency chains). >> >> And I am not seeing how inote-bases dlopen really helps here, if inode >> changes means that file was potentially changed (so you will need to >> proper dclose it). I think using filenames is in fact the proper way >> here, since Linux does the hard lifting of the inode cache and provide >> fast file access and mmap support for shared libraries. >> > You can't unload a dlopen that uses threads, at least not safely. So for > all intents and purposes you can't unload it. Instead you need to tell > it you're done with it, but not unload it, and load the new one. But > that's the problem - the filename-based stuff means you can't load the > new one. > Sure you can unload a dlopen library, the API makes the program responsible to synchronize the access (since dlsym/dlvsym returns an function pointer). If I understood correctly what you are suggesting is making dlclose a noop, so a newer dlopen will also be a noop if it is essentially the same shared object (what happen if the shared library is updated and the inode keep the same?). This is design choice to actually unload the shared object on dlclose and changing it because it might incurs in concurrent issues on programs that do not synchronize its access is a really bad motivation. There are multiple better ways to handle it, either by wrapping with a more user-friendly API or using a high level language. If the motivation is to avoid the potential synchronization issues libc itself need to handle (such as TLS and other shared resources), it could be a better motivation. But even it is a trade off of keep allocated resources even when caller states it does not need them anymore. As fair I know this the design musl-libc has done. We could do it, but we have been fixing an improving the dynamic loader over time that makes this approach also complex and with not large benefits. Also, we still need to do some filename caching to handle things as RUNPATH/RPATH and SONAME; so the implementation to also take in consideration inode might add even more complexity and have more corner cases.