From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qv1-xf2d.google.com (mail-qv1-xf2d.google.com [IPv6:2607:f8b0:4864:20::f2d]) by sourceware.org (Postfix) with ESMTPS id C14263853823 for ; Tue, 8 Jun 2021 16:51:13 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org C14263853823 Received: by mail-qv1-xf2d.google.com with SMTP id x6so6240679qvx.4 for ; Tue, 08 Jun 2021 09:51:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=ObBs+FMgkLULREXDiEiN2DzmVyiO9O/uXTJCBwsAw+g=; b=InkYn5vZgDUY4Ht3eL+KRFgBhTE75KkPWyOpZh77xQ61Nz0HD7fpe4yPZsPJeikW8k 2tFjDNeiEs5Rpop4GVxx+ZchLK3SE1McN0IfOe/VrmyrAeKADrtsYYasOFptMa8eaamE iU7EomK/cgF0SJSQ1SCHv6XHtjq+xKhVHVqcBqep//N/YiT55tjjLnLFLLVAVI0c4A7y YL2DOJCLK+23KE85Uit4h/ASRtlM3ZwMeAOoNaEGpaqXF72EGoonzSR5CPG/2zmMRkyq UPEfpUKb1vYgg/aZOQKIdOtCFu155uHQB0MAdTD6SntUwTqrdsn/FrlFRPNIINmKDkQB D5Vg== X-Gm-Message-State: AOAM533VrCpERJN42wvY3y07rS0MWmLkm+6o1ywdCqa/h7d2sdb98l8N DuZMuZr6xskUJMS1Pe9BNSD16hXBH+KR8A== X-Google-Smtp-Source: ABdhPJxYKTD4cSoiml1PfLAk4lIQetJ/XfuKeIThwYY3talvEdPG1XeLhzC8MsxzS0GZls7zDbxEIA== X-Received: by 2002:a05:6214:21e9:: with SMTP id p9mr974352qvj.55.1623171073275; Tue, 08 Jun 2021 09:51:13 -0700 (PDT) Received: from [192.168.1.4] ([177.194.59.218]) by smtp.gmail.com with ESMTPSA id q11sm9682873qtw.96.2021.06.08.09.51.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 08 Jun 2021 09:51:13 -0700 (PDT) Subject: Re: inode-based dlopen caching To: "Soni L." , Libc-help References: <62ae59a3-cada-f56c-5f34-5e0cc26ff4d4@linaro.org> <5eec9b26-e1cb-99ad-aab9-f57482b08987@linaro.org> From: Adhemerval Zanella Message-ID: Date: Tue, 8 Jun 2021 13:51:10 -0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-6.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-help@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-help mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jun 2021 16:51:15 -0000 On 08/06/2021 13:26, Soni L. wrote: > > > On 2021-06-08 10:14 a.m., Adhemerval Zanella wrote: >> >> >> On 07/06/2021 19:50, Soni L. wrote: >>> >>> >>> On 2021-06-07 6:53 p.m., Adhemerval Zanella wrote: >>>> >>>> >>>> On 05/06/2021 10:59, Soni L. via Libc-help wrote: >>>>> Currently dlopen caching is based on filenames, it'd be nice if it was >>>>> based on inodes to support better "re"loading (aka loading a new module >>>>> with the same name because unloading modules with threads is never a >>>>> good idea). This is good for stuff that deals with plugins. >>>>> >>>> >>>> What do you mean by 'caching' in this scenario? glibc does not maintain >>>> a cache of loaded libraries, different than other implementation it >>>> does try to unload the library on dlclose (there are cases where it >>>> is not readily possible due dependency chains). >>>> >>>> And I am not seeing how inote-bases dlopen really helps here, if inode >>>> changes means that file was potentially changed (so you will need to >>>> proper dclose it). I think using filenames is in fact the proper way >>>> here, since Linux does the hard lifting of the inode cache and provide >>>> fast file access and mmap support for shared libraries. >>>> >>> You can't unload a dlopen that uses threads, at least not safely. So for >>> all intents and purposes you can't unload it. Instead you need to tell >>> it you're done with it, but not unload it, and load the new one. But >>> that's the problem - the filename-based stuff means you can't load the >>> new one. >>> >> >> Sure you can unload a dlopen library, the API makes the program responsible >> to synchronize the access (since dlsym/dlvsym returns an function pointer). >> [snip] > If the shared library creates its own threads, those won't be killed > when the shared library is closed. If the shared library is then > unloaded, those threads will be running code from the void. That's a > problem. > > Being able to update the library, without getting rid of the original > (i.e. by unlinking the original file first), allows the program to > gracefully update loaded plugins without a full restart. > Again, this is a library issue that should be dealt by the provided API by the library (such as providing a cleanup handler to synchronize or cancel the threads execution). In this scenario you are describing you will end up with the library loaded in two different mapping with potentially two different code (since you are updating the library and dlclose might be a noop). I really don't see this as an improvement, it is rather a potential trigger to subtle issues, specially if the threads trying to sync with process shared resources.