From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 94477 invoked by alias); 20 Jul 2017 20:19:43 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Received: (qmail 94463 invoked by uid 89); 20 Jul 2017 20:19:43 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00,SPF_HELO_PASS,SPF_NEUTRAL autolearn=no version=3.3.2 spammy=pose, thousands, our, protect X-HELO: hera.aquilenet.fr Date: Thu, 20 Jul 2017 20:19:00 -0000 From: Samuel Thibault To: Carlos O'Donell Cc: libc-alpha@sourceware.org Subject: Re: [PATCH] Add RTLD_RELOAD to dlopen Message-ID: <20170720201934.o3ob5tipivwtidy7@var.youpi.perso.aquilenet.fr> References: <20170720191517.xah6rggoaeqgbokf@var.youpi.perso.aquilenet.fr> <169b9578-5546-2b44-1b5c-94e9215efc24@redhat.com> <20170720193428.qa6z7tvoq5xkssu6@var.youpi.perso.aquilenet.fr> <5dd4f2da-c951-ca9e-9a50-5b7bca547482@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5dd4f2da-c951-ca9e-9a50-5b7bca547482@redhat.com> User-Agent: NeoMutt/20170113 (1.7.2) X-SW-Source: 2017-07/txt/msg00758.txt.bz2 Carlos O'Donell, on jeu. 20 juil. 2017 16:03:46 -0400, wrote: > On 07/20/2017 03:34 PM, Samuel Thibault wrote: > > Carlos O'Donell, on jeu. 20 juil. 2017 15:31:38 -0400, wrote: > >> On 07/20/2017 03:15 PM, Samuel Thibault wrote: > >>> In our parallel programming projects, we would like to load some DSO > >>> several times within the same process, because we want to share the > >>> addresse space for passing data pointers between parallel executions, > >>> and the DSO has global variables and such which we want to see > >>> duplicated. > >>> > >>> Unfortunately, dlopen() does not re-load the DSO when it is already > >>> loaded. One workaround is to cp the file under another name, but that's > >>> ugly and does not share the memory pages. > >>> > >>> The patch proposed here simply adds an RTLD_RELOAD flag which disables > >>> checking for the DSO being already loaded, thus always loading the DSO > >>> again. There is no actual code modification, only the addition of two > >>> if()s and reindent. > >> > >> This is what Solaris designed dlmopen() for, is there any reason you > >> can't use dlmopen()? > > > > Because only that DSO should be reloaded. All the dependencies are fine > > to use as such, to avoid memory usage duplication. > > Is that really a problem? Have you measured this? Yes and yes. We are targetting loadling like thousands of replicates of the DSO, to emulate a very large parallel execution. > The kernel will share every page except data/bss. But that also eats addressing space and VMAs. > Using dlmopen will protect you from all kinds of issues with dependent > libraries only supporting a single global state since you'll get unique > state for each of the loaded libraries. Sure, but it's really too costly for us, and it loads glibc several times in the process, which can pose other problems. > I see RTLD_RELOAD as an shortcut to simply having multiple copies of the > shared library on disk. I believe you could use hardlinks and glibc should > load each as a unique library. Hardlinks would have the same st_ino, so libc would detect it is already loaded. Samuel