From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.126.187]) by sourceware.org (Postfix) with ESMTPS id 489C9388A819 for ; Tue, 21 Jul 2020 12:00:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 489C9388A819 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=cygwin.com Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=corinna-cygwin@cygwin.com Received: from calimero.vinschen.de ([217.91.18.234]) by mrelayeu.kundenserver.de (mreue010 [212.227.15.167]) with ESMTPSA (Nemesis) id 1Mi2eP-1kSluc2iOX-00e3BK for ; Tue, 21 Jul 2020 14:00:03 +0200 Received: by calimero.vinschen.de (Postfix, from userid 500) id 225FBA8091F; Tue, 21 Jul 2020 14:00:02 +0200 (CEST) Date: Tue, 21 Jul 2020 14:00:02 +0200 From: Corinna Vinschen To: cygwin-developers@cygwin.com Subject: Re: More (?) steps toward jemalloc within Cygwin DLL Message-ID: <20200721120002.GN16360@calimero.vinschen.de> Reply-To: cygwin-developers@cygwin.com Mail-Followup-To: cygwin-developers@cygwin.com References: <20200630092412.GA3499@calimero.vinschen.de> <064ac876-bb84-6e05-1e46-ba594e03f057@maxrnd.com> <20200703101115.GA3499@calimero.vinschen.de> <75b43adc-ebcd-1d98-bd3e-33b8337f0475@maxrnd.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <75b43adc-ebcd-1d98-bd3e-33b8337f0475@maxrnd.com> X-Provags-ID: V03:K1:bToBHA7X/O+DqqCnaCty8JS+D612AMI6I6URNnWTJKj88pU4D2b C2FPBYtq1FQERboeNO5g7Gn/bHEvLi4fqbPla0/WAblWB1tGjZAoQdoUGG7eUdUub/kfZGN 0UNHaXjQMDAh5rg63Upo7TiXy5Q11TvA2kTU/0Xq/PCiVELZqi3NmDQ6ycHPuiIbKeqCEEH 6BsP1Yig4OrjNKGn+Vkeg== X-UI-Out-Filterresults: notjunk:1;V03:K0:CpMOlrg1K2U=:DqVxqHJ2jfS3A/mUR6yPyD nwp9i+MtQbhiYPkfiedAaVcDM47y0cZoy9oxmJ1IF0giAftoidzJ8xfXho//Aa8mDHH+0lynR G2o5aqDDRM4Qwl22euP+RP20cu0N/fKrGqzMyv2yQFRELp3S9DIvj8PtGCdjfNXtaFDZkBNZi ULhmClPTO5wEtQD2g/g8/SoIzb0cAcXPqVmzA8QHVuYN2ne9Bl7p2v7z2pPw52nyD6rQhMpkv 5yu58C0aBuwTTTb5t0KimVnGyqYJYhRK4OeNh8n0qse83WMeXIUNybPnF1+To/52/YrgukGkB yAndFxHGC5stmgWl2jxnYlrkzmHcGikROSmKD+9mfERaLY+oBppvbG0UwZxGbvmg/nmBy9snC yWyxkqkGVvidylQJiD1Xa5OukPMUUEK36ybUQMCFJRlQXKAWz4TVbcRhr41/5Cbauw6EiMFNC dir5bFD58rTE08fmNDbD55zD+6/aApFH5AE/8XWpSjvxzrThrTlPp/r1xGILsDgjH55AhcHrA FUSQbmoukFP/oJPPrpjileO9ZhZinfaFA3y7AWc6wEVXVBXJprJX0/oVa2m0saZsqC9+nRrRJ 9DmIpxC4oP+nA4O6jCLHU63yzKaRq00ovEMGykaBbMwts18hio9J1cSe980jk1QqKHohgyAm8 3S8SMeJRVYxoR5V4aIqYKXw/Qp2YSRjVS1CKh0eZ8AS3GX6m3Sy4skjXQLCuCbSa3+QBXdDeb Vi9bUu3EcEZsShYooHZz5YG+hTTM4+7FkHNUiAf6w0TLOkaw9+jh45lyvhJsNha1belgmOmPr Bv/f66iSVAu849jFhnclJ5VhdtukgcJ0CPUFQ/r+aLEnQKMleXZ/IGagvGX/UDdMvgkx7Wukd pb/Gz89Vbyo6CDqxrbuQ== X-Spam-Status: No, score=-99.4 required=5.0 tests=BAYES_00, GOOD_FROM_CORINNA_CYGWIN, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NEUTRAL, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jul 2020 12:00:07 -0000 Hi Mark, On Jul 21 01:50, Mark Geisert wrote: > Corinna Vinschen wrote: > > [...] > > ...the big problem are dependencies on malloc during Cygwin startup, > > especially in fork/exec, so the real challenge is to get the new malloc > > still let Cygwin processes start up correctly first time and especially > > in fork/exec situations, and to make sure the malloc bookkeeping > > survives fork/exec. > > O.K., understood. > > > These malloc dependencies sometimes crop up in the weirdest situations, > > so that's something to look out for. For instance, using pthread > > functions may call malloc as well. If a problem can be solved by > > changing another part of Cygwin, don't hesitate to discuss this! > > Yes, a couple of the malloc packages I'm testing want to allocate locks and > TLS slots right off the bat so there's nasty recursion possible. Given these locks are process-only, it's probably a good idea to overload the functions with equivalent WinAPI function calls using, for instance, slim R/W Locks. If these locks are stored as global NO_COPY objects, they don't even have to be initialized at process startup explicitely. If they have to be created dynamically, they should probably go into the cygheap, so they are duplicated automagically. > [...] > Here's a question I didn't expect to come up: If it turns out a home-grown > wrapper on the Win32 HeapXXX functions performs better (hint: it does, 2.5 > to 3 times better) than any malloc package derived from dlmalloc, is there > any reason why we ought not use it? Assuming it can be made to work for all > those cases you mentioned above, of course. It won't work with fork. Malloc'd memory has to be duplicated in the exact same spot during fork (think application pointers to allocated memory). Windows Heaps are ASLR'ed and the mechanism the heap is allocating and freeing memory is not built for reproducability in another process. That's why we have our own process heap given away via sbrk, as well as diligent bookkeeping of mmap'ed memory. > > The only danger here is this: If you manage to get dlmalloc replaced > > reliably, you *will* get a pink plush hippo! > > Oh, gee, that sounds like a really nice reward... Wow, I'm gonna have to do > this project now for sure! I'm really looking forward to it! Thanks, Corinna -- Corinna Vinschen Cygwin Maintainer