From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.126.134]) by sourceware.org (Postfix) with ESMTPS id 3B9B43858D37 for ; Tue, 30 Jun 2020 09:24:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 3B9B43858D37 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=cygwin.com Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=corinna-cygwin@cygwin.com Received: from calimero.vinschen.de ([24.134.7.25]) by mrelayeu.kundenserver.de (mreue011 [212.227.15.167]) with ESMTPSA (Nemesis) id 1MWRi7-1jIVfx2M4x-00XuDg for ; Tue, 30 Jun 2020 11:24:14 +0200 Received: by calimero.vinschen.de (Postfix, from userid 500) id 91E27A80859; Tue, 30 Jun 2020 11:24:12 +0200 (CEST) Date: Tue, 30 Jun 2020 11:24:12 +0200 From: Corinna Vinschen To: cygwin-developers@cygwin.com Subject: Re: More (?) steps toward jemalloc within Cygwin DLL Message-ID: <20200630092412.GA3499@calimero.vinschen.de> Reply-To: cygwin-developers@cygwin.com Mail-Followup-To: cygwin-developers@cygwin.com References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Provags-ID: V03:K1:OdalhLlJX0aMBEbyeQiXVOmiPaU6ED+Yrgc4YPpqjV0BJrJl5cu H5SCpFyxvo9U3rcx2FRcP2mjdsCqRtIoBbp6ySYCzk80tUamX/q3issQae6D6tYxZmZmqhk jYKSvoBf8YTX9vdA8ea4qFV3CaCgagcabLvGfaRK4EX0KysPkJs24mOue9z5AsKG6jAleJ/ bJRNIkogkMmxwjR+ZJTJg== X-UI-Out-Filterresults: notjunk:1;V03:K0:/XYoyOGwPxs=:zBgGChd/k8nmWrUrhQTOf3 YazT97/jzickhC7Fw9ZQZC2ss9CSPWCLce+2bLqTUCM8FECbQwcLkZHOzhHoZY0+qZGyy1DtS 6SUA/zkMOIF5vAFuDq1VX1MIj6bf0QKwmC+WxRIe5AuARdTPFKE3AprhBaV922bjq2biJ3qNO TWdHOYw5r61/Gu2/7pN2VpAfA8j92Xt//5BP84+gz4qL2WkaqvGleeKDWi/dY0S0Vm1zmhUdK dXPuh+o1pBdhMbe9wUGk+wmwTvUw84fz76iK+/WabTNJ2CVmQXMgO/Xf1MTiOuHFQm7wp46n0 fYW8VQwqxQOQDaer/73+6bVfz559wun7sGOrMW0k5pN0L/y4c6KmAGxYmc7H02f3RCHNOw/r7 uKAI1oDMEfG6o68o8eZUihPzgoNZ/5cAcyhiMp3mj0Fctoj4BnRRl336NDgKD6hcAtEYvZhqM QWku+omH5IGfftaIkddR9PmLNTKk1D9gT0gkDexPiJJjSQ7Bv9OxS6qk7LfqpmaPbOUBeLaUW X+SIZCz50vUDZM5u3/Br6fKa99i5z1HUi+6v1SiQF3Rslso2VV6Mk7gMyhuZ79pirE3wi7mU3 8iWpCWvd7U+gkvJ78Wac2OwV3TQAIBb+rw/DyuTMCLulWniMfokDFOMr1ep5IEqTAffG4wv6g CAY+TbObKj+Gh+8//e/IYWyHVFReU0AJ/jlS4rcBb9xAockmw8ws4iM2ReZ5rt9c5Brstyvd9 Z7M7hrmo5sRUqmfAhWtZE9jRtuo6vq7rbmOzVNrtapW/1icZZb4NcZq0Xl/IQ/jVj/Os1yYU4 qnqkvsiIVW8ZDGXh7nIpP2QWeilaYcWX+pJxPrXi968n1t+dsk2+kIrr5QssHVIi5XGq4l0 X-Spam-Status: No, score=-99.2 required=5.0 tests=BAYES_00, GOOD_FROM_CORINNA_CYGWIN, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NEUTRAL, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jun 2020 09:24:35 -0000 Hi Mark, On Jun 16 02:16, Mark Geisert wrote: > I'm just putting a flag down on this new (to me) territory. If somebody > else has claimed this project already, let me know and I'll shove off. No, please. Just keep on working on that. If you manage to get jemalloc working and replacing dlmalloc, this would be really great. > It wasn't much trouble to build a jemalloc.lib and statically link it into > the Cygwin DLL when the latter is built. I'm still learning which jemalloc > configure options are required in order to get complete test coverage and to > initialize properly within cygwin1.dll. > > I'm currently using the "supply your own malloc" mechanism provided by > Cygwin's malloc_wrapper.cc to overlay the usual dlmalloc-sourced functions > with replacements from jemalloc. I suspect there will be allocation > collisions ahead... The real problem here is this: __malloc_lock (); dl_foo_function (); __malloc_unlock (); This locking is what makes our dlmalloc even slower in multi-threaded scenarios because it disallows using malloc/free calls concurrently. If you get jemalloc working, it would be nice in itself, but the main improvement would be the ability to get rid of these __malloc_lock/ __malloc_unlock brackets. > One question has popped up though. I see from the docs that if one wants > jemalloc to run thread-aware, it wraps pthread_create() to find out when the > app has gone from single-threaded to multi-threaded. But in Cygwin's case > we'll additionally need to consider cygthreads, won't we? Yes, and the pthread_create call should not be performed in jemalloc if possible. The best solution is probably letting jemalloc always work under threading assumtion, right from the start. Thanks, Corinna -- Corinna Vinschen Cygwin Maintainer