From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from m0.truegem.net (m0.truegem.net [69.55.228.47]) by sourceware.org (Postfix) with ESMTPS id 5DA51386F455 for ; Thu, 4 Mar 2021 09:05:17 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 5DA51386F455 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=maxrnd.com Authentication-Results: sourceware.org; spf=none smtp.mailfrom=mark@maxrnd.com Received: (from daemon@localhost) by m0.truegem.net (8.12.11/8.12.11) id 12495GhS062314 for ; Thu, 4 Mar 2021 01:05:16 -0800 (PST) (envelope-from mark@maxrnd.com) Received: from 162-235-43-67.lightspeed.irvnca.sbcglobal.net(162.235.43.67), claiming to be "[192.168.1.100]" via SMTP by m0.truegem.net, id smtpdt0U0VC; Thu Mar 4 01:05:13 2021 Subject: Re: segfault on 32bit cygwin snapshot To: cygwin@cygwin.com References: <9d7b9dc2-cb92-498b-7655-e9c618114c87@gmail.com> <20210221072954.db2dcbd523ed366e4dfcb0d0@nifty.ne.jp> <7480c946-8e02-aba2-c06f-6b39f630699f@gmail.com> <20210301095546.dce31a474bd0cec2c3518f87@nifty.ne.jp> <20210301212542.8b1749f92af62c01b008f25a@nifty.ne.jp> <20210302200308.62db4fe01f78fb35a538784f@nifty.ne.jp> <20210303185621.b048287526901af6a4c8200a@nifty.ne.jp> From: Mark Geisert Message-ID: Date: Thu, 4 Mar 2021 01:05:14 -0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.4 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, NICE_REPLY_A, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Mar 2021 09:05:20 -0000 Hi Corinna, Corinna Vinschen via Cygwin wrote: > [Ping Mark Geisert] > > On Mar 3 18:56, Takashi Yano via Cygwin wrote: >> Hi Corinna, >> >> On Tue, 2 Mar 2021 16:48:45 +0100 >> Corinna Vinschen wrote: >>> On Mar 2 20:03, Takashi Yano via Cygwin wrote: >>>>> The following check code does not work as expected if >>>>> newly build exe file is linked with old dll which calls >>>>> uname() as in this case. >>>>> >>>>> if (CYGWIN_VERSION_CHECK_FOR_UNAME_X) >>>>> return uname_x (in_name); >>>>> >>>>> Any idea? >>>> >>>> Ping Corinna? >>> >>> I have no idea how we could fix that, other than by rebuilding the DLLs >>> which call uname, too. We can't check the Cygwin build of all DLLs an >>> executable is linked to. >> >> I have checked all /usr/bin/*.dll in my cygwin installation and found >> following dlls call uname() rather than uname_x(). >> [...] >> Do you think rebuilding all of these (or maybe more) dlls is only >> the solution? > > No, we could also drop the above code snippet. > > Here's the problem: When we changed some datatypes in the early 2000s, > the old entry points have been conserved for backward compatibility, > while the new function using the new datatypes got a new name, e. g., > stat vs. _stat64. > > Next, libcygwin.a gets changed so that a newly built executable (using > the new datatypes as defined in the standard headers) calling stat is > redirected to _stat64. > > All is well for the next 15+ years or so. > > Then we discover that the exact same mechanism fails to work for > uname vs. the new uname_x in python. What happened? > > It turned out that python called uname dynamically Rather then just > calling uname, it calls dlopen();dlsym("uname"); > > This actually fetches the symbol for uname, not the symbol for uname_x. > The good old mechanism used for ages on standard function, fails as soon > as the caller uses dynamic loading of symbols. Surprise, surprise! > It was just never taken into consideration that standard libc functions > might be called dynamically, given it usually doesn't make sense. > > Given that this affects *all* of these redirected functions, we're in a > bit of a fix. Fortunately, for all other functions this only affects 32 > bit Cygwin, because the 64 bit version never had this backward > compatibility problem. > > Therefore, uname vs. uname_x is the only function affecting 64 bit > Cygwin as well, and that's why I added the above crude redirection only > to uname in the first place. > > So what we can do is this: > > - Either all old DLLs calling uname must be rebuilt. .. or patched (somehow). Also, IIUC you're both talking about Cygwin-supplied DLLs, not user builds of Cygwin or private DLLs. Dunno how we'd find every last one. > - Or we remove the above code snippet again and behave like for all > other redirected functions on 32 bit as well. Python's os.uname is > broken, but all the affected DLL sstart working again. I think you need to keep the above code snippet to handle old exe running with current Cygwin DLL, no? > Is there a way around that? I'm not quite sure, so let's brain storm > a bit, ok? > > - One thing we could try is to remove the above code, but add a python > hack to dlsym instead. This would let the "old" DLLs work again as > before and for python we could add a hack to dlsym, along these lines: > > if (CYGWIN_VERSION_CHECK_FOR_UNAME_X > && modulehandle == cygwin1.dll > && strcmp (symname, "uname")) > symname = "uname_x"; > > Thoughts? Other ideas? That's a sly fix, but it seems that it would do the job. That's good! On a different tack, I was thinking about how run time code could tell the difference between the versions of uname() being called. It can't. I looked at glibc and FreeBSD libc to see if they had to deal with this compatibility issue. It seems they don't, or I couldn't locate it if they do. But FreeBSD has an approach that looked interesting in another way. They have the standard uname() function taking the one usual arg. But it's just a wrapper on a worker function that takes the original arg plus another arg specifying the size of the fields in struct utsname. Paraphrasing, using Cygwin names: int uname(struct utsname *uptr) { return uname_x((int) _UTSNAME_LENGTH, uptr); } They allow the user to override what we call _UTSNAME_LENGTH. That seems like an invitation to exploit so I don't care for that. But what I was thinking is if we make that first arg to uname_x() be a uintptr_t, we could tell (with pretty good confidence) at run time inside uname_x() if we were passed a length (from new code passing two args) or a ptr-to-buf (from old code just passing one arg). I am unsure if this is useful. I am trying to consider the cross-product of exes and dlls we care about and whether we can identify which combination we have at run time. I think I need some time with a pad of paper, and maybe a drink. ..mark