From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.17.24]) by sourceware.org (Postfix) with ESMTPS id 2B064386F42D for ; Mon, 14 Dec 2020 11:33:02 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 2B064386F42D Received: from calimero.vinschen.de ([24.134.7.25]) by mrelayeu.kundenserver.de (mreue106 [212.227.15.183]) with ESMTPSA (Nemesis) id 1M1qbm-1kqxxE2nou-002Fml for ; Mon, 14 Dec 2020 12:33:00 +0100 Received: by calimero.vinschen.de (Postfix, from userid 500) id 1100CA82BD5; Mon, 14 Dec 2020 12:33:00 +0100 (CET) Date: Mon, 14 Dec 2020 12:33:00 +0100 From: Corinna Vinschen To: cygwin-developers@cygwin.com Subject: Re: Failure during build of Python 3.8 via cygport Message-ID: <20201214113300.GA4560@calimero.vinschen.de> Reply-To: cygwin-developers@cygwin.com Mail-Followup-To: cygwin-developers@cygwin.com References: <758d2138-587b-2970-6c35-69d5c655a598@maxrnd.com> <0f498c92-409b-6ac8-d0ba-93ac753ab4b8@maxrnd.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <0f498c92-409b-6ac8-d0ba-93ac753ab4b8@maxrnd.com> X-Provags-ID: V03:K1:lhgAgZKZjcV1vZ8lgvtLnyHdxtWP0EhiEE8sxGTutVzzvLFlTyH /0BOEHJih+YopXD6NH+iuwwruP1pRtTv8HoVJeQ0LWqLdzkOhZB0DEpyMW684UMuSK3zAm2 5B3TDiIRAPbicD4hBLmKNoBl0aW+KIYrSj470m8G7IlRS3oETjttBJCbOtb3ba9iz7EFic/ zjOqhfEwBxXttJDPdOuEg== X-UI-Out-Filterresults: notjunk:1;V03:K0:u/Xi4bSXYi4=:rhjRGqvKut95DAcInTzVJU wdUX5+MfQ4fCFaN4khFiteqc3x+sSayhA5W0kMCubqnsvjPLNMxnLe7sRPmLvymxx+KpLrOdG F0EfE3/oN4yviPXv2mzRsbswG55rXQa8Aq5CE5x+DpbgU4oOfFQMziniQregfXjcIfk3CbHXo hl1zpWbHV+eD5g/k34KUFHLfHkhc/X+Mqsr+I+gXHXyPpfZ7E2NxEoVxyyH56cdrpoJMM9dR5 8h84YQ7D5WsDCMs4L9SuisJKSI3cHIHZf5dqMrP+rHx4yWUC7jtSHxC75Cx29HKfzp1UXyUMw 921KXeWHp6NXmQl26TQPvoM5KG4/59WDV8A1GIbnAz1NgJly/4fY1gUF+RYBAYlVBBp+oOicX zE5qz1I/j9tS/mlqd2Mn/IUHGGW9PJqPoTdLv39S0ighB6GZ5y90QT6tdos5SgXMIT1LKcYXE aZPU7JsHJg== X-Spam-Status: No, score=-105.2 required=5.0 tests=BAYES_00, BODY_8BITS, GIT_PATCH_0, GOOD_FROM_CORINNA_CYGWIN, JMQ_SPF_NEUTRAL, KAM_DMARC_NONE, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NEUTRAL, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Dec 2020 11:33:06 -0000 On Dec 14 02:42, Mark Geisert wrote: > [Bringing this thread over from cygwin-apps...] > > Mark Geisert wrote: > > Mark Geisert wrote: > > > This seems to be a problem setting up a platform-specific build > > > directory. The sysconfig.py script wants to use "lib." + platform + > > > pythonversion but the platform string somehow gets corrupted into > > > non-utf8 bytes.  For instance, building Python 3.8 comes up with: > > >      lib.cygwin-\365\377\377o-\377o-3.8 > > > as the directory name.  Broken, but could work.  The build failure > > > happens because the script tries to write this directory name into a > > > file but it's not a valid utf8 string.  The directory name should > > > have been: > > >      lib.cygwin-3.2.0-x86_64-3.8 > > > > And the corruption is due to something about a recent change to the > > operation of Cygwin's uname() function.  The change was introduced in > > Cygwin API version 335; I'm running 340 on my test machine.  This being > > a fairly recent change might possibly explain why nobody else has run > > into this issue yet. > > > > Basically, os.uname within Python is calling Cygwin's uname() passing > > the address of a buffer declared to be 'struct utsname'.  The structure > > layout changed in API 335.  What I've hit is a mismatch between what > > Python expects and Cygwin delivers. > > Brian Inglis pointed out the API 335 change was made a couple years ago, so > strike the quoted conjecture about why nobody else has hit this. A new > conjecture follows... > > Debugging Python's os.uname showed Cygwin's uname() being called with a > 'struct uname' as defined in /usr/include/sys/utsname.h, which is fine. But > it's the "old" pre-335 uname() interface being called, not the "new" 335+ > interface uname_x(). Note that the famous 'mkimport' script, used when > building the Cygwin DLL, has an arg "--replace=uname=uname_x" which I > believe is supposed to equate the two names so the code in uname_x() is > called whether the interface is uname_x() or uname(). That's not happening. > > My first thought was, dang it, my "optimizations" of mkimport broke it. But > that's not the case. Restoring a previous version of mkimport doesn't help. > > 'nm' shows that both uname and uname_x exist in libcygwin.a and also > cygdll.a. And a newly-created Cygwin DLL has both functions, with different > addresses. > That's wrong, isn't it? No, it isn't. The old uname entry point is still required for old applications built prior to API version 335. If you build a new application it will actually call uname_x when it tries to call uname since that's how the libcygwin.a layer translates it. It's a bit unfortunate that uname_x is exported by libcygwin.a as well (in theory it should only export uname, pointing to uname_x in the DLL), but this is how it always was, as you can see in the 32 bit build with symbols only available there, e. g. aclcheck: $ nm libcygwin.a | grep aclcheck 00000000 I __imp___aclcheck 00000000 I __imp___aclcheck32 00000000 I __imp__aclcheck 00000000 T __aclcheck U __imp___aclcheck 00000000 T __aclcheck32 U __imp___aclcheck32 00000000 T _aclcheck U __imp___aclcheck32 In retrospect, uname_x should be named _uname_x or so, with a leading underscore, so as not to pollute the namespace, but either way, that isn;t your problem. The problem here might be that you get the old uname function if you dlopen the cygwin dll and dlsym(hdl, "uname"). Is that the case in python? If so, I have a simple, dirty workaround below. Can you check if that's the problem, please? Corinna >From 45519fc1faa616e18fc5e2d78c483699e0f1f475 Mon Sep 17 00:00:00 2001 From: Corinna Vinschen Date: Mon, 14 Dec 2020 12:29:23 +0100 Subject: [PATCH] Cygwin: Make sure newer apps get uname_x even when loading uname dynamically if an application built after API version 334 loads uname dynamically, it actually gets the old uname, rather than the new uname_x. Fix this by checking the apps API version in uname and call uname_x instead, if it's a newer app. Signed-off-by: Corinna Vinschen --- winsup/cygwin/include/cygwin/version.h | 3 +++ winsup/cygwin/uname.cc | 7 +++++++ 2 files changed, 10 insertions(+) diff --git a/winsup/cygwin/include/cygwin/version.h b/winsup/cygwin/include/cygwin/version.h index f21aba72669a..b39d0dbe95bb 100644 --- a/winsup/cygwin/include/cygwin/version.h +++ b/winsup/cygwin/include/cygwin/version.h @@ -74,6 +74,9 @@ details. */ #define CYGWIN_VERSION_CHECK_FOR_EXTRA_TM_MEMBERS \ (CYGWIN_VERSION_USER_API_VERSION_COMBINED >= 272) +#define CYGWIN_VERSION_CHECK_FOR_UNAME_X \ + (CYGWIN_VERSION_USER_API_VERSION_COMBINED >= 335) + #define CYGWIN_VERSION_CYGWIN_CONV 181 /* API_MAJOR 0.0: Initial version. API_MINOR changes: diff --git a/winsup/cygwin/uname.cc b/winsup/cygwin/uname.cc index 350216681fa1..48f7dda6029b 100644 --- a/winsup/cygwin/uname.cc +++ b/winsup/cygwin/uname.cc @@ -92,7 +92,14 @@ struct old_utsname extern "C" int uname (struct utsname *in_name) { + /* This occurs if the application fetches the uname symbol dynamically. + We must call uname_x for newer API versions, otherwise the idea of + struct utsname doesn't match. */ + if (CYGWIN_VERSION_CHECK_FOR_UNAME_X) + return uname_x (in_name); + struct old_utsname *name = (struct old_utsname *) in_name; + __try { char *snp = strstr (cygwin_version.dll_build_date, "SNP"); -- 2.29.2