From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 109597 invoked by alias); 29 Oct 2015 18:42:43 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 109584 invoked by uid 89); 29 Oct 2015 18:42:42 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.6 required=5.0 tests=AWL,BAYES_00,RP_MATCHES_RCVD,SPF_HELO_PASS,SPF_PASS autolearn=ham version=3.3.2 X-HELO: limerock02.mail.cornell.edu Received: from limerock02.mail.cornell.edu (HELO limerock02.mail.cornell.edu) (128.84.13.242) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Thu, 29 Oct 2015 18:42:41 +0000 X-CornellRouted: This message has been Routed already. Received: from authusersmtp.mail.cornell.edu (granite4.serverfarm.cornell.edu [10.16.197.9]) by limerock02.mail.cornell.edu (8.14.4/8.14.4_cu) with ESMTP id t9TIgcPf007632 for ; Thu, 29 Oct 2015 14:42:39 -0400 Received: from [10.13.22.3] (50-192-21-217-static.hfc.comcastbusiness.net [50.192.21.217]) (authenticated bits=0) by authusersmtp.mail.cornell.edu (8.14.4/8.12.10) with ESMTP id t9TIgbIu016891 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Thu, 29 Oct 2015 14:42:38 -0400 Subject: Re: Bug in collation functions? To: cygwin@cygwin.com References: <563148AF.1000502@cornell.edu> <5631996D.7040908@redhat.com> <20151029075050.GE5319@calimero.vinschen.de> <20151029083057.GH5319@calimero.vinschen.de> <56321815.7000203@cornell.edu> <20151029153516.GJ5319@calimero.vinschen.de> <56323F2E.4030807@cornell.edu> <56324598.9060604@cornell.edu> <56324E82.7000402@redhat.com> From: Ken Brown Message-ID: <563268A4.6000005@cornell.edu> Date: Thu, 29 Oct 2015 21:58:00 -0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <56324E82.7000402@redhat.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-IsSubscribed: yes X-SW-Source: 2015-10/txt/msg00553.txt.bz2 On 10/29/2015 12:51 PM, Eric Blake wrote: > On 10/29/2015 10:13 AM, Ken Brown wrote: > >> Never mind. My test case was flawed, because it didn't check for the >> possibility that wcscoll might return 0. Here's a revised definition of >> the "compare" function: >> >> void >> compare (const wchar_t *a, const wchar_t *b, const char *loc) >> { >> setlocale (LC_COLLATE, loc); >> int res = wcscoll (a, b); >> char c = res < 0 ? '<' : res > 0 ? '>' : '='; >> printf ("\"%ls\" %c \"%ls\" in %s locale\n", a, c, b, loc); >> } >> >> With this change (and the use of NORM_IGNORESYMBOLS) the test returns >> the following on Cygwin: >> >> $ ./wcscoll_test >> "11" > "1.1" in POSIX locale >> "11" = "1.1" in en_US.UTF-8 locale >> "11" > "1 2" in POSIX locale >> "11" < "1 2" in en_US.UTF-8 locale >> >> It still differs from Linux, but it's good enough to make the emacs test >> pass. Moreover, this behavior actually seems more reasonable to me than >> the Linux behavior. After all, if you're ignoring punctuation, how can >> you decide which of "11" or "1.1" comes first? > > Careful. POSIX is proposing some wording that say that normal locales > should always implement a fallback of last resort (and that locales that > do not do so should have a special name including '@', to make it > obvious). It is not standardized yet, but worth thinking about. > > http://austingroupbugs.net/view.php?id=938 > http://austingroupbugs.net/view.php?id=963 > > The intent of that wording is that if ignoring punctuation could cause > two strings to otherwise compare equal, the fallback of a total ordering > on all characters means that the final result of strcoll() will not be 0 > unless the two strings are identical. In that case, I think Cygwin should start by using NORM_IGNORESYMBOLS in non-POSIX locales, with the goal of eventually moving toward emulating glibc. I don't know what fallback glibc uses or how hard it would be to implement this on Cygwin. Here's a tangentially related issue, also motivated by a failing emacs test: Should setlocale return null to indicate an error if it's given an invalid locale name? This happens on Linux but not on Cygwin, as the following modified test case shows: $ cat wcscoll_test.c #include #include #include void compare (const wchar_t *a, const wchar_t *b, const char *loc) { if (! setlocale (LC_COLLATE, loc)) printf ("Unable to set locale to %s\n", loc); else { int res = wcscoll (a, b); char c = res < 0 ? '<' : res > 0 ? '>' : '='; printf ("\"%ls\" %c \"%ls\" in %s locale\n", a, c, b, loc); } } int main () { compare (L"11", L"1.1", "POSIX"); compare (L"11", L"1.1", "en_US.UTF-8"); compare (L"11", L"1 2", "POSIX"); compare (L"11", L"1 2", "en_US.UTF-8"); compare (L"11", L"1 2", "en_DE.UTF-8"); } On Cygwin (with NORM_IGNORESYMBOLS), the output is "11" > "1.1" in POSIX locale "11" = "1.1" in en_US.UTF-8 locale "11" > "1 2" in POSIX locale "11" < "1 2" in en_US.UTF-8 locale "11" < "1 2" in en_DE.UTF-8 locale but on Linux it is "11" > "1.1" in POSIX locale "11" < "1.1" in en_US.UTF-8 locale "11" > "1 2" in POSIX locale "11" < "1 2" in en_US.UTF-8 locale Unable to set locale to en_DE.UTF-8 Ken -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple