Re: Bug in collation functions?

[Corinna Vinschen <corinna-cygwin@cygwin.com>]

[-- Attachment #1: Type: text/plain, Size: 3141 bytes --]

Hi Ken,

On Oct 29 18:21, Ken Brown wrote:
> On 10/29/2015 5:49 PM, Ken Brown wrote:
> >On 10/29/2015 2:42 PM, Ken Brown wrote:
> >>On 10/29/2015 12:51 PM, Eric Blake wrote:
> >>>Careful.  POSIX is proposing some wording that say that normal locales
> >>>should always implement a fallback of last resort (and that locales that
> >>>do not do so should have a special name including '@', to make it
> >>>obvious).  It is not standardized yet, but worth thinking about.
> >>>
> >>>http://austingroupbugs.net/view.php?id=938
> >>>http://austingroupbugs.net/view.php?id=963
> >>>
> >>>The intent of that wording is that if ignoring punctuation could cause
> >>>two strings to otherwise compare equal, the fallback of a total ordering
> >>>on all characters means that the final result of strcoll() will not be 0
> >>>unless the two strings are identical.
> >>
> >>In that case, I think Cygwin should start by using NORM_IGNORESYMBOLS in
> >>non-POSIX locales, with the goal of eventually moving toward emulating
> >>glibc.  I don't know what fallback glibc uses or how hard it would be to
> >>implement this on Cygwin.
> >
> >I withdraw this suggestion.  I took a look at the glibc code, and I
> >don't see any reasonable way for Cygwin to emulate it precisely.  On the
> >other hand, I have an idea for a simple fallback.  I'll play with it a
> >little and then submit a patch.
> 
> The fallback I had in mind is to return the shorter string if they have
> different lengths and otherwise to revert to wcscmp.  Using this, both
> Cygwin and Linux give the following comparisons:
> 
> "11" > "1.1" in POSIX locale
> "11" < "1.1" in en_US.UTF-8 locale
> "11" > "1 2" in POSIX locale
> "11" < "1.2" in en_US.UTF-8 locale
> "1 1" < "1.1" in POSIX locale
> "1 1" < "1.1" in en_US.UTF-8 locale
> 
> If this seems reasonable, I'll test it more extensively and then submit a
> patch.

I had a longer look into this suggestion and the below code and it took
me some time to find out what bugged me with it:

What about str/wcsxfrm?

Per POSIX, calling strcmp on the result of strxfrm is equivalent to
calling strcoll (analogue with wcs*).  If you extend *coll to perform an
extra check on the length, you will have cases in which the above rule
fails.  You can't perform the length test on the result of *xfrm and
expect the same result as in *coll.

In fact, when calling LCMapStringW with NORM_IGNORESYMBOLS (you would
have to do this anyway if we add this flag in *coll), the resulting
transformed strings created from the input strings "11" and "1.1" would
be identical, so a length test on the xfrm string is not meaningful at
all.

The bottom line is, afaics, we must make sure that CompareStringW and
LCMapStringW are called the same way, and their result/output has to be
returned to the caller.  Performing an extra check in *coll which can't
be reliably performed in *xfrm is not feasible.

Does that make sense?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]