Re: Bug in collation functions?

[Corinna Vinschen <corinna-cygwin@cygwin.com>]

[-- Attachment #1: Type: text/plain, Size: 2608 bytes --]

On Oct 30 10:07, Ken Brown wrote:
> Hi Corinna,
> 
> On 10/30/2015 8:03 AM, Corinna Vinschen wrote:
> >On Oct 29 18:21, Ken Brown wrote:
> >>The fallback I had in mind is to return the shorter string if they have
> >>different lengths and otherwise to revert to wcscmp.
> >
> >I had a longer look into this suggestion and the below code and it took
> >me some time to find out what bugged me with it:
> >
> >What about str/wcsxfrm?
> >
> >Per POSIX, calling strcmp on the result of strxfrm is equivalent to
> >calling strcoll (analogue with wcs*).  If you extend *coll to perform an
> >extra check on the length, you will have cases in which the above rule
> >fails.  You can't perform the length test on the result of *xfrm and
> >expect the same result as in *coll.
> >
> >In fact, when calling LCMapStringW with NORM_IGNORESYMBOLS (you would
> >have to do this anyway if we add this flag in *coll), the resulting
> >transformed strings created from the input strings "11" and "1.1" would
> >be identical, so a length test on the xfrm string is not meaningful at
> >all.
> >
> >The bottom line is, afaics, we must make sure that CompareStringW and
> >LCMapStringW are called the same way, and their result/output has to be
> >returned to the caller.  Performing an extra check in *coll which can't
> >be reliably performed in *xfrm is not feasible.
> >
> >Does that make sense?
> 
> Yes, I see the problem, and I don't see a good way around it.  So I think we
> probably have to leave things as they are and live with the fact that we
> can't do comparisons that ignore whitespace and punctuation.
> 
> The alternative of allowing str/wcscoll to return 0 on unequal strings
> doesn't seem feasible in view of Eric's comments.
> 
> What about the other issue I raised: Should setlocale return null to
> indicate an error if it's given an invalid locale name like en_DE.UTF-8?

Huh.  Interesting.  You're runing Windows10, right?  After some digging
it turns out there's a bug in W10.  LocaleNameToLCID() does *not* fail
and return with an error if it doesn't know a locale.  That would be too
simple I guess.  Rather, it returns a value LOCALE_CUSTOM_UNSPECIFIED,
0x1000.  So all unknown locales are now treated as custom locale.  Duh!
I fear the answer when trying to report this.  Probably it's a feature...

I applied a patch to workaround this feature.


Thanks for the testcase, btw :)


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]