[Bug localedata/30149] en_GB thinks "㐬" and "㒼" are the same character, other locales are fine

["carlos at redhat dot com" <sourceware-bugzilla@sourceware.org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=30149

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |NOTABUG
                 CC|                            |carlos at redhat dot com
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #4 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to infinity0 from comment #3)
> However I'd like to reiterate that even for the broken locales, only a few
> characters are affected, which is surprising behaviour and hides the
> brokenness.

Just for clarity, the locales aren't broken.

Collation is going to vary by locale.

The collation of CJK ideographs in en_GB and en_US is not defined since the
collation entries do not map the code points to a given weight.

This lack of a weight means that the characters with no weights map to the same
weight and are considered the same for the purposes of collation. This is how
we have always handled collation for characters with unknown ordering. They
collate the same, and so a "unique" removal based on collation will collapse
all such characters.

If would be an RFE to treat all characters without weights (we'd have to
identify them specially) as-if they were sorted by the internal code point
(like in C.UTF-8). This would be special and given them a consistent ordering,
but it certainly a deviation from what we've been doing and would need careful
review.

I'm marking this as RESOLVED/NOTABUG.

-- 
You are receiving this mail because:
You are on the CC list for the bug.