[Bug localedata/23774] lv_LV collates Y/y incorrectly

["maiku.fabian at gmail dot com" <sourceware-bugzilla@sourceware.org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=23774

--- Comment #1 from Mike FABIAN <maiku.fabian at gmail dot com> ---
(In reply to Danko Alexeyev from comment #0)
> Commit 159738548130d5ac4fe6178977e940ed5f8cfdc4 introduced this change in
> the lv_LV locale:
> 
> -<U0079> <i>;<PCL>;<MIN>;IGNORE % y
> -<U0059> <i>;<PCL>;<CAP>;IGNORE % Y
> +<U0079> <S0069>;<LOWLINE>;<MIN>;IGNORE % y
> +<U0059> <S0069>;<LOWLINE>;<CAP>;IGNORE % Y
> 
> I don't know what "PCL" meant and whether "Y" was supposed to be "BASE" in
> the first place, but "LOWLINE" certainly looks like a bug.

PCL was an old collation symbol which was used in an older version of
the glibc/localedata/locales/iso14651_t1_common file.  It was a second
level collation symbol.

To get the right sort order, replacing it by any existing secondary
collation symbol except "BASE" works fine here.

The current glibc/localedata/locales/iso14651_t1_common contains:

    % Second-level collating symbols

    collating-symbol <BASE>
    collating-symbol <LOWLINE>  % COMBINING LOW LINE
    collating-symbol <PSILI>  % COMBINING COMMA ABOVE
    collating-symbol <DASIA>  % COMBINING REVERSED COMMA ABOVE
    collating-symbol <AIGUT>  % COMBINING ACUTE ACCENT
    ...

<BASE> means base letter, all the  following collation symbols can be used
to indicate secondary differences to base letters. As there is nothing
particularly appropriate for the difference between i and y, it doesn’t
really matter which one is used, so I did choose the first one, LOWLINE.

> Letter Y is not present in the Latvian alphabet, however it is present in
> Latgalian and is located after I, which is what the CLDR rule seems to
> suggest:
> 
> &I<<y<<<Y

This rule means that y is sorted after I *but* only as secondary difference
("<" is a primary difference, "<<" is a secondary difference, "<<<" is a
tertiary difference).
Secondary differences are "accent" differences, i.e. y is treated here
not as a really different letter from I (That would be a primary difference),
but as a "accent" variation of I. Tertiary differences are often used
for upper/lower differences, which is the case here, i.e. the difference
between y and Y is a upper/lower difference.

If you look at the sorting test file for lv_LV.UTF-8:

glibc/localedata/lv_LV.UTF-8.in

you will find that it contains:

i
I
ī
Ī
y
Y
ia
Ia
īa
Īa
ya
Ya
ib
Ib
īb
Īb
yb
Yb

If y were primary different from i, ya would be sorted *after* ib.
But as it is only a secondary difference, the primary difference between 
a and b decides the order for the strings ya and ib.

> I found this by accident while investigating the result of this command on
> my system (with LANG being lv_LV.UTF-8)
> 
> $ echo abcxyz | grep -Eo '[a-z]+'
> abcx
> z
> 
> I'm sorry if I misunderstood something as I've never worked with either
> glibc or CLDR locales directly before.

This fails for other reasons, not because of the use of LOWLINE.

-- 
You are receiving this mail because:
You are on the CC list for the bug.