From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 31860 invoked by alias); 6 Jun 2019 21:31:20 -0000 Mailing-List: contact libc-locales-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: libc-locales-owner@sourceware.org Received: (qmail 31817 invoked by uid 89); 6 Jun 2019 21:31:18 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-4.6 required=5.0 tests=AWL,BAYES_00,BODY_8BITS,GARBLED_BODY,KAM_MANYTO,RCVD_IN_DNSWL_NONE autolearn=no version=3.3.1 spammy=sticks, H*f:@kobylkin.com, H*f:sk:DDiRMB9, signs X-HELO: shared-ano163.rev.nazwa.pl X-Spam-Score: -1 Date: Thu, 06 Jun 2019 21:31:00 -0000 From: Rafal Luzynski To: Marko Myllynen , "Diego (Egor) Kobylkin" , Carlos O'Donell , "libc-alpha@sourceware.org" , "libc-locales@sourceware.org" , Siddhesh Poyarekar Cc: Mike Fabian Message-ID: <1640311749.1550210.1559856673283@poczta.nazwa.pl> In-Reply-To: References: <2030695416.914859.1559778544120@poczta.nazwa.pl> Subject: Re: [PING^8][PATCH v12] Locales: Cyrillic -> ASCII transliteration [BZ #2872] MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SW-Source: 2019-q2/txt/msg00080.txt.bz2 6.06.2019 11:42 Marko Myllynen wrote: >=20 >=20 > Hi, >=20 > On 06/06/2019 02.49, Rafal Luzynski wrote: > > 5.06.2019 08:47 "Diego (Egor) Kobylkin" wrote: > >> > >> ping > >=20 > > I second these pings. Marko, Carlos, Siddhesh, Mike, is there anything > > else I can do here? > [...] > My understanding of the overall situation here is that for 2.30 we try > to have Cyrillic->ASCII transliteration added into the built-in C locale Even if we have doubts and questions about transliteration of some characters? The question I asked applies to Cyrillic->ASCII. On the other hand, it does not apply to Cyrillic->Latin Extended (ISO 9) because it strictly sticks to the "one-letter-to-one-letter" rule. Maybe we should implement ISO 9 [1] without a fallback first because I am pretty sure we are able to do it easily? > and after that we would discuss more about translit rules used by other > locales, and that this C locale patch is pending on Carlos to complete > his verification efforts. >=20 > Does the above sound correct to you? Not really unless we agree that we want to push the transliteration even if it is not perfect and does not work good for everyone and we are going to fix it later. > [...] however are there > other notable open questions left around the C locale rules in addition > to this? (Ignoring the more generic tranlit rules or other locales for > the time being.) Yes, for example the letters like soft and hard sign which have uppercase and lowercase variant and they are transliterate to diacritical signs but I'd like to focus on this one first. > > How should we handle the upper/lower case when a single Cyrillic letter > > is transliterated to a Latin digraph (trigraph, etc.)? > >=20 > > Possible answers (Cyrillic -> Latin Extended -> ASCII): > >=20 > > 1. "=D0=A8" -> "=C5=A0" -> "SH" > >=20 > > e.g.: "=D0=A8=D0=B5=D0=BC=D0=B0" -> "=C5=A0ema" -> "SHema" > > "=D0=A1=D1=85=D0=B5=D0=BC=D0=B0" ----------> "Shema" > >=20 > > 2. "=D0=A8" -> "=C5=A0" -> "Sh" > >=20 > > e.g.: "=D0=A8=D0=B5=D0=BC=D0=B0" -> "=C5=A0ema" -> "Shema" > > "=D0=A1=D1=85=D0=B5=D0=BC=D0=B0" ----------> "Shema" > >=20 > > Personally I don't like the answer 1. because "SHema" looks weird > > to me. Egor in turn does not like the answer 2. because the output > > string becomes ambiguous. > >=20 > > Should we maybe have a smart algorithm which would select the title > > case or the upper case of the output characters depending on the > > context in the word? Note that it would not resolve the problem of > > the output text being ambiguous. >=20 > It seems clear that there is no one right/wrong answer but it's a matter > of preference, especially the way this currently works. It might be an > improvement to output (for instance) SH instead of Sh if all the other > letters of a word are upper-case as well but not sure what would help > with the result being unambiguous. I think you refer to the idea of implementing a smart algorithm which would adapt the lower/upper case depending on the context but indeed it would not resolve the problem of ambiguity. So, the smart algorithm aside, what should be the preferred transliteration rule? Regards, Rafal [1] https://en.wikipedia.org/wiki/ISO_9