Re: [PING^8][PATCH v12] Locales: Cyrillic -> ASCII transliteration [BZ #2872]

[Rafal Luzynski <digitalfreak@lingonborough.com>]

6.06.2019 11:42 Marko Myllynen <myllynen@redhat.com> wrote:
> 
> 
> Hi,
> 
> On 06/06/2019 02.49, Rafal Luzynski wrote:
> > 5.06.2019 08:47 "Diego (Egor) Kobylkin" <egor@kobylkin.com> wrote:
> >>
> >> ping
> > 
> > I second these pings.  Marko, Carlos, Siddhesh, Mike, is there anything
> > else I can do here?
> [...]
> My understanding of the overall situation here is that for 2.30 we try
> to have Cyrillic->ASCII transliteration added into the built-in C locale

Even if we have doubts and questions about transliteration of some
characters?  The question I asked applies to Cyrillic->ASCII.  On the other
hand, it does not apply to Cyrillic->Latin Extended (ISO 9) because it
strictly sticks to the "one-letter-to-one-letter" rule.

Maybe we should implement ISO 9 [1] without a fallback first because I am
pretty sure we are able to do it easily?

> and after that we would discuss more about translit rules used by other
> locales, and that this C locale patch is pending on Carlos to complete
> his verification efforts.
> 
> Does the above sound correct to you?

Not really unless we agree that we want to push the transliteration
even if it is not perfect and does not work good for everyone and we
are going to fix it later.

> [...] however are there
> other notable open questions left around the C locale rules in addition
> to this? (Ignoring the more generic tranlit rules or other locales for
> the time being.)

Yes, for example the letters like soft and hard sign which have uppercase
and lowercase variant and they are transliterate to diacritical signs but
I'd like to focus on this one first.

> > How should we handle the upper/lower case when a single Cyrillic letter
> > is transliterated to a Latin digraph (trigraph, etc.)?
> > 
> > Possible answers (Cyrillic -> Latin Extended -> ASCII):
> > 
> > 1. "Ш" -> "Š" -> "SH"
> > 
> >    e.g.: "Шема" -> "Šema" -> "SHema"
> >          "Схема" ----------> "Shema"
> > 
> > 2. "Ш" -> "Š" -> "Sh"
> > 
> >    e.g.: "Шема" -> "Šema" -> "Shema"
> >          "Схема" ----------> "Shema"
> > 
> > Personally I don't like the answer 1. because "SHema" looks weird
> > to me.  Egor in turn does not like the answer 2. because the output
> > string becomes ambiguous.
> > 
> > Should we maybe have a smart algorithm which would select the title
> > case or the upper case of the output characters depending on the
> > context in the word?  Note that it would not resolve the problem of
> > the output text being ambiguous.
> 
> It seems clear that there is no one right/wrong answer but it's a matter
> of preference, especially the way this currently works. It might be an
> improvement to output (for instance) SH instead of Sh if all the other
> letters of a word are upper-case as well but not sure what would help
> with the result being unambiguous.

I think you refer to the idea of implementing a smart algorithm which would
adapt the lower/upper case depending on the context but indeed it would
not resolve the problem of ambiguity.

So, the smart algorithm aside, what should be the preferred transliteration
rule?

Regards,

Rafal

[1] https://en.wikipedia.org/wiki/ISO_9