[PATCH] Add Phillipines peso symbol to translit_neutral

[PATCH] Add Phillipines peso symbol to translit_neutral

[Khem Raj]

Fixes
usr/share/i18n/locales/tl_PH:87: LC_MONETARY: unknown character in field `currency_symbol’

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 localedata/ChangeLog                | 4 ++++
 localedata/locales/translit_neutral | 2 ++
 2 files changed, 6 insertions(+)

diff --git a/localedata/ChangeLog b/localedata/ChangeLog
index 320bb8d..999e5b1 100644
--- a/localedata/ChangeLog
+++ b/localedata/ChangeLog
@@ -1,3 +1,7 @@
+2016-04-23  Khem Raj  <raj.khem@gmail.com>
+
+	* locales/translit_neutral: Add entry for U20B1.
+
 2016-04-16  Mike Frysinger  <vapier@gentoo.org>
 
 	* locales/fil_PH (LC_TELEPHONE): Change copy to en_PH.
diff --git a/localedata/locales/translit_neutral b/localedata/locales/translit_neutral
index b15d01f..02c60c5 100644
--- a/localedata/locales/translit_neutral
+++ b/localedata/locales/translit_neutral
@@ -567,6 +567,8 @@ include "translit_wide";""
 <U20AC> "<U0045><U0055><U0052>"
 % DRACHMA SIGN
 <U20AF> "<U0047><U0052><U0044>"
+% PHILIPPINES PESO SIGN
+<U20B1> "<U0050><U0068><U0050>"
 % HRYVNIA SIGN
 <U20B4> "<U0433><U0440><U043D>";"<U0055><U0041><U0048>"
 % INDIAN RUPEE SIGN
-- 
2.8.0

Re: [PATCH] Add Phillipines peso symbol to translit_neutral

[Florian Weimer]

* Khem Raj:

> Fixes
> usr/share/i18n/locales/tl_PH:87: LC_MONETARY: unknown character in field `currency_symbol’

Why does this fix the bug?

> +% PHILIPPINES PESO SIGN
> +<U20B1> "<U0050><U0068><U0050>"

Is this the correct transliteration?  It is not listed in Unicode.

Re: [PATCH] Add Phillipines peso symbol to translit_neutral

[Chris Leonard]

https://en.wikipedia.org/wiki/Philippine_peso_sign

see encoding section

On Sat, Apr 23, 2016 at 1:59 PM, Florian Weimer <fw@deneb.enyo.de> wrote:
> * Khem Raj:
>
>> Fixes
>> usr/share/i18n/locales/tl_PH:87: LC_MONETARY: unknown character in field `currency_symbol’
>
> Why does this fix the bug?
>
>> +% PHILIPPINES PESO SIGN
>> +<U20B1> "<U0050><U0068><U0050>"
>
> Is this the correct transliteration?  It is not listed in Unicode.

Re: [PATCH] Add Phillipines peso symbol to translit_neutral

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 892 bytes --]

On 23 Apr 2016 19:59, Florian Weimer wrote:
> * Khem Raj:
> > Fixes
> > usr/share/i18n/locales/tl_PH:87: LC_MONETARY: unknown character in field `currency_symbol’
> 
> Why does this fix the bug?

he didn't mention the encoding ... this is ISO-8859-1.  so the translit
changes it from the unicode/utf-8 ₱ to PHP as the former can't be encoded.

(i'm sure you know much of this already ... just trying to avoid anymore
confusion due to missing/assumed info)

> > +% PHILIPPINES PESO SIGN
> > +<U20B1> "<U0050><U0068><U0050>"
> 
> Is this the correct transliteration?  It is not listed in Unicode.

it is, the same as £ & ¥ -- just use the currency code in place of the
symbol.  where in the standard are you finding the definition for the
translit_neutral file ?  it doesn't seem to be handled by unicode-gen/
(yet?).  there's more symbols we need to add ...
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] Add Phillipines peso symbol to translit_neutral

[Florian Weimer]

* Mike Frysinger:

>> Is this the correct transliteration?  It is not listed in Unicode.
>
> it is, the same as £ & ¥ -- just use the currency code in place of the
> symbol.  where in the standard are you finding the definition for the
> translit_neutral file ?

Unicode sometimes give transliterations, but not in this case.  I
assume because the correct transliteration is locale-dependent.  If
you translate to the currency code, there are multiple choices for ¥
(for Japanese Yen and Chinese Yuan) and perhaps ₱ (as this was used in
other countries, not just the Philippines).  £ may also have been used
for Italian Lire.

Re: [PATCH] Add Phillipines peso symbol to translit_neutral

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1538 bytes --]

On 23 Apr 2016 22:15, Florian Weimer wrote:
> * Mike Frysinger:
> >> Is this the correct transliteration?  It is not listed in Unicode.
> >
> > it is, the same as £ & ¥ -- just use the currency code in place of the
> > symbol.  where in the standard are you finding the definition for the
> > translit_neutral file ?
> 
> Unicode sometimes give transliterations, but not in this case.  I
> assume because the correct transliteration is locale-dependent.  If
> you translate to the currency code, there are multiple choices for ¥
> (for Japanese Yen and Chinese Yuan) and perhaps ₱ (as this was used in
> other countries, not just the Philippines).  £ may also have been used
> for Italian Lire.

but we've made a choice already for ¥ & £ in this file:
% POUND SIGN
<U00A3> "<U0047><U0042><U0050>"
% YEN SIGN
<U00A5> "<U004A><U0050><U0059>"

Unicode has given a diff codepoint for the Lira: <U20A4>.  although they
did use £ in the past, and people might still (since it's more common on
keyboards).

you're right that the Yen-vs-Renminbi is not unambiguous, and considering
the way the Unicode consortium has treated CJK in the past (just see the
misguided "Han Unification" strategy), i don't expect this to change.  we
can pick an obvious "winner" for some locales (like zh_* & ja_*), but what
about common ones like en_US ?  do we leave ¥ unmapped ?

while you're right that other countries have used "the peso", have any
used the ₱ symbol ?  mexico for instance has used $ instead.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[PATCH] localedata: add more translit entries

[Mike Frysinger]

Some of the newer symbols we're using are missing translit entries which
causes troubles when generating the locales with older encodings.

tr_TR: ₺ -> "TL"
uz_UZ: Ê» -> "'"
common:
  ֏ -> "AMD"
  ₪ -> "ILS"
  ₱ -> "PHP"
  ₸ -> "KZT"
  ₾ -> "GEL"
---
 localedata/locales/tr_TR            |  3 +++
 localedata/locales/translit_neutral | 10 ++++++++++
 localedata/locales/uz_UZ            |  3 +++
 3 files changed, 16 insertions(+)

diff --git a/localedata/locales/tr_TR b/localedata/locales/tr_TR
index 3ef50dd..3d06d0e 100644
--- a/localedata/locales/tr_TR
+++ b/localedata/locales/tr_TR
@@ -4229,6 +4229,9 @@ class "combining_level3"; /
 
 translit_start
 include "translit_combining";""
+
+% TURKISH LIRA SIGN
+<U20BA> "<U0054><U004C>"
 translit_end
 END LC_CTYPE
 
diff --git a/localedata/locales/translit_neutral b/localedata/locales/translit_neutral
index b15d01f..e3639c0 100644
--- a/localedata/locales/translit_neutral
+++ b/localedata/locales/translit_neutral
@@ -353,6 +353,8 @@ include "translit_wide";""
 <U02DC> <U007E>
 % DOUBLE ACUTE ACCENT
 <U02DD> "<U0027><U0027>"
+% ARMENIAN DRAM SIGN
+<U058F> "<U0041><U004D><U0044>"
 % LATIN LETTER SMALL CAPITAL A
 <U1D00> <U0041>
 % LATIN LETTER SMALL CAPITAL AE
@@ -561,20 +563,28 @@ include "translit_wide";""
 <U20A7> "<U0050><U0074><U0073>"
 % WON SIGN
 <U20A9> "<U004B><U0052><U0057>"
+% NEW SHEQEL SIGN
+<U20AA> "<U0049><U004C><U0053>"
 % DONG SIGN
 <U20AB> "<U0110><U1ED3><U006E><U0067>";"<U0044><U006F><U006E><U0067>"
 % EURO SIGN
 <U20AC> "<U0045><U0055><U0052>"
 % DRACHMA SIGN
 <U20AF> "<U0047><U0052><U0044>"
+% PESO SIGN
+<U20B1> "<U0050><U0048><U0050>"
 % HRYVNIA SIGN
 <U20B4> "<U0433><U0440><U043D>";"<U0055><U0041><U0048>"
+% TENGE SIGN
+<U20B8> "<U004B><U005A><U0054>"
 % INDIAN RUPEE SIGN
 <U20B9> "<U0049><U004E><U0052>"
 % TURKISH LIRA SIGN
 <U20BA> "<U0054><U004C>"
 % RUBLE SIGN
 <U20BD> "<U0440><U0443><U0431>";"<U0052><U0055><U0042>"
+% GEORGIAN LARI SIGN
+<U20BE> "<U0047><U0045><U004C>"
 % PRESCRIPTION TAKE
 <U211E> "<U0052><U0078>"
 % TRADE MARK SIGN
diff --git a/localedata/locales/uz_UZ b/localedata/locales/uz_UZ
index b76a281..c4e7590 100644
--- a/localedata/locales/uz_UZ
+++ b/localedata/locales/uz_UZ
@@ -147,6 +147,9 @@ include "translit_combining";""
 <U049A> "<U0051>"        % k,= -> q
 <U0492> "<U0047><U0027>" % g-= -> g'
 <U04B2> "<U0048>"        % h,= -> h
+
+% various punctuation
+<U02BB> "<U0027>" % ‘ -> '
 translit_end
 END LC_CTYPE
 
-- 
2.7.4

Re: [PATCH] localedata: add more translit entries

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 2930 bytes --]


> On Apr 23, 2016, at 2:34 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> 
> Some of the newer symbols we're using are missing translit entries which
> causes troubles when generating the locales with older encodings.
> 
> tr_TR: ₺ -> "TL"
> uz_UZ: ʻ -> "'"
> common:
>  ֏ -> "AMD"
>  ₪ -> "ILS"
>  ₱ -> "PHP"
>  ₸ -> "KZT"
>  ₾ -> “GEL"


You can add if you like

Tested-by: Khem Raj <raj.khem@gmail.com>

I tested this patch and now all broken locales that I was excluding are fixed.


> ---
> localedata/locales/tr_TR            |  3 +++
> localedata/locales/translit_neutral | 10 ++++++++++
> localedata/locales/uz_UZ            |  3 +++
> 3 files changed, 16 insertions(+)
> 
> diff --git a/localedata/locales/tr_TR b/localedata/locales/tr_TR
> index 3ef50dd..3d06d0e 100644
> --- a/localedata/locales/tr_TR
> +++ b/localedata/locales/tr_TR
> @@ -4229,6 +4229,9 @@ class "combining_level3"; /
> 
> translit_start
> include "translit_combining";""
> +
> +% TURKISH LIRA SIGN
> +<U20BA> "<U0054><U004C>"
> translit_end
> END LC_CTYPE
> 
> diff --git a/localedata/locales/translit_neutral b/localedata/locales/translit_neutral
> index b15d01f..e3639c0 100644
> --- a/localedata/locales/translit_neutral
> +++ b/localedata/locales/translit_neutral
> @@ -353,6 +353,8 @@ include "translit_wide";""
> <U02DC> <U007E>
> % DOUBLE ACUTE ACCENT
> <U02DD> "<U0027><U0027>"
> +% ARMENIAN DRAM SIGN
> +<U058F> "<U0041><U004D><U0044>"
> % LATIN LETTER SMALL CAPITAL A
> <U1D00> <U0041>
> % LATIN LETTER SMALL CAPITAL AE
> @@ -561,20 +563,28 @@ include "translit_wide";""
> <U20A7> "<U0050><U0074><U0073>"
> % WON SIGN
> <U20A9> "<U004B><U0052><U0057>"
> +% NEW SHEQEL SIGN
> +<U20AA> "<U0049><U004C><U0053>"
> % DONG SIGN
> <U20AB> "<U0110><U1ED3><U006E><U0067>";"<U0044><U006F><U006E><U0067>"
> % EURO SIGN
> <U20AC> "<U0045><U0055><U0052>"
> % DRACHMA SIGN
> <U20AF> "<U0047><U0052><U0044>"
> +% PESO SIGN
> +<U20B1> "<U0050><U0048><U0050>"
> % HRYVNIA SIGN
> <U20B4> "<U0433><U0440><U043D>";"<U0055><U0041><U0048>"
> +% TENGE SIGN
> +<U20B8> "<U004B><U005A><U0054>"
> % INDIAN RUPEE SIGN
> <U20B9> "<U0049><U004E><U0052>"
> % TURKISH LIRA SIGN
> <U20BA> "<U0054><U004C>"
> % RUBLE SIGN
> <U20BD> "<U0440><U0443><U0431>";"<U0052><U0055><U0042>"
> +% GEORGIAN LARI SIGN
> +<U20BE> "<U0047><U0045><U004C>"
> % PRESCRIPTION TAKE
> <U211E> "<U0052><U0078>"
> % TRADE MARK SIGN
> diff --git a/localedata/locales/uz_UZ b/localedata/locales/uz_UZ
> index b76a281..c4e7590 100644
> --- a/localedata/locales/uz_UZ
> +++ b/localedata/locales/uz_UZ
> @@ -147,6 +147,9 @@ include "translit_combining";""
> <U049A> "<U0051>"        % k,= -> q
> <U0492> "<U0047><U0027>" % g-= -> g'
> <U04B2> "<U0048>"        % h,= -> h
> +
> +% various punctuation
> +<U02BB> "<U0027>" % ‘ -> '
> translit_end
> END LC_CTYPE
> 
> --
> 2.7.4
> 


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 204 bytes --]

Re: [PATCH] localedata: add more translit entries

[Marko Myllynen]

Hi,

On 2016-04-24 00:34, Mike Frysinger wrote:
> Some of the newer symbols we're using are missing translit entries which
> causes troubles when generating the locales with older encodings.
> 
> tr_TR: ₺ -> "TL"
> uz_UZ: Ê» -> "'"
> common:
>   ֏ -> "AMD"
>   ₪ -> "ILS"
>   ₱ -> "PHP"
>   ₸ -> "KZT"
>   ₾ -> "GEL"

The new common parts look good but I wonder when should we include
translit_neutral in locale files and when to add individual rules into
them instead (which are already defined in translit_neutral)?

Thanks,

-- 
Marko Myllynen

Re: [PATCH] localedata: add more translit entries

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1182 bytes --]

On 25 Apr 2016 16:16, Marko Myllynen wrote:
> On 2016-04-24 00:34, Mike Frysinger wrote:
> > Some of the newer symbols we're using are missing translit entries which
> > causes troubles when generating the locales with older encodings.
> > 
> > tr_TR: ₺ -> "TL"
> > uz_UZ: ʻ -> "'"
> > common:
> >   ֏ -> "AMD"
> >   ₪ -> "ILS"
> >   ₱ -> "PHP"
> >   ₸ -> "KZT"
> >   ₾ -> "GEL"
> 
> The new common parts look good but I wonder when should we include
> translit_neutral in locale files and when to add individual rules into
> them instead (which are already defined in translit_neutral)?

the vast majority of locales leverage translit_neutral (by way of copying
i18n).  the exceptions are:
	az_AZ
	crh_UA
	km_KH
	ku_TR
	tr_CY
	tr_TR
	tt_RU@iqtelif

all except km_KH eventually copy tr_TR.  seems like it should be possible
to rebase these two on top of translit_neutral (or at least i18n), but i
have no idea why they didn't in the first place.  considering how hacky
km_KH has been (looks like it was copied from lo_LA and then hacked to
bits to get a little km_KH), i wouldn't be surprised if it wasn't meant
to be by itself.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] localedata: add more translit entries

[Marko Myllynen]

Hi,

On 2016-04-26 07:40, Mike Frysinger wrote:
> On 25 Apr 2016 16:16, Marko Myllynen wrote:
>> On 2016-04-24 00:34, Mike Frysinger wrote:
>>> Some of the newer symbols we're using are missing translit entries which
>>> causes troubles when generating the locales with older encodings.
>>>
>>> tr_TR: ₺ -> "TL"
>>> uz_UZ: Ê» -> "'"
>>> common:
>>>   ֏ -> "AMD"
>>>   ₪ -> "ILS"
>>>   ₱ -> "PHP"
>>>   ₸ -> "KZT"
>>>   ₾ -> "GEL"
>>
>> The new common parts look good but I wonder when should we include
>> translit_neutral in locale files and when to add individual rules into
>> them instead (which are already defined in translit_neutral)?
> 
> the vast majority of locales leverage translit_neutral (by way of copying
> i18n).  the exceptions are:
> 	az_AZ
> 	crh_UA
> 	km_KH
> 	ku_TR
> 	tr_CY
> 	tr_TR
> 	tt_RU@iqtelif
> 
> all except km_KH eventually copy tr_TR.  seems like it should be possible
> to rebase these two on top of translit_neutral (or at least i18n), but i
> have no idea why they didn't in the first place.  considering how hacky
> km_KH has been (looks like it was copied from lo_LA and then hacked to
> bits to get a little km_KH), i wouldn't be surprised if it wasn't meant
> to be by itself.

Ok, thanks for checking, I think this can be investigated later in more
detail and now your patch is the right thing to do.

Cheers,

-- 
Marko Myllynen

Re: [PATCH] localedata: add more translit entries

[Chris Leonard]

+1

cjl

On Tue, Apr 26, 2016 at 2:04 AM, Marko Myllynen <myllynen@redhat.com> wrote:
> Hi,
>
> On 2016-04-26 07:40, Mike Frysinger wrote:
>> On 25 Apr 2016 16:16, Marko Myllynen wrote:
>>> On 2016-04-24 00:34, Mike Frysinger wrote:
>>>> Some of the newer symbols we're using are missing translit entries which
>>>> causes troubles when generating the locales with older encodings.
>>>>
>>>> tr_TR: ₺ -> "TL"
>>>> uz_UZ: ʻ -> "'"
>>>> common:
>>>>   ֏ -> "AMD"
>>>>   ₪ -> "ILS"
>>>>   ₱ -> "PHP"
>>>>   ₸ -> "KZT"
>>>>   ₾ -> "GEL"
>>>
>>> The new common parts look good but I wonder when should we include
>>> translit_neutral in locale files and when to add individual rules into
>>> them instead (which are already defined in translit_neutral)?
>>
>> the vast majority of locales leverage translit_neutral (by way of copying
>> i18n).  the exceptions are:
>>       az_AZ
>>       crh_UA
>>       km_KH
>>       ku_TR
>>       tr_CY
>>       tr_TR
>>       tt_RU@iqtelif
>>
>> all except km_KH eventually copy tr_TR.  seems like it should be possible
>> to rebase these two on top of translit_neutral (or at least i18n), but i
>> have no idea why they didn't in the first place.  considering how hacky
>> km_KH has been (looks like it was copied from lo_LA and then hacked to
>> bits to get a little km_KH), i wouldn't be surprised if it wasn't meant
>> to be by itself.
>
> Ok, thanks for checking, I think this can be investigated later in more
> detail and now your patch is the right thing to do.
>
> Cheers,
>
> --
> Marko Myllynen

Re: [PATCH] localedata: add more translit entries

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1699 bytes --]

On 26 Apr 2016 09:04, Marko Myllynen wrote:
> On 2016-04-26 07:40, Mike Frysinger wrote:
> > On 25 Apr 2016 16:16, Marko Myllynen wrote:
> >> On 2016-04-24 00:34, Mike Frysinger wrote:
> >>> Some of the newer symbols we're using are missing translit entries which
> >>> causes troubles when generating the locales with older encodings.
> >>>
> >>> tr_TR: ₺ -> "TL"
> >>> uz_UZ: ʻ -> "'"
> >>> common:
> >>>   ֏ -> "AMD"
> >>>   ₪ -> "ILS"
> >>>   ₱ -> "PHP"
> >>>   ₸ -> "KZT"
> >>>   ₾ -> "GEL"
> >>
> >> The new common parts look good but I wonder when should we include
> >> translit_neutral in locale files and when to add individual rules into
> >> them instead (which are already defined in translit_neutral)?
> > 
> > the vast majority of locales leverage translit_neutral (by way of copying
> > i18n).  the exceptions are:
> > 	az_AZ
> > 	crh_UA
> > 	km_KH
> > 	ku_TR
> > 	tr_CY
> > 	tr_TR
> > 	tt_RU@iqtelif
> > 
> > all except km_KH eventually copy tr_TR.  seems like it should be possible
> > to rebase these two on top of translit_neutral (or at least i18n), but i
> > have no idea why they didn't in the first place.  considering how hacky
> > km_KH has been (looks like it was copied from lo_LA and then hacked to
> > bits to get a little km_KH), i wouldn't be surprised if it wasn't meant
> > to be by itself.
> 
> Ok, thanks for checking, I think this can be investigated later in more
> detail and now your patch is the right thing to do.

i've filed two bugs for these so we don't lose track:
	https://sourceware.org/bugzilla/show_bug.cgi?id=20008 (km_KH)
	https://sourceware.org/bugzilla/show_bug.cgi?id=20009 (tr_TR)
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]