QUESTION: LC_COLLATE minimal requirements?

QUESTION: LC_COLLATE minimal requirements?

[Harshula]

Hi,

I was unable to find much documentation on LC_COLLATE except for [1].
Hence I have a few questions.

Firstly, some background information. The Sinhala collation sequence
(SLS1134) is relatively simple.

* It does not have multiple characters mapping to a single
collation element.
* It does not consider composed and decomposed dependent vowels as
equivalent [2].
* It does not have to deal with secondary and tertiary weights.
* It has a few simple tailoring rules [3] that need to be applied to the
DUCET [4].


Q1) Is it a requirement to use the collating-symbol keyword to define
ALL symbols? If not, is this patch sufficient and acceptable for glibc?
http://cvs.savannah.gnu.org/viewvc/sinhala/patches/iso14651_t1_common-glibc.patch?root=sinhala&view=log

Q2) Instead of explicitly listing all the characters in order, is it
possible to use the reorder-after keyword to only define variations to
the DUCET?

Q3) I couldn't find any documentation on:

translit_start
include  "translit_combining";""
translit_end

/usr/share/i18n/locales/translit_combining
------------------------------------------
% SINHALA VOWEL SIGN DIGA KOMBUVA
<U0DDA> "<U0DD9><U0DCA>"
% SINHALA VOWEL SIGN KOMBUVA HAA AELA-PILLA
<U0DDC> "<U0DD9><U0DCF>"
% SINHALA VOWEL SIGN KOMBUVA HAA DIGA AELA-PILLA
<U0DDD> "<U0DDC><U0DCA>"
% SINHALA VOWEL SIGN KOMBUVA HAA GAYANUKITTA
<U0DDE> "<U0DD9><U0DDF>"
------------------------------------------

Does translit_start have an affect on LC_COLLATE?

Thanks,
#

[1]
http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap07.html

[2]
http://sourceforge.net/mailarchive/forum.php?thread_name=1223803982.4898.16.camel%40B1.HOME&forum_name=sinhala-technical

[3]
http://www.nongnu.org/sinhala/doc/howto/sinhala-howto.html#DEV-DATABASES

[4] http://unicode.org/Public/UCA/latest/allkeys.txt

Re: QUESTION: LC_COLLATE minimal requirements?

[Pravin S]

2008/10/12 Harshula <harshula@gmail.com>:
> Hi,
>
> I was unable to find much documentation on LC_COLLATE except for [1].
> Hence I have a few questions.
>
> Firstly, some background information. The Sinhala collation sequence
> (SLS1134) is relatively simple.
>
> * It does not have multiple characters mapping to a single
> collation element.
> * It does not consider composed and decomposed dependent vowels as
> equivalent [2].
> * It does not have to deal with secondary and tertiary weights.
> * It has a few simple tailoring rules [3] that need to be applied to the
> DUCET [4].
>
>
> Q1) Is it a requirement to use the collating-symbol keyword to define
> ALL symbols? If not, is this patch sufficient and acceptable for glibc?
> http://cvs.savannah.gnu.org/viewvc/sinhala/patches/iso14651_t1_common-glibc.patch?root=sinhala&view=log
>
> Q2) Instead of explicitly listing all the characters in order, is it
> possible to use the reorder-after keyword to only define variations to
> the DUCET?
>
> Q3) I couldn't find any documentation on:
>
> translit_start
> include  "translit_combining";""
> translit_end
>
> /usr/share/i18n/locales/translit_combining
> ------------------------------------------
> % SINHALA VOWEL SIGN DIGA KOMBUVA
> <U0DDA> "<U0DD9><U0DCA>"
> % SINHALA VOWEL SIGN KOMBUVA HAA AELA-PILLA
> <U0DDC> "<U0DD9><U0DCF>"
> % SINHALA VOWEL SIGN KOMBUVA HAA DIGA AELA-PILLA
> <U0DDD> "<U0DDC><U0DCA>"
> % SINHALA VOWEL SIGN KOMBUVA HAA GAYANUKITTA
> <U0DDE> "<U0DD9><U0DDF>"
> ------------------------------------------
>
> Does translit_start have an affect on LC_COLLATE?
>
> Thanks,
> #
>
> [1]
> http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap07.html
>
> [2]
> http://sourceforge.net/mailarchive/forum.php?thread_name=1223803982.4898.16.camel%40B1.HOME&forum_name=sinhala-technical
>
> [3]
> http://www.nongnu.org/sinhala/doc/howto/sinhala-howto.html#DEV-DATABASES
>
> [4] http://unicode.org/Public/UCA/latest/allkeys.txt
>
>


Adding Ulrich in cc list,

Thanks & Regards,
----------------------
Pravin Satpute

Re: QUESTION: LC_COLLATE minimal requirements?

[Ulrich Drepper]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Pravin S wrote:
>> Q1) Is it a requirement to use the collating-symbol keyword to define
>> ALL symbols? If not, is this patch sufficient and acceptable for glibc?
>> http://cvs.savannah.gnu.org/viewvc/sinhala/patches/iso14651_t1_common-glibc.patch?root=sinhala&view=log

It's better to follow the example of the other languages.  This results
in better tables.  And it's trivial.  Just use

   <U0DF4>   <U0DF4>;<BAS>;<MIN>;IGNORE

etc


>> Q2) Instead of explicitly listing all the characters in order, is it
>> possible to use the reorder-after keyword to only define variations to
>> the DUCET?

I have no idea what this means?  Each Unicode position can only appear
once in the entire file.  If you add a new language with new characters,
then just put them in the right order.  If you need to change the
collation for existing characters, then you must use reorder_after the
the locale description outside the collation tables themselves.


>> Q3) I couldn't find any documentation on:
>>
>> translit_start
>> include  "translit_combining";""
>> translit_end

Just look at the files.  There is no magic.  It's a 1:N mapping.

- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkj030AACgkQ2ijCOnn/RHSBKACeIdqAhld7yAS4UUKZOye3QzYy
NakAn3hwJtj9Ft8V3x9hz09f2fQIj0RQ
=uJHD
-----END PGP SIGNATURE-----

Re: QUESTION: LC_COLLATE minimal requirements?

[Harshula]

Hi Ulrich,

On Tue, 2008-10-14 at 11:04 -0700, Ulrich Drepper wrote:

> >> Q1) Is it a requirement to use the collating-symbol keyword to define
> >> ALL symbols? If not, is this patch sufficient and acceptable for glibc?
> >> http://cvs.savannah.gnu.org/viewvc/sinhala/patches/iso14651_t1_common-glibc.patch?root=sinhala&view=log
> 
> It's better to follow the example of the other languages.  This results
> in better tables.  And it's trivial.  Just use
> 
>    <U0DF4>   <U0DF4>;<BAS>;<MIN>;IGNORE
> 
> etc

Thanks, I've made the changes.

Patch:
http://cvs.savannah.gnu.org/viewvc/sinhala/patches/iso14651_t1_common-glibc.patch?root=sinhala&view=log

Testcase:
http://cvs.savannah.gnu.org/viewvc/sinhala/patches/mysql-data-sinhala.txt?root=sinhala&view=log

Correct output from 'sort':
http://cvs.savannah.gnu.org/viewvc/sinhala/patches/glibc-collation-correct.txt?root=sinhala&view=log


> >> Q3) I couldn't find any documentation on:
> >>
> >> translit_start
> >> include  "translit_combining";""
> >> translit_end
> 
> Just look at the files.  There is no magic.  It's a 1:N mapping.

The original question was "Does translit_start have an affect on
LC_COLLATE?"

translit_combining file contains this comment:
-----------------------------------------------
% Transliterations that remove all combining characters (accents,
% pronounciation marks, etc.).
% Generated from UnicodeData.txt.
-----------------------------------------------

Should this be interpreted as always converting to the composed form?

Thanks,
#

Re: QUESTION: LC_COLLATE minimal requirements?

[Ulrich Drepper]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Harshula wrote:
> The original question was "Does translit_start have an affect on
> LC_COLLATE?"

Transliteration has nothing to do with collation.  Why should it?

- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAkj7ckQACgkQ2ijCOnn/RHSxpQCfUWbu2ZlxCkYdnG4hVZTgKsOW
MtkAn07peGanR39l5/XCKQcGVGi++vX3
=7kG0
-----END PGP SIGNATURE-----

Re: QUESTION: LC_COLLATE minimal requirements?

[Pravin S]

Hi All,

2008/10/19 Harshula <harshula@gmail.com>:
> Hi Ulrich,
>
> On Tue, 2008-10-14 at 11:04 -0700, Ulrich Drepper wrote:
>
>> >> Q1) Is it a requirement to use the collating-symbol keyword to define
>> >> ALL symbols? If not, is this patch sufficient and acceptable for glibc?
>> >> http://cvs.savannah.gnu.org/viewvc/sinhala/patches/iso14651_t1_common-glibc.patch?root=sinhala&view=log
>>
>> It's better to follow the example of the other languages.  This results
>> in better tables.  And it's trivial.  Just use
>>
>>    <U0DF4>   <U0DF4>;<BAS>;<MIN>;IGNORE
>>
>> etc
>
> Thanks, I've made the changes.
>
> Patch:
> http://cvs.savannah.gnu.org/viewvc/sinhala/patches/iso14651_t1_common-glibc.patch?root=sinhala&view=log
>
> Testcase:
> http://cvs.savannah.gnu.org/viewvc/sinhala/patches/mysql-data-sinhala.txt?root=sinhala&view=log
>
> Correct output from 'sort':
> http://cvs.savannah.gnu.org/viewvc/sinhala/patches/glibc-collation-correct.txt?root=sinhala&view=log
>
>

Harshula I think,
It will be nice if you filed bug @
http://sourceware.org/bugzilla/enter_bug.cgi?product=glibc
and submit patch there, it will be good for tracking





Thanks & Regards,
----------------------
Pravin Satpute

Re: QUESTION: LC_COLLATE minimal requirements?

[Harshula]

On Mon, 2008-10-20 at 12:20 +0530, Pravin S wrote:
> 2008/10/19 Harshula <harshula@gmail.com>:

> > Thanks, I've made the changes.
> >
> > Patch:
> > http://cvs.savannah.gnu.org/viewvc/sinhala/patches/iso14651_t1_common-glibc.patch?root=sinhala&view=log
> >
> > Testcase:
> > http://cvs.savannah.gnu.org/viewvc/sinhala/patches/mysql-data-sinhala.txt?root=sinhala&view=log
> >
> > Correct output from 'sort':
> > http://cvs.savannah.gnu.org/viewvc/sinhala/patches/glibc-collation-correct.txt?root=sinhala&view=log
> >
> >
> 
> Harshula I think,
> It will be nice if you filed bug @
> http://sourceware.org/bugzilla/enter_bug.cgi?product=glibc
> and submit patch there, it will be good for tracking

Done:
http://sourceware.org/bugzilla/show_bug.cgi?id=6968

cya,
#

Re: QUESTION: LC_COLLATE minimal requirements?

[Pravin S]

2008/10/21 Harshula <harshula@gmail.com>:
> On Mon, 2008-10-20 at 12:20 +0530, Pravin S wrote:
>> 2008/10/19 Harshula <harshula@gmail.com>:
>
>> > Thanks, I've made the changes.
>> >
>> > Patch:
>> > http://cvs.savannah.gnu.org/viewvc/sinhala/patches/iso14651_t1_common-glibc.patch?root=sinhala&view=log
>> >
>> > Testcase:
>> > http://cvs.savannah.gnu.org/viewvc/sinhala/patches/mysql-data-sinhala.txt?root=sinhala&view=log
>> >
>> > Correct output from 'sort':
>> > http://cvs.savannah.gnu.org/viewvc/sinhala/patches/glibc-collation-correct.txt?root=sinhala&view=log
>> >
>> >
>>
>> Harshula I think,
>> It will be nice if you filed bug @
>> http://sourceware.org/bugzilla/enter_bug.cgi?product=glibc
>> and submit patch there, it will be good for tracking
>
> Done:
> http://sourceware.org/bugzilla/show_bug.cgi?id=6968
>

Thanks Harshula

Best Regards,
-----------------
Pravin Satpute

Re: QUESTION: LC_COLLATE minimal requirements?

[Keld Jørn Simonsen]

On Mon, Oct 13, 2008 at 02:52:09AM +1100, Harshula wrote:
> Hi,
> 
> I was unable to find much documentation on LC_COLLATE except for [1].
> Hence I have a few questions.

There is a more recent spec at
http://www.open-std.org/JTC1/sc22/WG20/docs/n972-14652ft.pdf
> 
> [1]
> http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap07.html

best regards
keld