[Bug localedata/23048] iconv: add more macintosh tables and aliases

[Bug localedata/23048] iconv: add more macintosh tables and aliases

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=23048

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fweimer at redhat dot com,
                   |                            |libc-locales at sourceware dot org
          Component|libc                        |localedata
              Flags|                            |security-

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/23048] iconv: add more macintosh tables and aliases

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=23048

--- Comment #3 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Jan Engelhardt from comment #2)
> Something like this?
> https://en.wikipedia.org/wiki/Category:Mac_OS_character_encodings

Wikipedia is only a tertiary source (derived from secondary sources).  We need
the original source.  Unicode has tables for some Mac codepages, but I don't
know if they reflect reality.  Apple doesn't seem to publish anything.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/23048] iconv: add more macintosh tables and aliases

[jengelh at inai dot de]

https://sourceware.org/bugzilla/show_bug.cgi?id=23048

--- Comment #4 from Jan Engelhardt <jengelh at inai dot de> ---
>original source

Like, http://downloads.sf.net/cdrtools/cdrtools-3.02a09.tar.bz2 contains
cdrtools-3.02a09/libsiconv/tables/cp* ?

If there's no primary source but all secondary and tertiary sources agree with
one another, isn't that reason enough to pick the secondary source? Essentially
all ISOs were generated that way..

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/23048] iconv: add more macintosh tables and aliases

[jep at webfactory dot de]

https://sourceware.org/bugzilla/show_bug.cgi?id=23048

Jascha Eliano Paetzold <jep at webfactory dot de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jep at webfactory dot de

--- Comment #5 from Jascha Eliano Paetzold <jep at webfactory dot de> ---
https://www.gnu.org/software/libiconv supports all these encodings according to
its website. Maybe the implementation over there could be used as a reference?

Apart from that, Wikipedia lists the following secondary/primary sources:

mac-roman:
https://developer.apple.com/library/archive/documentation/mac/pdf/Text.pdf#page=89
mac-greek: https://www.unicode.org/Public/MAPPINGS/VENDORS/APPLE/GREEK.TXT
mac-centraleurope:
https://www.unicode.org/Public/MAPPINGS/VENDORS/APPLE/CENTEURO.TXT
mac-iceland: https://www.unicode.org/Public/MAPPINGS/VENDORS/APPLE/ICELAND.TXT
mac-turkish: https://www.unicode.org/Public/MAPPINGS/VENDORS/APPLE/TURKISH.TXT

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/23048] iconv: add more macintosh tables and aliases

[jep at webfactory dot de]

https://sourceware.org/bugzilla/show_bug.cgi?id=23048

--- Comment #6 from Jascha Eliano Paetzold <jep at webfactory dot de> ---
(In reply to Florian Weimer from comment #3)

Would you trust me if I verified these tables work on an Apple machine?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/23048] iconv: add more macintosh tables and aliases

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=23048

--- Comment #7 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Jascha Eliano Paetzold from comment #6)
> (In reply to Florian Weimer from comment #3)
> 
> Would you trust me if I verified these tables work on an Apple machine?

I think we could use iconv (the function) on macOS to enumerate the 8-bit bit
range, and then double-check the 20.1-bit Unicode space for any unexpected
mappings in the other direction. That would be enough verification for me.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/23048] iconv: add more macintosh tables and aliases

[jep at webfactory dot de]

https://sourceware.org/bugzilla/show_bug.cgi?id=23048

--- Comment #8 from Jascha Eliano Paetzold <jep at webfactory dot de> ---
(In reply to Florian Weimer from comment #7)
> (In reply to Jascha Eliano Paetzold from comment #6)
> > (In reply to Florian Weimer from comment #3)
> > 
> > Would you trust me if I verified these tables work on an Apple machine?
> 
> I think we could use iconv (the function) on macOS to enumerate the 8-bit
> bit range, and then double-check the 20.1-bit Unicode space for any
> unexpected mappings in the other direction. That would be enough
> verification for me.

Thanks for your offer to collaborate on that topic!

I have little to no experience with encodings, but I assume that it would be
sufficient for me to generate a text file spanning all possible characters in
the 8 bit range and pass it to the iconv command in the MacOS shell (setting it
to convert from unicode to mac-greek etc.) and then post the output here?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/23048] iconv: add more macintosh tables and aliases

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=23048

--- Comment #9 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Jascha Eliano Paetzold from comment #8)
> I have little to no experience with encodings, but I assume that it would be
> sufficient for me to generate a text file spanning all possible characters
> in the 8 bit range and pass it to the iconv command in the MacOS shell
> (setting it to convert from unicode to mac-greek etc.) and then post the
> output here?

For 8-bit input lie that, you'd have to convert *from* mac-greek.

For conversion to mac-greek, you'd have to enumerate all Unicode codepoints (or
maybe just the BMP), and somehow skip over unconvertable characters. That is
probably best done programmatically because iconv (the shell command) may not
provide enough information for skipped/non-convertable Unicode codepoints.

-- 
You are receiving this mail because:
You are on the CC list for the bug.