Adding support for gb2312 and sift-jis to newlib iconv

Adding support for gb2312 and sift-jis to newlib iconv

[Anas N.]

[-- Attachment #1: Type: text/plain, Size: 704 bytes --]


Hi newlib team,
In the project I am working there is requirement to covert UCS2 to following code pages

1.       Chinese: gb2312
2.       Japanese: shift_ jis
3.       Russian : cp1251
4.       Hungrian, Polish and Cesky: cp1252
5.       Default:cp1250
I could see that items 3-5 are supported in newlib iconv library.

For gb2312  or gbk  I could see that it's supported in Cygwin-windows only. Is there a feasibility issue to support gb2312 for embedded cortex M controllers?

I can see Jis implementation in newlib and but is not available for Cygwin.Is It available for embedded   cortex M controllers?

What is the difference between jis and shit-jis

Thank & Regards,
Anas N

Re: Adding support for gb2312 and sift-jis to newlib iconv

[Corinna Vinschen]

On Dec 24 13:54, Anas N. via Newlib wrote:
> 
> Hi newlib team,
> In the project I am working there is requirement to covert UCS2 to following code pages
> 
> 1.       Chinese: gb2312
> 2.       Japanese: shift_ jis
> 3.       Russian : cp1251
> 4.       Hungrian, Polish and Cesky: cp1252
> 5.       Default:cp1250
> I could see that items 3-5 are supported in newlib iconv library.
> 
> For gb2312  or gbk  I could see that it's supported in Cygwin-windows
> only.

GB2312 is incompatible with the portable codeset, i.e. ASCII, and we
can't handle that in Cygwin.  The underlying code translates GB2312
into codepage 936 which defines GBK.  So GB2312 in Cygwin is actually GBK.

> Is there a feasibility issue to support gb2312 for embedded
> cortex M controllers?

Somebody would have to provide code...

> I can see Jis implementation in newlib and but is not available for
> Cygwin.Is It available for embedded   cortex M controllers?

The newlib code should support it.  Cygwin couldn't due to some
incompatibility problem.

> What is the difference between jis and shit-jis

https://de.wikipedia.org/wiki/Shift_JIS


Corinna

RE: Adding support for gb2312 and sift-jis to newlib iconv

[Anas N.]

[-- Attachment #1: Type: text/plain, Size: 2251 bytes --]

Hi Newlib team,


I have partially able to do for shift_jis by creating tables using
1. wget ftp://sourceware.org/pub/newlib/newlib-4.2.0.20211231.tar.gz
2. tar -xvf newlib-4.2.0.20211231.tar.gz
3. cd newlib/libc/iconv/ccs/

4. wget https://www.unicode.org/Public/MAPPINGS/OBSOLETE/EASTASIA/JIS/SHIFTJIS.TXT
5. ./mktbl.pl  -S SHIFTJIS.TXT
 6. cd ../ces/
 7. ./mkdeps.pl
It seems I need to handle more in the scenario when a string is mixed with English and Japanese


Regards,
Anas N
From: Anas N.
Sent: 24 December 2022 19:25
To: newlib@sourceware.org
Subject: Adding support for gb2312 and sift-jis to newlib iconv


Hi newlib team,
In the project I am working there is requirement to covert UCS2 to following code pages

1.       Chinese: gb2312
2.       Japanese: shift_ jis
3.       Russian : cp1251
4.       Hungrian, Polish and Cesky: cp1252
5.       Default:cp1250
I could see that items 3-5 are supported in newlib iconv library.

For gb2312  or gbk  I could see that it's supported in Cygwin-windows only. Is there a feasibility issue to support gb2312 for embedded cortex M controllers?

I can see Jis implementation in newlib and but is not available for Cygwin.Is It available for embedded   cortex M controllers?

What is the difference between jis and shit-jis

Thank & Regards,
Anas N

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast, a leader in email security and cyber resilience. Mimecast integrates email defenses with brand protection, security awareness training, web security, compliance and other essential capabilities. Mimecast helps protect large and small organizations from malicious activity, human error and technology failure; and to lead the movement toward building a more resilient world. To find out more, visit our website.