public inbox for libc-locales@sourceware.org
 help / color / mirror / Atom feed
* iconv: illegal input sequence
@ 2021-03-10  9:09 Lokesh Janghel
  2021-03-10  9:20 ` Florian Weimer
  0 siblings, 1 reply; 8+ messages in thread
From: Lokesh Janghel @ 2021-03-10  9:09 UTC (permalink / raw)
  To: libc-alpha, libc-locales; +Cc: Umesh Kalappa

[-- Attachment #1: Type: text/plain, Size: 527 bytes --]

Hi,

I am working on the charset conversion from cp932 to utf8.
And got the following problem with gcc-8.3:
$ccppc -E test.cxx -finput-charset=cp932
cc1plus: error: conversion from cp932 to UTF-8 not supported by iconv

And I am facing error like "iconv: illegal input sequence at position 85"
When I try to convert with:
$iconv -f cp932 -t utf8 test.cxx

Can you help me with the issue if I am following the wrong step or
missing something?
Please let me know your thoughts on the problem.

Thanks and regards,
Lokesh Janghel

[-- Attachment #2: test.cxx --]
[-- Type: text/plain, Size: 150 bytes --]

#if 0
#include <vxWorks.h>
#include <stdio.h>
#endif

void test_sjis()
{
    printf("�����Shift-JIS�̃e�X�g�ł��B\n");
}

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: iconv: illegal input sequence
  2021-03-10  9:09 iconv: illegal input sequence Lokesh Janghel
@ 2021-03-10  9:20 ` Florian Weimer
  2021-03-30  5:22   ` Lokesh Janghel
  0 siblings, 1 reply; 8+ messages in thread
From: Florian Weimer @ 2021-03-10  9:20 UTC (permalink / raw)
  To: Lokesh Janghel via Libc-alpha; +Cc: libc-locales, Lokesh Janghel, Umesh Kalappa

* Lokesh Janghel via Libc-alpha:

> And I am facing error like "iconv: illegal input sequence at position 85"
> When I try to convert with:
> $iconv -f cp932 -t utf8 test.cxx

Apparently, this is your input file:

00000000: 2369 6620 300a 2369 6e63 6c75 6465 203c  #if 0.#include <
00000010: 7678 576f 726b 732e 683e 0a23 696e 636c  vxWorks.h>.#incl
00000020: 7564 6520 3c73 7464 696f 2e68 3e0a 2365  ude <stdio.h>.#e
00000030: 6e64 6966 0a0a 766f 6964 2074 6573 745f  ndif..void test_
00000040: 736a 6973 2829 0a7b 0a20 2020 2070 7269  sjis().{.    pri
00000050: 6e74 6628 22ef bfbd efbf bdef bfbd efbf  ntf("...........
00000060: bdef bfbd 5368 6966 742d 4a49 53ef bfbd  ....Shift-JIS...
00000070: cc83 65ef bfbd 58ef bfbd 67ef bfbd c582  ..e...X...g.....
00000080: efbf bdef bfbd 425c 6e22 293b 0a7d       ......B\n");.}

The byte sequence at offset 85 is 0xef 0xbf 0xbd.  That doesn't look
like CP932, but UTF-8.  It looks like your input file already underwent
UTF-8 conversion at some point.

Thanks,
Florian


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: iconv: illegal input sequence
  2021-03-10  9:20 ` Florian Weimer
@ 2021-03-30  5:22   ` Lokesh Janghel
  2021-03-30  5:46     ` Florian Weimer
  2021-03-30  8:22     ` Andreas Schwab
  0 siblings, 2 replies; 8+ messages in thread
From: Lokesh Janghel @ 2021-03-30  5:22 UTC (permalink / raw)
  To: Florian Weimer; +Cc: Lokesh Janghel via Libc-alpha, libc-locales

Hi,

As my file is the Shift_JIS format:
...
$ nkf --guess test_sjis.cxx
Shift_JIS (LF)
...

And got the following problem with gcc-8.3:
$./cc1plus -E test_sjis.cxx -finput-charset=cp932
cc1plus: error: conversion from cp932 to UTF-8 not supported by iconv

Can you share your knowledge if I am following the wrong conversion?
Please let me know your thoughts on the problem.

Thanks,
Lokesh

On Wed, Mar 10, 2021 at 2:50 PM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Lokesh Janghel via Libc-alpha:
>
> > And I am facing error like "iconv: illegal input sequence at position 85"
> > When I try to convert with:
> > $iconv -f cp932 -t utf8 test.cxx
>
> Apparently, this is your input file:
>
> 00000000: 2369 6620 300a 2369 6e63 6c75 6465 203c  #if 0.#include <
> 00000010: 7678 576f 726b 732e 683e 0a23 696e 636c  vxWorks.h>.#incl
> 00000020: 7564 6520 3c73 7464 696f 2e68 3e0a 2365  ude <stdio.h>.#e
> 00000030: 6e64 6966 0a0a 766f 6964 2074 6573 745f  ndif..void test_
> 00000040: 736a 6973 2829 0a7b 0a20 2020 2070 7269  sjis().{.    pri
> 00000050: 6e74 6628 22ef bfbd efbf bdef bfbd efbf  ntf("...........
> 00000060: bdef bfbd 5368 6966 742d 4a49 53ef bfbd  ....Shift-JIS...
> 00000070: cc83 65ef bfbd 58ef bfbd 67ef bfbd c582  ..e...X...g.....
> 00000080: efbf bdef bfbd 425c 6e22 293b 0a7d       ......B\n");.}
>
> The byte sequence at offset 85 is 0xef 0xbf 0xbd.  That doesn't look
> like CP932, but UTF-8.  It looks like your input file already underwent
> UTF-8 conversion at some point.
>
> Thanks,
> Florian
>

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: iconv: illegal input sequence
  2021-03-30  5:22   ` Lokesh Janghel
@ 2021-03-30  5:46     ` Florian Weimer
  2021-03-30  6:30       ` Lokesh Janghel
  2021-03-30  8:22     ` Andreas Schwab
  1 sibling, 1 reply; 8+ messages in thread
From: Florian Weimer @ 2021-03-30  5:46 UTC (permalink / raw)
  To: Lokesh Janghel via Libc-alpha; +Cc: Lokesh Janghel, libc-locales

* Lokesh Janghel via Libc-alpha:

> Hi,
>
> As my file is the Shift_JIS format:
> ...
> $ nkf --guess test_sjis.cxx
> Shift_JIS (LF)
> ...

Please post a hex dump of the input file.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: iconv: illegal input sequence
  2021-03-30  5:46     ` Florian Weimer
@ 2021-03-30  6:30       ` Lokesh Janghel
  0 siblings, 0 replies; 8+ messages in thread
From: Lokesh Janghel @ 2021-03-30  6:30 UTC (permalink / raw)
  To: Florian Weimer; +Cc: Lokesh Janghel via Libc-alpha, libc-locales

> Please post a hex dump of the input file.

$ hd test_sjis.cxx
00000000  0a 23 69 66 20 30 0a 23  69 6e 63 6c 75 64 65 20  |.#if
0.#include |
00000010  3c 76 78 57 6f 72 6b 73  2e 68 3e 0a 23 65 6e 64
 |<vxWorks.h>.#end|
00000020  69 66 0a 23 69 6e 63 6c  75 64 65 20 3c 73 74 64  |if.#include
<std|
00000030  69 6f 2e 68 3e 0a 0a 0a  76 6f 69 64 20 74 65 73  |io.h>...void
tes|
00000040  74 5f 73 6a 69 73 28 29  0a 7b 0a 20 20 20 20 70  |t_sjis().{.
 p|
00000050  72 69 6e 74 66 28 22 82  b1 82 ea 82 cd 53 68 69
 |rintf("......Shi|
00000060  66 74 2d 4a 49 53 82 cc  83 65 83 58 83 67 82 c5
 |ft-JIS...e.X.g..|
00000070  82 b7 81 42 5c 6e 22 29  3b 0a 7d 0a              |...B\n");.}.|
0000007c


On Tue, Mar 30, 2021 at 11:16 AM Florian Weimer <fw@deneb.enyo.de> wrote:

> * Lokesh Janghel via Libc-alpha:
>
> > Hi,
> >
> > As my file is the Shift_JIS format:
> > ...
> > $ nkf --guess test_sjis.cxx
> > Shift_JIS (LF)
> > ...
>
> Please post a hex dump of the input file.
>

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: iconv: illegal input sequence
  2021-03-30  5:22   ` Lokesh Janghel
  2021-03-30  5:46     ` Florian Weimer
@ 2021-03-30  8:22     ` Andreas Schwab
  2021-03-31 13:02       ` Lokesh Janghel
  1 sibling, 1 reply; 8+ messages in thread
From: Andreas Schwab @ 2021-03-30  8:22 UTC (permalink / raw)
  To: Lokesh Janghel via Libc-alpha
  Cc: Florian Weimer, Lokesh Janghel, libc-locales

On Mär 30 2021, Lokesh Janghel via Libc-alpha wrote:

> cc1plus: error: conversion from cp932 to UTF-8 not supported by iconv

Apparently your iconv installation is incomplete.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: iconv: illegal input sequence
  2021-03-30  8:22     ` Andreas Schwab
@ 2021-03-31 13:02       ` Lokesh Janghel
  2021-03-31 13:12         ` Andreas Schwab
  0 siblings, 1 reply; 8+ messages in thread
From: Lokesh Janghel @ 2021-03-31 13:02 UTC (permalink / raw)
  To: Andreas Schwab
  Cc: Lokesh Janghel via Libc-alpha, Florian Weimer, libc-locales

On Mär 30 2021, Andreas Schwab via Libc-alpha wrote:

>> cc1plus: error: conversion from cp932 to UTF-8 not supported by iconv

>Apparently your iconv installation is incomplete.

iconv installation is working fine in the system.


Thanks,
Lokesh


On Tue, Mar 30, 2021 at 1:52 PM Andreas Schwab <schwab@linux-m68k.org>
wrote:

> On Mär 30 2021, Lokesh Janghel via Libc-alpha wrote:
>
> > cc1plus: error: conversion from cp932 to UTF-8 not supported by iconv
>
> Apparently your iconv installation is incomplete.
>
> Andreas.
>
> --
> Andreas Schwab, schwab@linux-m68k.org
> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
> "And now for something completely different."
>

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: iconv: illegal input sequence
  2021-03-31 13:02       ` Lokesh Janghel
@ 2021-03-31 13:12         ` Andreas Schwab
  0 siblings, 0 replies; 8+ messages in thread
From: Andreas Schwab @ 2021-03-31 13:12 UTC (permalink / raw)
  To: Lokesh Janghel
  Cc: Lokesh Janghel via Libc-alpha, Florian Weimer, libc-locales

On Mär 31 2021, Lokesh Janghel wrote:

> On Mär 30 2021, Andreas Schwab via Libc-alpha wrote:
>
>>> cc1plus: error: conversion from cp932 to UTF-8 not supported by iconv
>
>>Apparently your iconv installation is incomplete.
>
> iconv installation is working fine in the system.

Does gcc use the same iconv library?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2021-03-31 13:12 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-10  9:09 iconv: illegal input sequence Lokesh Janghel
2021-03-10  9:20 ` Florian Weimer
2021-03-30  5:22   ` Lokesh Janghel
2021-03-30  5:46     ` Florian Weimer
2021-03-30  6:30       ` Lokesh Janghel
2021-03-30  8:22     ` Andreas Schwab
2021-03-31 13:02       ` Lokesh Janghel
2021-03-31 13:12         ` Andreas Schwab

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).