[Bug locale/28409] New: iconv corrupted output when input has convertable and non-convertable chars

[Bug locale/28409] New: iconv corrupted output when input has convertable and non-convertable chars

[soko246 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=28409

            Bug ID: 28409
           Summary: iconv corrupted output when input has convertable and
                    non-convertable chars
           Product: glibc
           Version: unspecified
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: locale
          Assignee: unassigned at sourceware dot org
          Reporter: soko246 at gmail dot com
  Target Milestone: ---

Using iconv results in corrupted output, when "-c" flag is used for input where
characters that *can* and *cannot* be converted appear together.
The issue only manifests for rather large inputs (presumably > 32K).

Steps to reproduce: run the following in bash:
>export LANG=C
>perl -E 'say "\x58\xe2\x58\xc3\x92\x58\xe2\x58\x58\xe2\x58\xc3\x92\x58\xe2\x58\n" x 15000' | iconv -c -f ISO-8859-3 -t UTF-8 | sort | uniq -c

Expected output:
>15000 XâX�XâXXâX�XâX

Actual output:
> 1
> 2 XXâX�XâX
> 2 XâX�XXâX
> 2 XâX�XâX
> 1 XâX�XâXX
> 2 XâX�XâXXâX�X�XâXXâX�XâX
> 14917 XâX�XâXXâX�XâX

As can be seen, many lines just disappear (14917+2+1+2+2+2+1 don't sum up to
15000). 

Actual specific input does not matter, as long as it has a mix of convertable
and non-convertable characters.
Reducing number of input lines to smaller number (ex. 1000) and all works as
expected:
>1000 XâX�XâXXâX�XâX

I tried this for ISO-8859-3 and ISO-8859-8 (same input) with similar (wrong)
results.

Using piconv (Perl variant of iconv) instead of iconv produces correct results.

-- 
You are receiving this mail because:
You are on the CC list for the bug.