public inbox for glibc-bugs@sourceware.org help / color / mirror / Atom feed
From: "soko246 at gmail dot com" <sourceware-bugzilla@sourceware.org> To: glibc-bugs@sourceware.org Subject: [Bug locale/28409] New: iconv corrupted output when input has convertable and non-convertable chars Date: Fri, 01 Oct 2021 22:23:25 +0000 [thread overview] Message-ID: <bug-28409-131@http.sourceware.org/bugzilla/> (raw) https://sourceware.org/bugzilla/show_bug.cgi?id=28409 Bug ID: 28409 Summary: iconv corrupted output when input has convertable and non-convertable chars Product: glibc Version: unspecified Status: UNCONFIRMED Severity: normal Priority: P2 Component: locale Assignee: unassigned at sourceware dot org Reporter: soko246 at gmail dot com Target Milestone: --- Using iconv results in corrupted output, when "-c" flag is used for input where characters that *can* and *cannot* be converted appear together. The issue only manifests for rather large inputs (presumably > 32K). Steps to reproduce: run the following in bash: >export LANG=C >perl -E 'say "\x58\xe2\x58\xc3\x92\x58\xe2\x58\x58\xe2\x58\xc3\x92\x58\xe2\x58\n" x 15000' | iconv -c -f ISO-8859-3 -t UTF-8 | sort | uniq -c Expected output: >15000 XâX�XâXXâX�XâX Actual output: > 1 > 2 XXâX�XâX > 2 XâX�XXâX > 2 XâX�XâX > 1 XâX�XâXX > 2 XâX�XâXXâX�X�XâXXâX�XâX > 14917 XâX�XâXXâX�XâX As can be seen, many lines just disappear (14917+2+1+2+2+2+1 don't sum up to 15000). Actual specific input does not matter, as long as it has a mix of convertable and non-convertable characters. Reducing number of input lines to smaller number (ex. 1000) and all works as expected: >1000 XâX�XâXXâX�XâX I tried this for ISO-8859-3 and ISO-8859-8 (same input) with similar (wrong) results. Using piconv (Perl variant of iconv) instead of iconv produces correct results. -- You are receiving this mail because: You are on the CC list for the bug.
reply other threads:[~2021-10-01 22:23 UTC|newest] Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=bug-28409-131@http.sourceware.org/bugzilla/ \ --to=sourceware-bugzilla@sourceware.org \ --cc=glibc-bugs@sourceware.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).