public inbox for glibc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug libc/13518] New: iconv truncates input with //IGNORE
@ 2011-12-18 22:34 ezyang at mit dot edu
2011-12-18 22:56 ` [Bug libc/13518] " ezyang at mit dot edu
` (4 more replies)
0 siblings, 5 replies; 6+ messages in thread
From: ezyang at mit dot edu @ 2011-12-18 22:34 UTC (permalink / raw)
To: glibc-bugs
http://sourceware.org/bugzilla/show_bug.cgi?id=13518
Bug #: 13518
Summary: iconv truncates input with //IGNORE
Product: glibc
Version: 2.14
Status: NEW
Severity: normal
Priority: P2
Component: libc
AssignedTo: drepper.fsp@gmail.com
ReportedBy: ezyang@mit.edu
Classification: Unclassified
iconv seems to truncate inputs at around 8157 bytes if they contain invalid
characters for the target set, even if IGNORE is specified.
Steps to reproduce:
1. Download iconv.html
ezyang@javelin:~$ wget http://www.oppcharts.com/iconv.html
2. Attempt to convert UTF-8 to iso-8859-1//IGNORE
Expected behavior (from libiconv-1.14):
ezyang@javelin:~/Dev/glibc/build$ ~/Desktop/libiconv-1.14/src/iconv_no_i18n -f
utf-8 -t iso-8859-1//IGNORE ~/iconv.html | wc -c
15312
Actual behavior (from latest Git glibc-2.14-567-ga4647e7):
ezyang@javelin:~/Dev/glibc/build$ ./testrun.sh iconv/iconv_prog -f utf-8 -t
iso-8859-1//IGNORE ~/iconv.html | wc -c
iconv/iconv_prog: illegal input sequence at position 8168
8157
--
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 6+ messages in thread* [Bug libc/13518] iconv truncates input with //IGNORE
2011-12-18 22:34 [Bug libc/13518] New: iconv truncates input with //IGNORE ezyang at mit dot edu
@ 2011-12-18 22:56 ` ezyang at mit dot edu
2011-12-22 23:42 ` drepper.fsp at gmail dot com
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: ezyang at mit dot edu @ 2011-12-18 22:56 UTC (permalink / raw)
To: glibc-bugs
http://sourceware.org/bugzilla/show_bug.cgi?id=13518
--- Comment #1 from Edward Z. Yang <ezyang at mit dot edu> 2011-12-18 22:55:50 UTC ---
Created attachment 6117
--> http://sourceware.org/bugzilla/attachment.cgi?id=6117
Alpha, followed by a lot of x's
Here's a better, more minimal test-case.
ezyang@javelin:~$ Dev/glibc/build/testrun.sh Dev/glibc/build/iconv/iconv_prog
-f utf-8 -t ascii//IGNORE < test.txt | wc -c
Dev/glibc/build/iconv/iconv_prog: illegal input sequence at position 8161
8159
ezyang@javelin:~$ Desktop/libiconv-1.14/src/iconv_no_i18n -f utf-8 -t
ascii//IGNORE < test.txt | wc -c
11059
--
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 6+ messages in thread* [Bug libc/13518] iconv truncates input with //IGNORE
2011-12-18 22:34 [Bug libc/13518] New: iconv truncates input with //IGNORE ezyang at mit dot edu
2011-12-18 22:56 ` [Bug libc/13518] " ezyang at mit dot edu
@ 2011-12-22 23:42 ` drepper.fsp at gmail dot com
2011-12-23 0:43 ` [Bug libc/13518] iconv program doesn't handle //IGNORE flag correctly ezyang at mit dot edu
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: drepper.fsp at gmail dot com @ 2011-12-22 23:42 UTC (permalink / raw)
To: glibc-bugs
http://sourceware.org/bugzilla/show_bug.cgi?id=13518
Ulrich Drepper <drepper.fsp at gmail dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |INVALID
--- Comment #2 from Ulrich Drepper <drepper.fsp at gmail dot com> 2011-12-22 23:42:30 UTC ---
The iconv program cannot be used with the magic //IGNORE suffix. You have to
use the -c parameter.
--
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 6+ messages in thread* [Bug libc/13518] iconv program doesn't handle //IGNORE flag correctly
2011-12-18 22:34 [Bug libc/13518] New: iconv truncates input with //IGNORE ezyang at mit dot edu
2011-12-18 22:56 ` [Bug libc/13518] " ezyang at mit dot edu
2011-12-22 23:42 ` drepper.fsp at gmail dot com
@ 2011-12-23 0:43 ` ezyang at mit dot edu
2011-12-23 1:02 ` ezyang at mit dot edu
2014-06-27 11:27 ` fweimer at redhat dot com
4 siblings, 0 replies; 6+ messages in thread
From: ezyang at mit dot edu @ 2011-12-23 0:43 UTC (permalink / raw)
To: glibc-bugs
http://sourceware.org/bugzilla/show_bug.cgi?id=13518
Edward Z. Yang <ezyang at mit dot edu> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |REOPENED
Resolution|INVALID |
Summary|iconv truncates input with |iconv program doesn't
|//IGNORE |handle //IGNORE flag
| |correctly
--- Comment #3 from Edward Z. Yang <ezyang at mit dot edu> 2011-12-23 00:43:16 UTC ---
I think there still is a bug here. If //IGNORE is not supported by iconv_prog,
the behavior between -t with IGNORE and -c should be the same. However, this is
not the case:
ezyang@javelin:~$ Dev/glibc/build/testrun.sh Dev/glibc/build/iconv/iconv_prog
-f utf-8 -t ascii//IGNORE < test.txt | wc -c
Dev/glibc/build/iconv/iconv_prog: illegal input sequence at position 8161
8159
ezyang@javelin:~$ Dev/glibc/build/testrun.sh Dev/glibc/build/iconv/iconv_prog
-f utf-8 -t ascii < test.txt | wc -c
Dev/glibc/build/iconv/iconv_prog: illegal input sequence at position 0
0
For reference, here is iconv running with an invalid extra flag:
ezyang@javelin:~$ Dev/glibc/build/testrun.sh Dev/glibc/build/iconv/iconv_prog
-f utf-8 -t ascii//FOOBAR < test.txt | wc -c
Dev/glibc/build/iconv/iconv_prog: illegal input sequence at position 0
0
--
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 6+ messages in thread* [Bug libc/13518] iconv program doesn't handle //IGNORE flag correctly
2011-12-18 22:34 [Bug libc/13518] New: iconv truncates input with //IGNORE ezyang at mit dot edu
` (2 preceding siblings ...)
2011-12-23 0:43 ` [Bug libc/13518] iconv program doesn't handle //IGNORE flag correctly ezyang at mit dot edu
@ 2011-12-23 1:02 ` ezyang at mit dot edu
2014-06-27 11:27 ` fweimer at redhat dot com
4 siblings, 0 replies; 6+ messages in thread
From: ezyang at mit dot edu @ 2011-12-23 1:02 UTC (permalink / raw)
To: glibc-bugs
http://sourceware.org/bugzilla/show_bug.cgi?id=13518
Edward Z. Yang <ezyang at mit dot edu> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|REOPENED |RESOLVED
Resolution| |INVALID
--- Comment #4 from Edward Z. Yang <ezyang at mit dot edu> 2011-12-23 01:02:05 UTC ---
OK, I think I understand the underlying issue better. I'll file a new bug.
--
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 6+ messages in thread* [Bug libc/13518] iconv program doesn't handle //IGNORE flag correctly
2011-12-18 22:34 [Bug libc/13518] New: iconv truncates input with //IGNORE ezyang at mit dot edu
` (3 preceding siblings ...)
2011-12-23 1:02 ` ezyang at mit dot edu
@ 2014-06-27 11:27 ` fweimer at redhat dot com
4 siblings, 0 replies; 6+ messages in thread
From: fweimer at redhat dot com @ 2014-06-27 11:27 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=13518
Florian Weimer <fweimer at redhat dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Flags| |security-
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-06-27 11:27 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-12-18 22:34 [Bug libc/13518] New: iconv truncates input with //IGNORE ezyang at mit dot edu
2011-12-18 22:56 ` [Bug libc/13518] " ezyang at mit dot edu
2011-12-22 23:42 ` drepper.fsp at gmail dot com
2011-12-23 0:43 ` [Bug libc/13518] iconv program doesn't handle //IGNORE flag correctly ezyang at mit dot edu
2011-12-23 1:02 ` ezyang at mit dot edu
2014-06-27 11:27 ` fweimer at redhat dot com
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).