[Bug libc/10460] "iconv" corrupts all files over 17 KB from UTF8 to UTF16

[Bug libc/10460] "iconv" corrupts all files over 17 KB from UTF8 to UTF16

[krichter722 at aol dot de]

http://sourceware.org/bugzilla/show_bug.cgi?id=10460

Kalle Richter <krichter722 at aol dot de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
                 CC|                            |krichter722 at aol dot de
            Version|2.9                         |2.17
         Resolution|WORKSFORME                  |---

--- Comment #3 from Kalle Richter <krichter722 at aol dot de> ---
This behavior persists in iconv (Ubuntu EGLIBC 2.17-0ubuntu5) 2.17 if you
choose input file and output file to be the same (maybe only on an x86_64). Did
you one try to reproduce the error with a set of identical input and output
files? If I process the files in the following python script (using temporary
files) everything works fine:
<code>
#!/usr/bin/python

import os
import tempfile
import shutil

# The intention of this script is to avoid error "bus error" when processing
file iconv with identical input and output file

for (dirpath, dirnames, filenames) in
os.walk("/home/richter/sources/Aristoteles", topdown=True, onerror=None,
followlinks=False):
    #print(dirpath, dirnames, filenames)
    for filename in filenames:
        _file = os.path.join(dirpath, filename)
        #print(_file)
        file_ext_pair = _file.split(".")
        if len(file_ext_pair) > 1 and file_ext_pair[len(file_ext_pair)-1] ==
"java":
            _tempfile = tempfile.mkstemp()[1]
            os.system("iconv -f ISO-8859-15 -t utf-8 \""+_file+"\" -o
\""+_tempfile+"\"")
            os.remove(_file)
            shutil.move(_tempfile, _file)
</code>

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/10460] "iconv" corrupts all files over 17 KB from UTF8 to UTF16

[krichter722 at aol dot de]

http://sourceware.org/bugzilla/show_bug.cgi?id=10460

--- Comment #4 from Kalle Richter <krichter722 at aol dot de> ---
The idea behind is to add a warning if input and output file are identical that
the execution is discouraged.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/10460] "iconv" corrupts all files over 17 KB from UTF8 to UTF16

[bugdal at aerifal dot cx]

http://sourceware.org/bugzilla/show_bug.cgi?id=10460

Rich Felker <bugdal at aerifal dot cx> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bugdal at aerifal dot cx

--- Comment #5 from Rich Felker <bugdal at aerifal dot cx> ---
None of the standard utilities work, or can be expected to work, when the input
and output files are the same. As far as I know this is well documented in
POSIX. If it's not documented in the manual for the GNU iconv utility, this
should probably be reclassified as a manual bug, or a new bug filed against the
manual.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug manual/10460] "iconv" corrupts all files over 17 KB from UTF8 to UTF16

[krichter722 at aol dot de]

http://sourceware.org/bugzilla/show_bug.cgi?id=10460

Kalle Richter <krichter722 at aol dot de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mtk.manpages at gmail dot com,
                   |                            |roland at gnu dot org
          Component|libc                        |manual

--- Comment #6 from Kalle Richter <krichter722 at aol dot de> ---
One of a the information that input and output have to be different or a link
to the general POSIX information should be added. 
(As I have to choose one of duplicate, fixed, invalid, wontfix and worksforme
and moved, which all don't match with reopened, I choose duplicate without any
purpose)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug manual/10460] "iconv" corrupts all files over 17 KB from UTF8 to UTF16

[neleai at seznam dot cz]

https://sourceware.org/bugzilla/show_bug.cgi?id=10460

Ondrej Bilka <neleai at seznam dot cz> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |neleai at seznam dot cz
           Severity|normal                      |enhancement

--- Comment #7 from Ondrej Bilka <neleai at seznam dot cz> ---
This asks for new functionality, easiest way to resolve this is write a patch
yourself. 

Relevant part is  iconv/iconv_prog.c around line 300. One solution would be
that if input is same as output first load entire input file to memory and
proceed like in mmap case.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug manual/10460] "iconv" corrupts all files over 17 KB from UTF8 to UTF16

[bugdal at aerifal dot cx]

https://sourceware.org/bugzilla/show_bug.cgi?id=10460

--- Comment #8 from Rich Felker <bugdal at aerifal dot cx> ---
There's another open bug, #6050, which is to be fixed by removing the useless
and harmful attempt to read the whole input file into memory, so I don't think
adding more cases to do this would be a good idea. Using the same file for
input and output is just ALWAYS wrong for all the standard utilities; iconv is
no exception.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug manual/10460] "iconv" corrupts all files over 17 KB from UTF8 to UTF16

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=10460

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fweimer at redhat dot com
              Flags|                            |security-

--- Comment #9 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Rich Felker from comment #8)
> There's another open bug, #6050, which is to be fixed by removing the
> useless and harmful attempt to read the whole input file into memory, so I
> don't think adding more cases to do this would be a good idea. Using the
> same file for input and output is just ALWAYS wrong for all the standard
> utilities; iconv is no exception.

There are exceptions, sort being one of them.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/10460] "iconv" corrupts all files over 17 KB from UTF8 to UTF16

[drepper at redhat dot com]

------- Additional Comments From drepper at redhat dot com  2009-08-01 03:44 -------
I cannot reproduce any problem.  Neither on x86 nor x86-64.  Not with glibc 2.8,
2.10, or the current development version.  You didn't specify the architecture
so I don't know whether it's an alignment issue or just a broken binary on your
side.  Such vague problems should always be reported to the distribution maker.

If you have concrete information on how you can reproduce the problem with the
current sources reopen the bug.  Otherwise you're on your own.  Particularly if
you're not using the latest code.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |WORKSFORME


http://sourceware.org/bugzilla/show_bug.cgi?id=10460

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/10460] "iconv" corrupts all files over 17 KB from UTF8 to UTF16

[murielle dot descerisiers at gmail dot com]

------- Additional Comments From murielle dot descerisiers at gmail dot com  2009-07-29 18:18 -------
Created an attachment (id=4100)
 --> (http://sourceware.org/bugzilla/attachment.cgi?id=4100&action=view)
One of the files that "iconv" failed to convert.


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10460

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.