[Bug libc/13757] New: mbstowcs(3) unable to handle 8bit characters.

[Bug libc/13757] New: mbstowcs(3) unable to handle 8bit characters.

[sdrake at xnet dot co.nz]

http://sourceware.org/bugzilla/show_bug.cgi?id=13757

             Bug #: 13757
           Summary: mbstowcs(3) unable to handle 8bit characters.
           Product: glibc
           Version: 2.13
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libc
        AssignedTo: unassigned@sourceware.org
        ReportedBy: sdrake@xnet.co.nz
                CC: drepper.fsp@gmail.com
    Classification: Unclassified


Created attachment 6246
  --> http://sourceware.org/bugzilla/attachment.cgi?id=6246
Simple mbstowcs test program

Compiling and running the attached program with glibc emmits:
ERROR: mbstowcs: Invalid or incomplete multibyte or wide character

Compiling and running the progam on a system with a different libc
implentaion it gives the expected output.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13757] mbstowcs(3) unable to handle 8bit characters.

[schwab@linux-m68k.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=13757

Andreas Schwab <schwab@linux-m68k.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #6246|application/octet-stream    |text/plain
          mime type|                            |

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13757] mbstowcs(3) unable to handle 8bit characters.

[schwab@linux-m68k.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=13757

--- Comment #1 from Andreas Schwab <schwab@linux-m68k.org> 2012-02-26 08:47:46 UTC ---
You need to use a locale that defines a meaning to this byte.  The default
(ASCII) locale doesn't.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13757] mbstowcs(3) unable to handle 8bit characters.

[sdrake at xnet dot co.nz]

http://sourceware.org/bugzilla/show_bug.cgi?id=13757

--- Comment #2 from Steven Drake <sdrake at xnet dot co.nz> 2012-02-26 09:11:06 UTC ---
(In reply to comment #1)
> You need to use a locale that defines a meaning to this byte.  The default
> (ASCII) locale doesn't.

$ env LANG=en_US.iso88591 ./test-mbstowcs
ERROR: mbstowcs: Invalid or incomplete multibyte or wide character

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13757] mbstowcs(3) unable to handle 8bit characters.

[schwab@linux-m68k.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=13757

--- Comment #3 from Andreas Schwab <schwab@linux-m68k.org> 2012-02-26 09:37:05 UTC ---
You don't use a non-default locale without calling setlocale.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13757] mbstowcs(3) unable to handle 8bit characters.

[sdrake at xnet dot co.nz]

http://sourceware.org/bugzilla/show_bug.cgi?id=13757

--- Comment #4 from Steven Drake <sdrake at xnet dot co.nz> 2012-02-29 05:54:36 UTC ---
(In reply to comment #1)
> You need to use a locale that defines a meaning to this byte.  The default
> (ASCII) locale doesn't.

Thats wrong, 'locale charmap' gives 'ANSI_X3.4-1968' and there lies the
problem, the charmap for the 'C' locale should probable be ISO-8896-1.

To be more accurate it should be the charmap that is used by system
calls (e.g. readdir and readlink).

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13757] mbstowcs(3) unable to handle 8bit characters.

[schwab@linux-m68k.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=13757

Andreas Schwab <schwab@linux-m68k.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |INVALID

--- Comment #5 from Andreas Schwab <schwab@linux-m68k.org> 2012-02-29 09:16:58 UTC ---
If you want something else than the C locale you must use setlocale.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13757] mbstowcs(3) unable to handle 8bit characters.

[sdrake at xnet dot co.nz]

http://sourceware.org/bugzilla/show_bug.cgi?id=13757

Steven Drake <sdrake at xnet dot co.nz> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|INVALID                     |

--- Comment #6 from Steven Drake <sdrake at xnet dot co.nz> 2012-03-02 02:29:16 UTC ---
Please ignore comment 2, the problem is not the locale in use but the charmap
of the C locale.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13757] mbstowcs(3) unable to handle 8bit characters.

[bugdal at aerifal dot cx]

http://sourceware.org/bugzilla/show_bug.cgi?id=13757

Rich Felker <bugdal at aerifal dot cx> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bugdal at aerifal dot cx

--- Comment #7 from Rich Felker <bugdal at aerifal dot cx> 2012-03-03 13:41:43 UTC ---
The charmap for the C locale should definitely not be ISO-8859-anything. All
that does is encourage broken, non-portable program behavior. If you are going
to use mbrtowc and family and intend to process characters not in the portable
character set, you MUST call setlocale for the LC_CTYPE category.

The system calls you referred to (e.g. readdir and readlink) do not use any
character map. They process bytes. In any case, if you wanted the C locale to
match the filesystem's encoding, it would have to be UTF-8, not ISO-8859-1, at
least on any modern system, and I'm pretty sure that's not what you want since
you seem to be advocating for very backwards behavior...

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13757] mbstowcs(3) unable to handle 8bit characters.

[drepper.fsp at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=13757

Ulrich Drepper <drepper.fsp at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |INVALID

--- Comment #8 from Ulrich Drepper <drepper.fsp at gmail dot com> 2012-03-07 08:44:55 UTC ---
The charmap for the C locale is ANSI.  Just use an appropriate locale as you
have been told several times already.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13757] mbstowcs(3) unable to handle 8bit characters.

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=13757

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|                            |security-

-- 
You are receiving this mail because:
You are on the CC list for the bug.