From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 18640 invoked by alias); 4 Nov 2011 08:47:09 -0000 Received: (qmail 18588 invoked by uid 22791); 4 Nov 2011 08:46:46 -0000 X-Spam-Check-By: sourceware.org Received: from aquarius.hirmke.de (HELO calimero.vinschen.de) (217.91.18.234) by sourceware.org (qpsmtpd/0.83/v0.83-20-g38e4449) with ESMTP; Fri, 04 Nov 2011 08:46:22 +0000 Received: by calimero.vinschen.de (Postfix, from userid 500) id E326F2C0485; Fri, 4 Nov 2011 09:46:19 +0100 (CET) Date: Fri, 04 Nov 2011 08:47:00 -0000 From: Corinna Vinschen To: cygwin@cygwin.com Subject: Re: Possible Bug (clarification) in Cygwin 1.7.5 -- findfirstfile (and findnextfile) yeild bad cfilename when file names have special characters. Works in cygwin 1.5, fails in 1.7 Message-ID: <20111104084619.GM9159@calimero.vinschen.de> Reply-To: cygwin@cygwin.com Mail-Followup-To: cygwin@cygwin.com References: <135801cc9a69$f73ceaf0$e5b6c0d0$@vaultnow.com> <4EB30DF9.2080006@cwilson.fastmail.fm> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4EB30DF9.2080006@cwilson.fastmail.fm> User-Agent: Mutt/1.5.21 (2010-09-15) Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com X-SW-Source: 2011-11/txt/msg00057.txt.bz2 On Nov 3 17:56, Charles Wilson wrote: > On 11/3/2011 4:48 PM, Leon Vanderploeg wrote: > > With cygwin 1.7.5, cFileName with a special characters such as ñ (n > > with tidle above it) fail be properly extracted from a > > WIN32_FIND_DATA structure with findFirstFile (or findNextFile). > > > > To set up a simple test scenario, I created a file in C:\Testing > > named Mañana.docx. I compiled the code at the end of this message > > on Cygwin 1.7.9 with GCC version 3.4.4 on Server 2008 32 bit system. > > On this system (and on a Windows 7 32 bit machine), it returns: > > a) Why are you using native Win32 APIs in a cygwin program? You should > be using the POSIX interfaces instead -- see /usr/include/dirent.h. > > DIR *opendir (const char *); > DIR *fdopendir (int); > struct dirent *readdir (DIR *); > int readdir_r (DIR *, struct dirent *, struct dirent **); > void rewinddir (DIR *); > int closedir (DIR *); ACK++ > b) What you observe is an artifact of cygwin-1.7's new *support* for > i18n. In cygwin-1.5, it just didn't care and passed all the bytes back > exactly as found without transliteration. In 1.7, it (correctly) > transcodes strings into the current locale -- and your current locale > does not appear to support ñ -- or, at least, you haven't told cygwin to > use the correct one. > > (I'm probably thoroughly botching this explanation, but the point is, Just a bit. What you have to keep in mind is that Windows stores all object names, including filenames, as UTF-16 strings, UNICODE in Windows terminology. When you use the ANSI Win32 API as in this example, then the UTF-16 names are converted to the currently defined ANSI charset on output, for instance codepage 1252 for Western Europe languages. Cygwin 1.5 either used the ANSI API, or it converted strings from UTF-16 to the current Windows ANSI charset or vice versa. Cygwin 1.7 doesn't use the ANSI API anymore, rather it uses UNICODE to talk to Windows only, and the multibyte charset is defined through the environment(*) as defined in POSIX. UTF-8 is the default now. > you need to check your LC_* and LANG env vars, and maybe call > setlocale(LC_ALL, "") in your application.) And even than the code won't work. If you don't define UNICODE, FindFirstFile/FindNextFile will use the ANSI versions of this API, FindFirstFileA/FindNextFileA. If you didn't set your LANG/LC_CTYPE/LC_ALL variables to use your current Windows ANSI charset *and* called setlocale, Cygwin will use UTF-8 by default. Therefore, the character ñ will have another multibyte encoding, 0xc3 0xb1, rather than, say, 0xf1 in Windows codepage 1252. To avoid this problem, you can use the UNICODE API FindFirstFileW/ FindNextFileW and convert the filename the current multibyte charset via wcstombs and friends. However, as Chuck has pointed out, the obviously right thing to do is to use the POSIX API opendir/readdir/closedir instead. Corinna (*) http://cygwin.com/cygwin-ug-net/setup-locale.html http://cygwin.com/cygwin-ug-net/using-utils.html#locale -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple