From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 19707 invoked by alias); 23 Apr 2014 08:41:32 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 19698 invoked by uid 89); 23 Apr 2014 08:41:32 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-3.6 required=5.0 tests=AWL,BAYES_50,UNSUBSCRIBE_BODY autolearn=no version=3.3.2 X-HELO: calimero.vinschen.de Received: from aquarius.hirmke.de (HELO calimero.vinschen.de) (217.91.18.234) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 23 Apr 2014 08:41:02 +0000 Received: by calimero.vinschen.de (Postfix, from userid 500) id 049118E142F; Wed, 23 Apr 2014 10:40:57 +0200 (CEST) Date: Wed, 23 Apr 2014 08:41:00 -0000 From: Corinna Vinschen To: cygwin@cygwin.com Subject: Re: fstat st_size on open files on Parallels filesystem is wrong Message-ID: <20140423084056.GJ2339@calimero.vinschen.de> Reply-To: cygwin@cygwin.com Mail-Followup-To: cygwin@cygwin.com References: <21333.25325.11106.958642@compute01.cs.columbia.edu> <227151856.20140421223417@yandex.ru> <21333.26515.393838.380071@compute01.cs.columbia.edu> <20140422081628.GC2339@calimero.vinschen.de> <21334.55207.784319.488271@compute01.cs.columbia.edu> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="eDB11BtaWSyaBkpc" Content-Disposition: inline In-Reply-To: <21334.55207.784319.488271@compute01.cs.columbia.edu> User-Agent: Mutt/1.5.21 (2010-09-15) X-SW-Source: 2014-04/txt/msg00507.txt.bz2 --eDB11BtaWSyaBkpc Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-length: 3843 On Apr 22 16:57, lennox@cs.columbia.edu wrote: > On Tuesday, April 22 2014, "Corinna Vinschen" wrote to "cygwin at cygwin.= com" saying: > > On Apr 21 14:46, lennox at cs.columbia.edu wrote: > > > [...] > > > This is using Posix APIs -- open() / write() -- not C APIs, fopen() / > > > fwrite(), so there shouldn't be a buffer? Notice that the test behav= es as I > > > expect for a file on NTFS. > > >=20 > > > Adding a call to fsync() prior to the fstat() call doesn't change any= thing. > >=20 > > This is actually a bad sign. The problem you're describing occurs on > > NFS, too. If you write to the file, a subsequent call to fetch stat > > attributes does not return the actual size of the file, but the size at > > the time the handle has been opened. > >=20 > > However, on NFS, a call to FlushFileBuffers helps to kick stat back into > > shape. That's the Win32 function called from fsync as well. What is > > Cygwin supposed to do if that doesn't work? >=20 > Okay, interesting further investigation. >=20 > The Parallels filesystem appears to work correctly when I repeat the test > case using Windows kernel32 APIs -- specifically, FileInformationByHandle= -- > so something's different between the kernel32 APIs and the ntdll APIs that > Cygwin uses. >=20 > Sample code for Win32 test attached. Works identically with Cygwin, MinG= W, > or Visual C++. >=20 > Just spitballing here, but -- I see that cygwin's file_get_fnoi function > (which is where fhandler_base::fstat_by_handle gets its size parameter) > tries NtQueryInformationFile(FileNetworkOpenInformation) before > NtQueryInformationFile(FileStandardInformation). Is it possible that the > Parallels filesystem isn't filling out all fields of that API properly? = Is > there a straightforward way I could debug this? You could use the same functions. Since you had a look into the sources anyway, you probably saw the comments: /* Some FSes (Netapps) don't implement FileNetworkOpenInformation. */ [...call NtQueryInformationFile(FileNetworkOpenInformation) unless the skip_network_open_inf flag is given...] if (status =3D=3D STATUS_INVALID_PARAMETER) /* Apart from accessing Netapps, this also occurs when accessing SMB share root dirs hosted on NT4. */ [...call NtQueryInformationFile(FileBasicInformation) and NtQueryInformationFile(FileStandardInformation)...] Rather than calling GetFileInformationByHandle, try this #include [...] NTSTATUS status; IO_STATUS_BLOCK io; FILE_NETWORK_OPEN_INFORMATION fnoi; =20=20 status =3D NtQueryInformationFile (file, &io, &fnoi, sizeof fnoi, FileNetworkOpenInformation); if (!NT_SUCCESS (status)) { fprintf (stderr, "NtQueryInformationFile: 0x%08x\n", status); [...] } printf("%s: NtQueryInformationFile: nFileSize=3D%" PRIu64 "\n", argv[i], fnoi.EndOfFile.QuadPart); Maybe that's really the problem. If so, either the EndOfFile info is wrong, or (more likely) the call to NtQueryInformationFile returns with a non-0 status code, which would be interesting to know. It's probably not STATUS_INVALID_PARAMETER, otherwise you probably wouldn't have seen a problem at all. Replacing the above call with FILE_STANDARD_INFORMATION fsi; status =3D NtQueryInformationFile (file, &io, &fsi, sizeof fsi, FileStandardInformation); [...] printf("%s: NtQueryInformationFile: nFileSize=3D%" PRIu64 "\n", argv[i], fsi.EndOfFile.QuadPart); should then give the correct result. Also, to add handling for the Parallels filesystem to Cygwin, I'd need the info printed by the getVolInfo tool from the csih package: $ /usr/lib/csih/getVolInfo /cygdrive/z Corinna --=20 Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat --eDB11BtaWSyaBkpc Content-Type: application/pgp-signature Content-length: 819 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJTV3yYAAoJEPU2Bp2uRE+gqKcP/1ArA4fpFeZoNOWraRgLNxlu V20ltT5Ls+ErC4MIf5hQrxABOc4SUODmsLXgkmOdcbRXgn/OR/RMWbr3m1XDKEyL hTUG7mf4ko4T+BuLHOW9iwgnkVCKw11XFozjLO+egN2AFck6hHrM16jtsWBmYS2L jPB3idmoHBz00oIYGUt0kT28miZ1BcaO+MSusdzm7/KXbT5guKTsBpSfjERJx9E6 Sm0LxQMkmKzvvXaP2oOqyxaxIzhwu3wp+kki6F2JRv1P21cVt1K8/wgC1/79Ayxh hIV3/ZBwvHyVJE4f5KBueKp9xfLYy6HE6rzOJjsaYCbwunXH/6eDCnM8od/EZQ+P hoIPeNrqYnTK3TTpUyPjE9Z63J/ihIiJR1XH4PiUbd/UCVgz7Rll65aj1C9d+jdC vW3HzdVgD4ddCSvczBd0VZzAJPFv25KmUfDH4VStjY/DsrCvJOUhQywK4ec42QpU +PNPH6yMGfX7oAHG+1lgq46T/LUjVo6SCBKqWH45Ja7mU2rRFwhqEJ5lcH7yiq5p cHcbgsOIJ+wax2qnYkDwVEXIfZDf75iHC9rzMvMMsO5l4udpK7FpdqaKu2p3twYs 9N6+EyRnIfnQZXVFO9nA5Pb7Yiy1pMZrXLQMebnTWTaPeweOh/yjdDk5G0Udbt7Q nlgERudKE1SmI4RhWCfb =VrZ/ -----END PGP SIGNATURE----- --eDB11BtaWSyaBkpc--