From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 2155) id 60C5C38582A3; Wed, 6 Mar 2024 13:28:16 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 60C5C38582A3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com; s=default; t=1709731696; bh=R4a0EC1RTT1puEvZ0QVebCy4a2A0t3PEDfV6efi/d1U=; h=Date:From:To:Subject:Reply-To:References:In-Reply-To:From; b=vIN2Qm4mhtHnZJLJFaMcHQ5yTclb8ZES7I9LiUeCyQFDS3M7LES9qDI7Q7kqX2MoM a8ExfmO2GHKyYLA4rfZb0hzR5jlAG1/uhzCDJ4WgQisWEz3H2525AkLKPn6UpM47sU OZL7S9NgqZQQvAgqbg17z+yuoD1p/v7pZGlRl0Yo= Received: by calimero.vinschen.de (Postfix, from userid 500) id 6BCEAA80DA3; Wed, 6 Mar 2024 14:28:14 +0100 (CET) Date: Wed, 6 Mar 2024 14:28:14 +0100 From: Corinna Vinschen To: cygwin@cygwin.com Subject: Re: ls/stat on OneDrive causes download of files Message-ID: Reply-To: cygwin@cygwin.com Mail-Followup-To: cygwin@cygwin.com References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: List-Id: On Mar 6 14:22, Corinna Vinschen via Cygwin wrote: > On Mar 5 19:54, Marcin Wisnicki via Cygwin wrote: > > If I invoke ls or anything else that does stat inside OneDrive folder > > it will trigger download of all files. > > > > OneDrive uses placeholder files[1] to represent remote files. > > > > I'm guessing reading file content in stat is to support detection of > > actually executable files as in here[2]? > > > > I think this should be disabled on non-hydrated placeholder files. > > Running `find` or 'ls -R` and having your entire OneDrive downloaded > > is extremely problematic. > > > > I could live without executable scripts in the OneDrive folder and > > it's easy to mark files as always offline to solve it. > > > > Another idea is to skip checking files with extensions known to be > > non-executable such as jpg (or just any extensions that is not known > > to be executable). > > Nothing of this makes sense from a POSIX library POV. The library can > either not handle placeholder files specially, as today, or it can > handle them all the same way. > > Given these placeholder files are actually reparse points of type > IO_REPARSE_TAG_FILE_PLACEHOLDER, we can handle them as symbolic links. > > However, the structure of the IO_REPARSE_TAG_FILE_PLACEHOLDER reparse > data buffer is undocumented. It would be helpful if somebody using > OneDrive would examine the content of the attached REPARSE_DATA_BUFFER. > > > [2] https://github.com/msys2/msys2-runtime/blob/msys2-3.4.10/winsup/cygwin/fhandler/disk_file.cc#L548 > > The NtReadFile call at this point is not the problem. It would be > helpful to point to Cygwin's source instead of MSYS2, btw. Oh, btw., this is from https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/c8e77b37-3909-4fe6-a4ea-2b9d423b1ee4: IO_REPARSE_TAG_FILE_PLACEHOLDER 0x80000015 Obsolete. --------- Used by Windows Shell for legacy placeholder files in Windows 8.1. Server-side interpretation only, not meaningful over the wire. So even if we support them, what is their replacement in W10 and later? Corinna