From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by sourceware.org (Postfix) with ESMTPS id E80653858034 for ; Wed, 6 Mar 2024 13:54:41 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org E80653858034 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=SystematicSW.ab.ca Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=SystematicSW.ab.ca ARC-Filter: OpenARC Filter v1.0.0 sourceware.org E80653858034 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=216.40.44.13 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709733284; cv=none; b=rGQS53+c90HAOl8irTotVNJ7WOz0mLyUM5JODCcdk7tUDAxpgingYUkDf3rGpnkIG10o2q5DIJxHZSTKtz31oUlRpHKIuQTIIhkMd6PO7X6emBbk3+ZGlOvDhUSuAgqq9UDbO1UswCHcgR0tVrEACwJnACme5QGPYuocmdmSYB0= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709733284; c=relaxed/simple; bh=8rP1dTasS36hG+V8YxbKr1t9MUev6d90NedD50ga3Hs=; h=Message-ID:Date:MIME-Version:Subject:To:From; b=ehFdCpyVVAGemHPtwAfNquEBO26Ujd09dYQPuqY7OI+kZWyOl3YV+J/yc9Of61Y5E1erHHIyVGPho1ec3Q8Axj47IgGCRFg/pQf8nGlt4ukvmn4xnNSYVb0YNL6RdgB2Y4f32lm1Rh+LPHQytqD4raVEYfAP1gDKXmLieBw3/58= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from omf11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 629EAC0E91 for ; Wed, 6 Mar 2024 13:54:41 +0000 (UTC) Received: from [HIDDEN] (Authenticated sender: Brian.Inglis@SystematicSW.ab.ca) by omf11.hostedemail.com (Postfix) with ESMTPA id E1B3520029 for ; Wed, 6 Mar 2024 13:54:39 +0000 (UTC) Message-ID: <208962d1-78cd-4e2d-9e4a-a1edcbf5e809@SystematicSW.ab.ca> Date: Wed, 6 Mar 2024 06:54:39 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Reply-To: cygwin@cygwin.com Subject: Re: ls/stat on OneDrive causes download of files Content-Language: en-CA To: cygwin@cygwin.com References: From: Brian Inglis Organization: Systematic Software In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: E1B3520029 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,KAM_DMARC_STATUS,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Stat-Signature: ifbw9orrx5dra3i3mxh79ecr9yqkiozf X-Rspamd-Server: rspamout04 X-Session-Marker: 427269616E2E496E676C69734053797374656D6174696353572E61622E6361 X-Session-ID: U2FsdGVkX1/dOvHJ37m1NoNkCgJRgDlmb7kVGzGxCjg= X-HE-Tag: 1709733279-307392 X-HE-Meta: U2FsdGVkX18DNvQmXzBJFXjKYELFZ1hv1SfnvmTGqvy4jvXxR/iKeIQEoUe++KuAgFe2mLBCM3Ec76ZhwrXVOZ0HC6UUw1OlWPFx1AfIVFHgRaANxp9CzcVNwO8Vt2si8NmZBh+VKJ4jobf/tupfAfIZC9pONl0sduJLIdYHKwK9qUXBUqENKtgm4WhPHzBw13cxLKjsve1FxhYO8BobOz82QFgFJa/YFKvtRQVAC/0gDO/JheXJSu230fea6sA46XpCGW16G2OuVA7RnYe/1x+lEoxdOCdzgWxOXPawK/jBMd+ZiQ8n38gXpeBdYpJ7z8uOt2m70VtDkS1XGE1xvbPDxvbbxIXCAgLDhm8afvqYW6lpaRfy7Qnv5XerThYWjJ9p1Fq8llTImI3gXz7L95GVsqDfCguP X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 2024-03-06 06:28, Corinna Vinschen via Cygwin wrote: > On Mar 6 14:22, Corinna Vinschen via Cygwin wrote: >> On Mar 5 19:54, Marcin Wisnicki via Cygwin wrote: >>> If I invoke ls or anything else that does stat inside OneDrive folder >>> it will trigger download of all files. >>> >>> OneDrive uses placeholder files[1] to represent remote files. >>> >>> I'm guessing reading file content in stat is to support detection of >>> actually executable files as in here[2]? >>> >>> I think this should be disabled on non-hydrated placeholder files. >>> Running `find` or 'ls -R` and having your entire OneDrive downloaded >>> is extremely problematic. >>> >>> I could live without executable scripts in the OneDrive folder and >>> it's easy to mark files as always offline to solve it. >>> >>> Another idea is to skip checking files with extensions known to be >>> non-executable such as jpg (or just any extensions that is not known >>> to be executable). >> >> Nothing of this makes sense from a POSIX library POV. The library can >> either not handle placeholder files specially, as today, or it can >> handle them all the same way. >> >> Given these placeholder files are actually reparse points of type >> IO_REPARSE_TAG_FILE_PLACEHOLDER, we can handle them as symbolic links. >> >> However, the structure of the IO_REPARSE_TAG_FILE_PLACEHOLDER reparse >> data buffer is undocumented. It would be helpful if somebody using >> OneDrive would examine the content of the attached REPARSE_DATA_BUFFER. >> >>> [2] https://github.com/msys2/msys2-runtime/blob/msys2-3.4.10/winsup/cygwin/fhandler/disk_file.cc#L548 >> >> The NtReadFile call at this point is not the problem. It would be >> helpful to point to Cygwin's source instead of MSYS2, btw. > > Oh, btw., this is from > https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/c8e77b37-3909-4fe6-a4ea-2b9d423b1ee4: > > IO_REPARSE_TAG_FILE_PLACEHOLDER > 0x80000015 > > Obsolete. > --------- > Used by Windows Shell for legacy placeholder files in Windows 8.1. > Server-side interpretation only, not meaningful over the wire. > > So even if we support them, what is their replacement in W10 and later? May or not help: https://stackoverflow.com/questions/59152220/cant-get-reparse-point-information-for-the-onedrive-folder -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut -- Antoine de Saint-Exupéry