Editing with vim clears Windows 10 file system archive bit.

Editing with vim clears Windows 10 file system archive bit.

[Steve Ward]

Description of problem:
While using vim 8.2 on cygwin 3.3 (x86_64) on Windows 10,
when editing an existing file with vim and saving it, the Window’s
file system archive bit is always left cleared (not modified state).
This happens, whether the archive bit was set (is modified) or
clear (not modified) initially.  Window’s backup tools rely on this
archive bit for making incremental backups.

The following scenarios DO work:
1. Using vim to create a new file,
   leaves the new file’s archive bit set (is modified).
2. “cp” and redirect with append (“>>”),
   leave the target file’s archive bit set (is modified).
3. “emacs” (emacs-w32) (ver. 27.2) worked for both editing and creating!
4. Editing with vim sets the “last modified date” correctly
   for Linux and Windows.

Tools used to confirm the problem:
The cygwin tool, “cygcheck –c” listed “OK” as the status of every package.
The archive bit settings were confirmed with Window’s “File Explorer” and
cygwin’s “attrib”.
The cygwin tool, “chattr,” could be used as a crude workaround,
but I rather have vim do it.

My search for solutions:
I have perused 8 years of “The Cygwin Archives” for this issue,
but could find nothing relevant.
Google Web searches, and stackoverflow.com, vim.org and
vi.stackexchange.com searches turned up nothing relevant.
On my local machine, a search of /usr/share/vim/vim82/doc/*.txt
turned up nothing relevant.

My update history:
10/2013 Cygwin-1.7.25 Windows  8 vim-7.3.1314 No problem with archive bit.
09/2021 Cygwin-3.2.0  Windows 10 vim-8.2.0486 Problem with archive bit.
11/2021 Cygwin-3.3.2  Windows 10 vim-8.2.0486 Problem with archive bit.

Can anyone shed some light on this issue?

Thanks,
Steve

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows

Re: Editing with vim clears Windows 10 file system archive bit.

[Christian Franke]

Steve Ward via Cygwin wrote:
> Description of problem:
> While using vim 8.2 on cygwin 3.3 (x86_64) on Windows 10,
> when editing an existing file with vim and saving it, the Window’s
> file system archive bit is always left cleared (not modified state).
> This happens, whether the archive bit was set (is modified) or
> clear (not modified) initially.

The problem also occurs with 'cp' command:

$ touch file1

$ /bin/cp file1 file2

$ /bin/cp --preserve=mode file1 file3

$ lsattr file?
---a-------- file1
---a-------- file2
------------ file3

Some Cygwin functions apparently clear the archive attribute 
unexpectedly, for example:

int fd = open(filename, O_CREAT|O_TRUNC|O_WRONLY, 0644);
write(fd, "Test\n", 5);
fchmod(fd, 0644); // clears archive attribute
close(fd);

Same with facl(., SETACL, ...). The variants chmod() and acl() are not 
affected.

Regards,
Christian

Re: Editing with vim clears Windows 10 file system archive bit.

[Corinna Vinschen]

On Nov 15 19:54, Christian Franke wrote:
> Steve Ward via Cygwin wrote:
> > Description of problem:
> > While using vim 8.2 on cygwin 3.3 (x86_64) on Windows 10,
> > when editing an existing file with vim and saving it, the Window’s
> > file system archive bit is always left cleared (not modified state).
> > This happens, whether the archive bit was set (is modified) or
> > clear (not modified) initially.
> 
> The problem also occurs with 'cp' command:
> 
> $ touch file1
> 
> $ /bin/cp file1 file2
> 
> $ /bin/cp --preserve=mode file1 file3
> 
> $ lsattr file?
> ---a-------- file1
> ---a-------- file2
> ------------ file3
> 
> Some Cygwin functions apparently clear the archive attribute unexpectedly,
> for example:
> 
> int fd = open(filename, O_CREAT|O_TRUNC|O_WRONLY, 0644);
> write(fd, "Test\n", 5);
> fchmod(fd, 0644); // clears archive attribute
> close(fd);
> 
> Same with facl(., SETACL, ...). The variants chmod() and acl() are not
> affected.

It's funny that it took so long that somebody actually noticed this.
This behaviour is present at least since 2004.  Cygwin *never* actually
cared for the ARCHIVE attribute explicitely.  The reason for the above
observation is the open call containing the O_CREAT flag.  When Cygwin
creates files, it sets the attributes to FILE_ATTRIBUTE_NORMAL only, not
adding the FILE_ATTRIBUTE_ARCHIVE flag. 

Changing that is actually pretty simple, just set FILE_ATTRIBUTE_ARCHIVE
as soon as the underlying NtCreateFile is called for an open(O_CREAT).

Fixed in current git.


Thanks,
Corinna

Re: Editing with vim clears Windows 10 file system archive bit.

[Gary Johnson]

On 2021-11-15, Corinna Vinschen via Cygwin wrote:
> On Nov 15 19:54, Christian Franke wrote:
> > Steve Ward via Cygwin wrote:
> > > Description of problem:
> > > While using vim 8.2 on cygwin 3.3 (x86_64) on Windows 10,
> > > when editing an existing file with vim and saving it, the Window’s
> > > file system archive bit is always left cleared (not modified state).
> > > This happens, whether the archive bit was set (is modified) or
> > > clear (not modified) initially.
> > 
> > The problem also occurs with 'cp' command:
> > 
> > $ touch file1
> > 
> > $ /bin/cp file1 file2
> > 
> > $ /bin/cp --preserve=mode file1 file3
> > 
> > $ lsattr file?
> > ---a-------- file1
> > ---a-------- file2
> > ------------ file3
> > 
> > Some Cygwin functions apparently clear the archive attribute unexpectedly,
> > for example:
> > 
> > int fd = open(filename, O_CREAT|O_TRUNC|O_WRONLY, 0644);
> > write(fd, "Test\n", 5);
> > fchmod(fd, 0644); // clears archive attribute
> > close(fd);
> > 
> > Same with facl(., SETACL, ...). The variants chmod() and acl() are not
> > affected.
> 
> It's funny that it took so long that somebody actually noticed this.
> This behaviour is present at least since 2004.  Cygwin *never* actually
> cared for the ARCHIVE attribute explicitely.  The reason for the above
> observation is the open call containing the O_CREAT flag.  When Cygwin
> creates files, it sets the attributes to FILE_ATTRIBUTE_NORMAL only, not
> adding the FILE_ATTRIBUTE_ARCHIVE flag. 
> 
> Changing that is actually pretty simple, just set FILE_ATTRIBUTE_ARCHIVE
> as soon as the underlying NtCreateFile is called for an open(O_CREAT).
> 
> Fixed in current git.

I had thought that this might be a bug in Vim, so did a git bisect
to find the offending commit.  For the record, it was this one:


commit cd142e3369db8888163a511dbe9907bcd138829c
Author: Bram Moolenaar <Bram@vim.org>
Date:   Thu Nov 16 17:03:45 2017 +0100

    patch 8.0.1300: file permissions may end up wrong when writing

    Problem:    File permissions may end up wrong when writing.
    Solution:   Use fchmod() instead of chmod() when possible.  Don't truncate
                until we know we can change the file.

 src/auto/configure    |  4 +--
 src/config.h.in       |  4 ++-
 src/configure.ac      |  4 +--
 src/fileio.c          | 80 ++++++++++++++++++++++++++++++++++++++++-----------
 src/os_unix.c         | 17 +++++++++--
 src/proto/os_unix.pro |  1 +
 src/version.c         |  2 ++
 7 files changed, 88 insertions(+), 24 deletions(-)


The only change I see to an open() call was removing O_TRUNC on
systems with ftruncate() and adding a later call to ftruncate() on
systems that have it.  There were also some changes to the setting
of permissions (fchown(), fchmod() and chmod()).  These changes were
all in the buf_write() function in fileio.c.

That open() call had the O_CREAT flag before and after the change.

Regards,
Gary

Re: Editing with vim clears Windows 10 file system archive bit.

[Corinna Vinschen]

On Nov 15 14:18, Gary Johnson wrote:
> On 2021-11-15, Corinna Vinschen via Cygwin wrote:
> > Changing that is actually pretty simple, just set FILE_ATTRIBUTE_ARCHIVE
> > as soon as the underlying NtCreateFile is called for an open(O_CREAT).
> > 
> > Fixed in current git.
> 
> I had thought that this might be a bug in Vim, so did a git bisect
> to find the offending commit.  For the record, it was this one:
> [...]
> The only change I see to an open() call was removing O_TRUNC on
> systems with ftruncate() and adding a later call to ftruncate() on
> systems that have it.  There were also some changes to the setting
> of permissions (fchown(), fchmod() and chmod()).  These changes were
> all in the buf_write() function in fileio.c.
> 
> That open() call had the O_CREAT flag before and after the change.

You are sooo right.

My bugfix appears to work, but it's fixing a non-existent bug.
NtCreateFile actually sets the ARCHIVE bit all the time when creating a
new file, even if it's not explicitely given as parameter.  That makes a
lot of sense, of course, given how the archiving mechanism works on
Windows.

The *real* problem is in fact that Cygwin caches the wrong file
attribute bits when creating a new file, and that's where fchmod fails:
It writes back the wrongly cached bits.  This doesn't happen with
chmod, because it re-opens the existing file and fetches the correct
attributes bit at the time.

I guess I'll revert my patch and create a new one which is more to the
point.


Thanks,
Corinna