tar incremental backups and ctime‏ problem

tar incremental backups and ctime‏ problem

[x y]

Hello,

The MS Office applications are changing the ctime value of .doc and
.xls files even if they are not modified. Opening and closing an
office file without modifying the content is changing ctime and the
unmodified document is included in the incremental tar archive. Could
it be possible to use the mtime value instead of ctime to avoid the
the reappearance of the unchanged files in the incremental archive?

Test OS  : Windows 7 Sp1

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: tar incremental backups and ctime‏ problem

[Marco Atzeri]

On 23/05/2016 10:40, x y wrote:
> Hello,
>
> The MS Office applications are changing the ctime value of .doc and
> .xls files even if they are not modified. Opening and closing an
> office file without modifying the content is changing ctime and the
> unmodified document is included in the incremental tar archive. Could
> it be possible to use the mtime value instead of ctime to avoid the
> the reappearance of the unchanged files in the incremental archive?
>
> Test OS  : Windows 7 Sp1
>


find -ctime should be able you to build the list of files
that are changed and that tar should process

Regards
Marco

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: tar incremental backups and ctime‏ problem

[x y]

Hi Marco,

Thanks for your reply. The problem is that there are a lot of files to
be checked so replacing ctime by time to support tar incremental
backup should be more practical.

On Mon, May 23, 2016 at 11:46 AM, Marco Atzeri <marco.atzeri@gmail.com> wrote:
> On 23/05/2016 10:40, x y wrote:
>>
>> Hello,
>>
>> The MS Office applications are changing the ctime value of .doc and
>> .xls files even if they are not modified. Opening and closing an
>> office file without modifying the content is changing ctime and the
>> unmodified document is included in the incremental tar archive. Could
>> it be possible to use the mtime value instead of ctime to avoid the
>> the reappearance of the unchanged files in the incremental archive?
>>
>> Test OS  : Windows 7 Sp1
>>
>
>
> find -ctime should be able you to build the list of files
> that are changed and that tar should process
>
> Regards
> Marco
>
> --
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: tar incremental backups and ctime‏ problem

[Marco Atzeri]

On 23/05/2016 10:54, x y wrote:
> Hi Marco,
>
> Thanks for your reply. The problem is that there are a lot of files to
> be checked so replacing ctime by time to support tar incremental
> backup should be more practical.

Bottom post on this mailing list, please.

It is not clear to me your expectation:
- are you asking how to use ctime to select the file with tar alone ?
   It is not possible for my understanding of the manual.

- Are you asking the package maintainer to change the behaviour of
   cygwin tar ? Unlikely to happen, but I leave to him.

Regards
Marco


>
> On Mon, May 23, 2016 at 11:46 AM, Marco Atzeri <marco.atzeri@gmail.com> wrote:
>> On 23/05/2016 10:40, x y wrote:
>>>
>>> Hello,
>>>
>>> The MS Office applications are changing the ctime value of .doc and
>>> .xls files even if they are not modified. Opening and closing an
>>> office file without modifying the content is changing ctime and the
>>> unmodified document is included in the incremental tar archive. Could
>>> it be possible to use the mtime value instead of ctime to avoid the
>>> the reappearance of the unchanged files in the incremental archive?
>>>
>>> Test OS  : Windows 7 Sp1
>>>
>>
>>
>> find -ctime should be able you to build the list of files
>> that are changed and that tar should process
>>
>> Regards
>> Marco
>>


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: tar incremental backups and ctime‏ problem

[x y]

It is not clear to me your expectation:
- are you asking how to use ctime to select the file with tar alone ?
  It is not possible for my understanding of the manual.

- Are you asking the package maintainer to change the behaviour of
  cygwin tar ? Unlikely to happen, but I leave to him.

Regards
Marco


Hi Marco,

Sorry, I am new to the mailing list. If I am not wrong, tar is
checking both of the ctime and mtime values to compare files during
incremental backups. Since opening and closing a MS document without
changing the content updates ctime, it would be preferable to add a
new option to tar to use only mtime for file comparing during
incremental backups.

On Mon, May 23, 2016 at 12:06 PM, Marco Atzeri <marco.atzeri@gmail.com> wrote:
> On 23/05/2016 10:54, x y wrote:
>>
>> Hi Marco,
>>
>> Thanks for your reply. The problem is that there are a lot of files to
>> be checked so replacing ctime by time to support tar incremental
>> backup should be more practical.
>
>
> Bottom post on this mailing list, please.
>
> It is not clear to me your expectation:
> - are you asking how to use ctime to select the file with tar alone ?
>   It is not possible for my understanding of the manual.
>
> - Are you asking the package maintainer to change the behaviour of
>   cygwin tar ? Unlikely to happen, but I leave to him.
>
> Regards
> Marco
>
>
>
>>
>> On Mon, May 23, 2016 at 11:46 AM, Marco Atzeri <marco.atzeri@gmail.com>
>> wrote:
>>>
>>> On 23/05/2016 10:40, x y wrote:
>>>>
>>>>
>>>> Hello,
>>>>
>>>> The MS Office applications are changing the ctime value of .doc and
>>>> .xls files even if they are not modified. Opening and closing an
>>>> office file without modifying the content is changing ctime and the
>>>> unmodified document is included in the incremental tar archive. Could
>>>> it be possible to use the mtime value instead of ctime to avoid the
>>>> the reappearance of the unchanged files in the incremental archive?
>>>>
>>>> Test OS  : Windows 7 Sp1
>>>>
>>>
>>>
>>> find -ctime should be able you to build the list of files
>>> that are changed and that tar should process
>>>
>>> Regards
>>> Marco
>>>
>
>
> --
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: tar incremental backups and ctime‏ problem

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1060 bytes --]

On 05/23/2016 03:18 AM, x y wrote:
> It is not clear to me your expectation:
> - are you asking how to use ctime to select the file with tar alone ?
>   It is not possible for my understanding of the manual.
> 
> - Are you asking the package maintainer to change the behaviour of
>   cygwin tar ? Unlikely to happen, but I leave to him.
> 
> Regards
> Marco
> 
> 
> Hi Marco,
> 
> Sorry, I am new to the mailing list. If I am not wrong, tar is
> checking both of the ctime and mtime values to compare files during
> incremental backups. Since opening and closing a MS document without
> changing the content updates ctime, it would be preferable to add a
> new option to tar to use only mtime for file comparing during
> incremental backups.

mtime is fakeable, ctime is not.  Using only mtime makes it likely that
your incremental backup will miss files.  I don't have any good reason
to differ from upstream behavior here.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: tar incremental backups and ctime‏ problem

[x y]

>mtime is fakeable, ctime is not.  Using only mtime makes it likely that
>your incremental backup will miss files.  I don't have any good reason
>to differ from upstream behavior here.

Hi Eric,

The problem is not faking time stamps. Even commercial Windows backup
programs are checking the modification time to identify the modified
files.

Consider that you have a lot of files opened and closed without any
modification in your company. Because of the priority of the ctime
time stamp, reintroducing all of those files to the incremental backup
does not make any sense. tar has also the capacity to create
differential backups with the condition of taking care of the snapshot
file. The ctime issue can result in unnecessarily big differential
backups filled with unmodified files.

Cygwin tar can be a good  alternative for Windows users to do
differential \ incremental backups but the ctime problem must be
solved.



On Mon, May 23, 2016 at 5:29 PM, Eric Blake <eblake@redhat.com> wrote:
> On 05/23/2016 03:18 AM, x y wrote:
>> It is not clear to me your expectation:
>> - are you asking how to use ctime to select the file with tar alone ?
>>   It is not possible for my understanding of the manual.
>>
>> - Are you asking the package maintainer to change the behaviour of
>>   cygwin tar ? Unlikely to happen, but I leave to him.
>>
>> Regards
>> Marco
>>
>>
>> Hi Marco,
>>
>> Sorry, I am new to the mailing list. If I am not wrong, tar is
>> checking both of the ctime and mtime values to compare files during
>> incremental backups. Since opening and closing a MS document without
>> changing the content updates ctime, it would be preferable to add a
>> new option to tar to use only mtime for file comparing during
>> incremental backups.
>
> mtime is fakeable, ctime is not.  Using only mtime makes it likely that
> your incremental backup will miss files.  I don't have any good reason
> to differ from upstream behavior here.
>
> --
> Eric Blake   eblake redhat com    +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: tar incremental backups and ctime‏ problem

[Vince Rice]

> On Mon, May 23, 2016 at 5:29 PM, Eric Blake <eblake@redhat.com> wrote:
>> On 05/23/2016 03:18 AM, x y wrote:
>>> It is not clear to me your expectation:
>>> - are you asking how to use ctime to select the file with tar alone ?
>>>  It is not possible for my understanding of the manual.
>>> 
>>> - Are you asking the package maintainer to change the behaviour of
>>>  cygwin tar ? Unlikely to happen, but I leave to him.
>>> 
>>> 
>>> Hi Marco,
>>> 
>>> Sorry, I am new to the mailing list. If I am not wrong, tar is
>>> checking both of the ctime and mtime values to compare files during
>>> incremental backups. Since opening and closing a MS document without
>>> changing the content updates ctime, it would be preferable to add a
>>> new option to tar to use only mtime for file comparing during
>>> incremental backups.
>> 
>> mtime is fakeable, ctime is not.  Using only mtime makes it likely that
>> your incremental backup will miss files.  I don't have any good reason
>> to differ from upstream behavior here.
> 
> Hi Eric,
> 
> The problem is not faking time stamps. Even commercial Windows backup
> programs are checking the modification time to identify the modified
> files.
> 
> Consider that you have a lot of files opened and closed without any
> modification in your company. Because of the priority of the ctime
> time stamp, reintroducing all of those files to the incremental backup
> does not make any sense. tar has also the capacity to create
> differential backups with the condition of taking care of the snapshot
> file. The ctime issue can result in unnecessarily big differential
> backups filled with unmodified files.
> 
> Cygwin tar can be a good  alternative for Windows users to do
> differential \ incremental backups but the ctime problem must be
> solved.

This is ultimately Eric’s decision, but…

It doesn’t _have_ to be solved. If someone wants to use Cygwin tar as a backup, then that someone lives with the fact that tar uses ctime. The differential might be a little too big, but no actual harm is done. So what.

I personally don’t want to have to guess at tar’s behavior. I want to know it’s the same on Cygwin as elsewhere.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: tar incremental backups and ctime‏ problem

[Marco Atzeri]

On 23/05/2016 16:57, x y wrote:
>> mtime is fakeable, ctime is not.  Using only mtime makes it likely that
>> your incremental backup will miss files.  I don't have any good reason
>> to differ from upstream behavior here.
>
> Hi Eric,
>
> The problem is not faking time stamps. Even commercial Windows backup
> programs are checking the modification time to identify the modified
> files.
>
> Consider that you have a lot of files opened and closed without any
> modification in your company. Because of the priority of the ctime
> time stamp, reintroducing all of those files to the incremental backup
> does not make any sense. tar has also the capacity to create
> differential backups with the condition of taking care of the snapshot
> file. The ctime issue can result in unnecessarily big differential
> backups filled with unmodified files.
>
> Cygwin tar can be a good  alternative for Windows users to do
> differential \ incremental backups but the ctime problem must be
> solved.
>

It is always possible to create file list with find and use that
to tar whatever using --files-from=FILE option

I don't see the need to change tar behaviour to meet your wish.

Regards
Marco








--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: tar incremental backups and ctime‏ problem

[x y]

>It is always possible to create file list with find and use that
>to tar whatever using --files-from=FILE option

>I don't see the need to change tar behaviour to meet your wish.

Hi Marco,

Consider that you are working in the IT department of a company and
you have thousands of documents in your file server. Concerning the
--file-from option, how can you guess the name of the files which are
just only going to be viewed by the workers? Those files would be not
modified but since the ctime stamp will be updated, they will be
included in the next incremental backup. Why to lose time with the
--files-from option and make things more difficult? Adding an option
to tar.exe to ignore the cname time stamp during differential \
incremental backups should be possible. This should be the natural
method. Tools like rsync does not suffer from such problems so my
request should make sense.

On Mon, May 23, 2016 at 6:35 PM, Marco Atzeri <marco.atzeri@gmail.com> wrote:
> On 23/05/2016 16:57, x y wrote:
>>>
>>> mtime is fakeable, ctime is not.  Using only mtime makes it likely that
>>> your incremental backup will miss files.  I don't have any good reason
>>> to differ from upstream behavior here.
>>
>>
>> Hi Eric,
>>
>> The problem is not faking time stamps. Even commercial Windows backup
>> programs are checking the modification time to identify the modified
>> files.
>>
>> Consider that you have a lot of files opened and closed without any
>> modification in your company. Because of the priority of the ctime
>> time stamp, reintroducing all of those files to the incremental backup
>> does not make any sense. tar has also the capacity to create
>> differential backups with the condition of taking care of the snapshot
>> file. The ctime issue can result in unnecessarily big differential
>> backups filled with unmodified files.
>>
>> Cygwin tar can be a good  alternative for Windows users to do
>> differential \ incremental backups but the ctime problem must be
>> solved.
>>
>
> It is always possible to create file list with find and use that
> to tar whatever using --files-from=FILE option
>
> I don't see the need to change tar behaviour to meet your wish.
>
>
> Regards
> Marco
>
>
>
>
>
>
>
>
> --
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: tar incremental backups and ctime‏ problem

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1635 bytes --]

On 05/23/2016 10:56 AM, x y wrote:
>> It is always possible to create file list with find and use that
>> to tar whatever using --files-from=FILE option
> 
>> I don't see the need to change tar behaviour to meet your wish.
> 
> Hi Marco,
> 
> Consider that you are working in the IT department of a company and
> you have thousands of documents in your file server. Concerning the
> --file-from option, how can you guess the name of the files which are
> just only going to be viewed by the workers? Those files would be not
> modified but since the ctime stamp will be updated, they will be
> included in the next incremental backup. Why to lose time with the
> --files-from option and make things more difficult? Adding an option
> to tar.exe to ignore the cname time stamp during differential \
> incremental backups should be possible. This should be the natural
> method. Tools like rsync does not suffer from such problems so my
> request should make sense.

If it makes that much sense, then propose the extension upstream
(bug-tar@gnu.org); once upstream implements it, Cygwin will pick it up
automatically.  Until then, I don't see a reason for Cygwin to
needlessly differ from upstream.

> 
> On Mon, May 23, 2016 at 6:35 PM, Marco Atzeri <marco.atzeri@gmail.com> wrote:
>> On 23/05/2016 16:57, x y wrote:
>>>>
>>>> mtime is fakeable, ctime is not.  Using only mtime makes it likely that

You're still top-posting.  It's okay to trim the portion of the email
you don't reply to.


-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: tar incremental backups and ctime‏ problem

[Erik Soderquist]

On Mon, May 23, 2016 at 4:40 AM, x y wrote:
> Hello,
>
> The MS Office applications are changing the ctime value of .doc and
> .xls files even if they are not modified. Opening and closing an
> office file without modifying the content is changing ctime and the
> unmodified document is included in the incremental tar archive.

I would say this is an MS Office flaw.  If _nothing_ is, neither ctime
nor mtime should be updated.


> Could it be possible to use the mtime value instead of ctime to avoid
> the the reappearance of the unchanged files in the incremental archive?

Use a find command to build a file list of all files changed since
last backup.  Personally I highly recommend differential against last
full rather than incremental, but that is a different discussion.

Also, you might reach out to Microsoft for why they are modifying any
timestamp if nothing was changed.

-- Erik

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple