* Recursively recreate hierarchy with NTFS hardlinks by Cygwin
@ 2017-08-02 11:33 Oleksandr Gavenko
2017-08-02 11:50 ` Oleksandr Gavenko
0 siblings, 1 reply; 3+ messages in thread
From: Oleksandr Gavenko @ 2017-08-02 11:33 UTC (permalink / raw)
To: cygwin
I am going to write backup solution for my personal laptop.
I have 240GB SSD for development work and 500GB HDD for media files and
backups.
I am care only about project files and don't care about windows files or
program installation (as registry also should be preserved for backup to be
useful).
On Linux I uses:
cp -al /backup/proj/DATEOLD /backup/proj/DATENEW
rsync ... /home/user/proj/ /backup/proj/DATENEW/
and employ hardlinks to preserve space.
NTFS FS has hardlinks. With ``cygutils-extra`` I can:
$ winln $FROM $TO
$ echo >>$FROM
$ cmp $FROM $TO && echo OK
But I don't understand how to emulate recursive behavior of ``cp -r -l``.
Cygwin ``cp -l`` can use hard links but only with:
CYGWIN=winsymlinks:native
``rsync --hard-links`` isn't reliable:
bash# echo 1 >orig.txt
bash# rsync -a --hard-links orig.txt new.txt
bash# echo 2 >>orig.txt
bash# diff -u orig.txt new.txt
--- orig.txt 2017-08-02 14:04:36.976875300 +0300
+++ new.txt 2017-08-02 14:04:16.547209000 +0300
@@ -1,2 +1 @@
1
-2
There is Windows build-in ``robocopy`` utility but its documentation looks too
complex, I don't like to mix Cygwin and Windows paths and looks like it
doesn't support hard link from "Robocopy.exe /? | grep -i hard".
As development of rdiff-backup stalled in 2009 I don't believe that it
supports hard links in native Windows build.
As a bonus I am also interested to hear about solution with include data
integrity checks to detect data altering or rotting of storage bits in backup.
I think about employing md5sum utility...
--
http://defun.work/
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Recursively recreate hierarchy with NTFS hardlinks by Cygwin
2017-08-02 11:33 Recursively recreate hierarchy with NTFS hardlinks by Cygwin Oleksandr Gavenko
@ 2017-08-02 11:50 ` Oleksandr Gavenko
2017-08-02 13:26 ` Oleksandr Gavenko
0 siblings, 1 reply; 3+ messages in thread
From: Oleksandr Gavenko @ 2017-08-02 11:50 UTC (permalink / raw)
To: cygwin
On 2017-08-02, Oleksandr Gavenko wrote:
> On Linux I uses:
>
> cp -al /backup/proj/DATEOLD /backup/proj/DATENEW
> rsync ... /home/user/proj/ /backup/proj/DATENEW/
>
> and employ hardlinks to preserve space.
>
> ``rsync --hard-links`` isn't reliable:
>
> bash# echo 1 >orig.txt
>
> bash# rsync -a --hard-links orig.txt new.txt
>
> bash# echo 2 >>orig.txt
>
> bash# diff -u orig.txt new.txt
> --- orig.txt 2017-08-02 14:04:36.976875300 +0300
> +++ new.txt 2017-08-02 14:04:16.547209000 +0300
> @@ -1,2 +1 @@
> 1
> -2
>
Experiments shown that my goal can be archived in single command:
mkdir orig
echo 1 >>orig/my.txt
mkdir backup
rsync -a orig/ backup/1
rsync -a --link-dest=../1 orig/ backup/2
echo 2 >>backup/2/my.txt
cmp backup/1/my.txt backup/2/my.txt && echo ok
cmp orig/my.txt backup/2/my.txt || echo ok
Thanks for Cygwin & rsync with NTFS link support!
--
http://defun.work/
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Recursively recreate hierarchy with NTFS hardlinks by Cygwin
2017-08-02 11:50 ` Oleksandr Gavenko
@ 2017-08-02 13:26 ` Oleksandr Gavenko
0 siblings, 0 replies; 3+ messages in thread
From: Oleksandr Gavenko @ 2017-08-02 13:26 UTC (permalink / raw)
To: cygwin
On 2017-08-02, Oleksandr Gavenko wrote:
> Experiments shown that my goal can be archived in single command:
>
> mkdir orig
> echo 1 >>orig/my.txt
>
> mkdir backup
> rsync -a orig/ backup/1
> rsync -a --link-dest=../1 orig/ backup/2
>
> echo 2 >>backup/2/my.txt
> cmp backup/1/my.txt backup/2/my.txt && echo ok
> cmp orig/my.txt backup/2/my.txt || echo ok
``fsutil`` is built-in Windows utility:
bash# fsutil hardlink list backup/2/my.txt
\home\tmp\backup\1\my.txt
\home\tmp\backup\2\my.txt
bash# mv backup/1 backup/3
bash# fsutil hardlink list backup/2/my.txt
\home\tmp\backup\3\my.txt
\home\tmp\backup\2\my.txt
That is additional proof of work.
My bash backup script looks like:
LOG=/cygdrive/c/home/backup/backup-job.log
DST=/cygdrive/d/backup
DATE=`date +%F`
COMMON_RSYNC_OPT=( -a --delete )
log() {
echo `date +'%F %T'` "$@" >>$LOG
}
backup_simple() {
local SRC=$1
local BASE=${SRC##*/}
if [ ! -d $SRC ]; then
log "$SRC does not exist, leave"
return
fi
if [ -d $DST/$BASE/$DATE ]; then
log "$SRC is already backuped to $DST/$BASE/$DATE"
return
fi
if [ ! -d $DST/$BASE ]; then
mkdir -p $DST/$BASE
fi
local LAST=`cd $DST/$BASE; find . -maxdepth 1 -type d | grep '[0-9]\{4\}-[0-9]\{2\}-[0-9]\{2\}' | sort -r | head`
local RSYNC_OPT=( "${COMMON_RSYNC_OPT[@]}" )
if [ -n "$LAST" ]; then
RSYNC_OPT+=( --link-dest=../$LAST )
fi
rsync "${RSYNC_OPT[@]}" $SRC/ $DST/$BASE/$DATE/
log "$BASE is backuped"
}
backup_simple /cygdrive/c/home/devel/soapui
backup_simple /cygdrive/c/home/devel/postman
================================================================
What options is recommended for using Cygwin's rsync?
Backuped files don't need to preserve permission, ACL, etc. The goal is to
preserve hierarchy and content and make backuping lightweight (with
timestamps/size checks and hardlinks).
I collected list of:
-t --no-p --no-l --no-acls --no-o --no-g --no-D -O -J -m --no-partial --delete
to ignore most of file meta-information and make time/size bases checks.
================================================================
What is the better way to schedule backup task?
I've got familiar with Windows Task Scheduler, it has interesting options:
* run task as soon as possible after scheduler start is missed
* stop task if it runs longer then
* wake the computer to run this task
Do I need to bother with Cygwin CRON?
--
http://defun.work/
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2017-08-02 13:26 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-08-02 11:33 Recursively recreate hierarchy with NTFS hardlinks by Cygwin Oleksandr Gavenko
2017-08-02 11:50 ` Oleksandr Gavenko
2017-08-02 13:26 ` Oleksandr Gavenko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).