[BBLISA] Odd Rsync Behavior...

[John Stoffel]

>>>>> "Richard" == Richard 'Doc' Kinne <rkinne at aavso.org> writes:

Richard> The partition I am trying to back up is coming from an NFS mounted  
Richard> volume and going to an attached external Firewire drive. The command  
Richard> I used in this test was:

Richard> /usr/bin/rsync -ral --progress --stats --times --perms
Richard> /Volumes/mira/ scsi /Volumes/burly_1/Mira/ --delete

In this case, the -ral is redundant.  'a' mode already includes 'l'.
I usually just do:

  rsync -arv --delete <src>/ <dest>/

In your case, you might also want to do '-H' for hard links if you
think you have them.  To make sure, I usually spell out all the 'a'
options in scripts, so that it's crystal clear what I expect.

Richard> Initially /Volumes/burly_1/Mira was empty.

Richard> All went fine for a few hours. Then, seemingly toward the end, with  
Richard> the "web" directory (probably the most important, of course!) I  
Richard> started getting lines like:

Richard> file has vanished: "/Volumes/mira/scsi/web/vsx/submit/ 
Richard> qry_getRemarks.php"
Richard> file has vanished: "/Volumes/mira/scsi/web/vsx/submit/qry_getStar.php"
Richard> file has vanished: "/Volumes/mira/scsi/web/vsx/submit/ 
Richard> qry_getVarTypes.php"
Richard> file has vanished: "/Volumes/mira/scsi/web/vsx/submit/ 
Richard> qry_updateMethod.php"
Richard> scsi/web/zapper/
Richard> file has vanished: "/Volumes/mira/scsi/web/zapper/.DS_Store"
Richard> file has vanished: "/Volumes/mira/scsi/web/zapper/._index.shtml"
Richard> file has vanished: "/Volumes/mira/scsi/web/zapper/._zapper.jnlp"
Richard> file has vanished: "/Volumes/mira/scsi/web/zapper/._zapperButton.jpg"
Richard> file has vanished: "/Volumes/mira/scsi/web/zapper/._zapperdoc.shtml"
Richard> file has vanished: "/Volumes/mira/scsi/web/zapper/._zapperhelp.shtml"
Richard> file has vanished: "/Volumes/mira/scsi/web/zapper/Zapper.jar"

Richard> This is a small sample toward the end. There were hundreds of lines  
Richard> like this, almost as if rsync had just decided to give up.

Richard> I went to /Volumes/mira/scsi/web/* and I was able to determine that  
Richard> these files were indeed there!

Richard> Rsync ended with this data:

Richard> Number of files: 766415
Richard> Number of files transferred: 746551
Richard> Total file size: 66995757081 bytes
Richard> Total transferred file size: 66995737633 bytes
Richard> Literal data: 43101421090 bytes
Richard> Matched data: 0 bytes
Richard> File list size: 17422528
Richard> File list generation time: 779.208 seconds
Richard> File list transfer time: 0.000 seconds
Richard> Total bytes sent: 43129805144
Richard> Total bytes received: 16543326

Richard> sent 43129805144 bytes  received 16543326 bytes  6411523.66 bytes/sec
Richard> total size is 66995757081  speedup is 1.55
Richard> rsync warning: some files vanished before they could be transferred  
Richard> (code 24) at /SourceCache/rsync/rsync-35.2/rsync/main.c(992)  
Richard> [sender=2.6.9]

You should be ok here, 2.6.9 is the last of the 2.6 series.  I was
thinking you might have run into an rsync bug here.

As others have said, just re-run the exact same rsync command and see
what it says.  That's the beauty of rsync.

Also look into 'rdiff' for multiple backups, with compression and
differences, so that you only keep the differences between the various
versions, instead of multiple full copies.

John