[BBLISA] Odd Rsync Behavior...

[John Stoffel]

>>>>> "Edward" == Edward Ned Harvey <bblisa3 at nedharvey.com> writes:

>> > Actually, I have the same behavior - with files "vanished" - But when
>> I =
>> > do my rsync, the source is a snapshot.  So it's impossible for the
>> file =
>> > to have changed in between the initial scan and actual attempted
>> copy.  =
>> > So truly I have no explanation.  And I am made uneasy by this as
>> well.
>> 
>> If you are able to reproduce this, strace/ktrace/truss ought to
>> shed some enlightenment on what's going on here; the only way that
>> rsync should be getting the idea that a file doesn't exist is because
>> the kernel said so.

Edward> Alright - I never tried digging into this before, so I'll give
Edward> it a try now.  It will probably take a few days to get any
Edward> results, cuz it's a 6 hour rsync job that I'll have to wait to
Edward> run on Monday morning at midnight, just to get the list of
Edward> vanished files, and then if I'm lucky, I might be able to
Edward> reproduce it with a small rsync job during the day on Monday.

Another thought, is that you're running into a bug in rsync.  Make
sure you're running the latest 2.6.X version, or the latest 3.0.x
version of rsync on both ends.  I ran into a bunch of wierd bugs with
it at one point.  For example, I see I have 2.6.4 on my systems, as
well as 3.0.3 and 3.0.4 (3.0.4 is my default now).  

Also, the 3.0.x series has improved support for large numbers of
files.  Before it would build up an in-memory list of all files.  Now
it chunks it up and starts sending before it finishes the scans of all
files.  With my 40+ million file rsyncs across the US, this is a big
help.  I still needed to manually split up stuff though, just too big
otherwise.

John