[BBLISA] System Backup thoughts and questions...

[Bob Webber]

My first suggestion is that whatever you use for file copying, run it  
in a verbose mode and collect the output.  Then use ls or find to  
produce an audit of the files that should be in the backup and compare  
that with the output of the file copying.  Compare the two hierarchies  
on disk, too!

In fact, you might want to do that first, before concluding that your  
copying mechanism is broken now: du reports in blocks, and (a) you  
might have different block sizes on source and target systems; (b)  
minimum allocation size might be different on source and target  
systems.  Just doing an "ls -alR | sort " and collecting the outputs  
from the source and target hierarchies should give you files you can  
simply diff to check for missing files in the target.

Note that the output of du will not tell you if some files are  
truncated if truncation is less than a block...

Using tar is definitely possible, and I'm not aware of any limitation  
on the number of inodes it will copy, but some versions of tar run  
into limits on the path + filename length (256 characters, if my old  
brain cells don't deceive me) and if you have symbolic links in your  
source directory you'll need to specify copying the links rather than  
the files they point to.  As Kathryn Smith noted, using tar with an  
absolute path is a very bad practice, so don't do that.

Rsync should also work reasonably well, and with the right options  
(and reasonably synchronous system clocks) will probably be reasonably  
speedy: you likely will want to specify checking modification time  
(file changes) or change time (inode changes) and only checking  
checksums if there's a discrepancy.

If you have files with unusual structures, such as sparse files,  
piping the output of the "find" command, with appropriate arguments,  
into cpio (e.g. find [arguments] | cpio -pdlm [destination directory  
path]) should also work.  Cpio has an advantage if you're using mixed- 
endian systems in that it can swap bytes or words as needed.

Solaris 10 ufsdump will let you select directories to be backed up  
within a filesystem, so if you're on a current Solaris release, that's  
another possibility.

On 8-Jan-09, at 4:06 PM, Richard 'Doc' Kinne wrote:

> Hi Folks:
>
> I'm looking at backups - simple backups right now.
>
> We have a strategy where an old computer is mounted with a large  
> external, removable hard drive. Directories - large directories -  
> that we have on our other production servers are mounted on this  
> small computer via NFS. A cron job then does a simple "cp" from the  
> NFS mounted production drive partitions to to the large, external,  
> removable hard drive.
>
> I thought it was an elegant solution, myself, except for one small,  
> niggling detail.
>
> It doesn't work.
>
> The process doesn't copy all the files. Oh, we're not having a  
> problem with file locks, no. When you do a "du -sh <directory>"  
> comparison between the /scsi/web directory on the backup drive and  
> the production /scsi/web directory the differences measure in the  
> GB. For example my production /scsi partition has 62GB on it. The  
> most recently done backup has 42GB on it!
>
> What our research found is that the cp command apparently has a  
> limit of copying 250,000 inodes. I have image directories on the  
> webserver that have 114,000 files so this is the limit I think I'm  
> running into.
>
> While I'm looking at solutions like Bacula and Amanda, etc., I'm  
> wondering if RSYNCing the files may work.  Or will I run into the  
> same limitation?
>
> Any thoughts?
> ---
> Richard 'Doc' Kinne, [KQR]
> American Association of Variable Star Observers
> <rkinne @ aavso.org>
>
>
>
> _______________________________________________
> bblisa mailing list
> bblisa at bblisa.org
> http://www.bblisa.org/mailman/listinfo/bblisa