[BBLISA] Fwd: Moving 100 GB and 1.3 million files

[Ian Stokes-Rees]

> Until this thread I had always assumed it was because the read operations
> were having to seek all around the original disks, not that writes were
> being a pain.  If you actually do end up debugging it I'd be interested
> in the results.

Thanks to everyone for the input.  I will hopefully find some time to
try out the various suggestions next week and will report back what I
find, esp if I discover how to improve these kinds of operations.

The operation that inspired this post finished over night.  The trace of its
output is below.  To briefly answer a few questions people have posed: I
used "mv" between NFS mounted filesystems.  The XRaid (destination) uses
a fibre channel connected RAID-5 array (4 TB usable) which is managed by
an XServe (hence my expectation that the destination FS was HFS).

It took a little over 1.5 days wall clock to move 1.3 million files and
100 GB, and a little under 3 hours CPU time on the destination system (I
started the move from the server hosting the destination file system).

Ian

root# tick=`date +%s`; date; time mv [23]*
/Volumes/MIXRAID/ijstokes/mr/phaser/temp/; date; tock=`date +%s`; echo
$(( $tock - $tick ))
Wed Jul 21 16:22:39 EDT 2010

mv: 3fze/group/results/._3fze-partial-20100208.dgraph: could not copy
extended attributes to
/Volumes/MIXRAID/ijstokes/mr/phaser/temp/3fze/group/results/._3fze-partial-20100208.dgraph:
Operation not permitted
mv: 3fze/group/results/3fze-partial-20100208.dgraph/Contents/._PkgInfo:
could not copy extended attributes to
/Volumes/MIXRAID/ijstokes/mr/phaser/temp/3fze/group/results/3fze-partial-20100208.dgraph/Contents/._PkgInfo:
Operation not permitted

real    2263m18.835s
user    3m37.100s
sys     154m26.706s
Fri Jul 23 06:05:58 EDT 2010
135799


-------------- next part --------------
A non-text attachment was scrubbed...
Name: ijstokes.vcf
Type: text/x-vcard
Size: 383 bytes
Desc: not available
Url : http://www.bblisa.org/pipermail/bblisa/attachments/20100723/37389515/attachment.vcf