[BBLISA] Fwd: Moving 100 GB and 1.3 million files

[Edward Ned Harvey]

> From: bblisa-bounces at bblisa.org [mailto:bblisa-bounces at bblisa.org] On
> Behalf Of David Allan
> 
> Is my math right?  I'm calculating the OP is getting 650kbps
> throughput.
> That seems wrong for any local file transfer on modern gear.  I don't
> believe my own calculation, though.
> 
> 94GB, 50% complete = 47GB = 47000MB
> 47000MB / 20 hr. = 2350MB/hr. = .652MB/s

Without even checking your numbers, I'll say, your math is probably right,
and your logic is probably wrong.

Suppose you write a 1k file.  Suppose there's 9ms to create the file, and
another 9ms to write the contents of the file, and another 9ms to update the
journal.  (This is probably all an underestimate.)  Then you're only going
to be able to write 1k every 27ms, which is 37 K/s.  Obviously very slow,
and the reason is high latency to write a small piece of data to disk.