[BBLISA] ZFS Anyone?

[Dewey Sasser]

Rudie, Tony wrote:

> > So, what would be the overall plan?  Buy faster storage hardware and also rebuild the storage controller node?  Or just rebuild the storage controller node, and assume that the underlying storage is really fast enough?   
> >   
>   

#include <forehead-slap.h>

Buy vs. build and new vs. repurposed is all open with the exception that
I cannot have this system off line for longer than a weekend and even
that will pinch a bit.

Basically my goal is "make it better" and my operations area is ~4-6
TB.  From looking at the list price, I could easily buy a low end 7210
and *might* be able to get a mid-range one.

"Make it better" means that users never experience noticeable
performance slowdowns and the system doesn't hang (ESX by itself doesn't
really meet my expectations in this case and LabManager really does NOT
behave well).

I know that spreading my load across more spindles would make it faster
though I can't gang my existing spindles all into a single RAID (they're
3 different sizes/types).  I actually *don't* have an idea of when
things fall off a cliff -- the other day I observed 2800 IOps @ 173ms
average service time with 75% write load, but when I plot average
service time vs IOps I don't see anything sensible.  I'm sure my
controller cache is helping me out some places but I don't believe I
have the tools in place to know when or measure how much.  If I had all
15k drives I strongly suspect my performance would never be a problem
and I'd only be griping about RedHat hanging on resize.

(By way of sanity test, my average of 990 IOps is faster than I
calculate my RAID10 of 7.2k drives should be able to sustain so I'm
pretty sure I'm getting a good boost from the cache.)


Peter Galvin wrote:

> > ZFS will go at spindle speed for sequential I/O. For random I/O it's best to
> > have flash memory for the ZIL (intent log). I assume you'd use NFS if you
> > went with an external device? If so that would be mostly random I/O and
> > you'd want the ZIL to be on flash. 
>   

"Spindle speed" is an interesting concept.  I know I need to go way
faster than a single spindle will give me.  I have an idea of how
various RAIDs scale with number of spindles, but how does ZFS do?  If I
buy a 7210 with 46 drives, do I effectively see 46 * (1 drive's worth of
IOps)?  With RAID 6 I would expect 1/6*46*(1 drive's worth of IOps).

thanks,

--
Dewey