[BBLISA] ZFS Anyone?

[Theo Van Dinter]

On Sat, Apr 4, 2009 at 12:18 AM, Edward Ned Harvey
<bblisa3 at nedharvey.com> wrote:
>>     * Does anyone have experience with ZFS?  What has your experience
>> been?
>
> I am also new to ZFS, but I agree it's the greatest thing since the
> transistor.

There are some issues with it, but I agree that it's probably the best
choice of filesystem currently out there.

Some issues that come to mind: lack of user, group, and tree quotas
(volume quotas aren't always good enough); ZFS property size limit is
1k bytes which isn't enough for some things (really long nfs export
host listings, for instance); scaling issues (yes, scaling.  try
making several thousand volumes with a few dozen snapshots each, which
is really easy if doing volume per user w/ snapshots for homedirs ...
also deleting heavily used multi TB volumes takes a long long time
...)

ZFS replication is also very minimal and it has some oddities -- if
replicating a compressed volume over, say, ssh, to an identically
configured volume we have to read off disk, uncompress it, recompress
it, send through ssh, uncompress it, compress it, write to disk.
d'oh!

>>     * Is it stable/reliable/etc.  Honestly, before this experience LVM2
>>       has been rock solid for me across many deployments.
>
> Speaking from everyone else's experience and not my own - Yes, it's totally
> stable and solid.

I would agree.  It's also really cool to see a raidz2 group w/ more
than 2 failed disks still functioning and able to fully reconstruct.

It is a little weird that you can import failed pools, and still poke
around the filesystem for the files/dirs that still exist in the pool,
but it's probably more feature than bug.

>>     * What are the worst/most annoying parts of dealing with ZFS?
> Solaris.

+1!  At work we spent a lot of effort making a NAS appliance out of
OpenSolaris and ZFS, and we had to continuously fight with Solaris.

> Why is ZFS constrained to Solaris?  Because the license terms are
> incompatible with linux.  Free open source, yes.  Not legal to build into
> the linux kernel.

I very much look forward to linux+zfs.

> Snapshots make it so users always have a safety net, and they can 99% of the
> time restore their own files without asking you to restore from tape.
> Really good feature.

+1

> Checksumming.  I love this one.  Whether you notice it or not, all disks
> make mistakes undetected - accidentally writing a 1 when it meant to write a
> 0 or whatnot.

++1!  Most people don't think about this, but if you can't be sure
that when you write a 1 that you'll get back a 1 when you read ...
just give up, imo.  Especially when you start getting into multi-TB
storage.