[BBLISA] Long term data storage - archiving solutions - other ideas

[John Stoffel]

Stephen> We currently have Isilon NAS with 600 TB of data. We find we
Stephen> have 20 to 40 Tb of lab data we would like to offload
Stephen> spinning disk. We have TSM available from the campus IT
Stephen> group.  They offer an archiving service.

Stephen> We would like to hear from others organizations that have the
Stephen> same need and listen to how they are handling this issue.

We have the same type of problem at work.  We have chip design data
that we need to hold onto for 20 years.  Hah!  The state of the art in
terms of tape storage, tools, etc has moved on quite a bit in just the
ten years I've been here.  I've got a couple of drawers full of 8mm
tapes and even a small library I could try to read them with.  Not
likely to happen though...

What we do now is:

1. Write two tapes for redundancy.  Actually we write one tape, then
   read it back and write a second.  Slower, but means we know with
   more confidence that the media is good.  

2. Keeping aound the tools/processes/know how to re-read the data and
   use it is the hard part. 

The hard part is when you get about ten years down the road and it's
time to move to new media.  Have you asked the TSM group how they plan
to do this transition and what the costs are?  

And can you handle the cost?  Or can then?  

And I'm sure you'll also have grown your data bout about 10 times in
the next ten years anyway... how will you archive the next 200Tb of
data you need saved?  And what about the indexes to *find* the needle
in the haystack down the line?

John