Re: [ARSCLIST] Looking for LTO Help

["Daniel G. Epstein" <dan@xxxxxxxxxxxx>]

To my mind, the first big issue for LTO (or any file-based linear
digital format) archive management is that read and write times are
quite large for the amount of data you're likely to be handling. The
LTO-4 spec of 120MBps (for an 800GB uncompressed tape) is not generally
realizable and, depending on your hardware and backup scheme, it
can take many hours to write a single tape (even optimally, one can
expect two hours per tape with no compression) and a 10TB pool of data
MINIMALLY would take a full day and 12.5 tapes.  Retrieval poses similar
issues, particularly if you are seeking through a brace of tapes for a
handful of files.  For this reason, many folks use intermediary pools
of disk storage, sometimes called Virtual Tape Libraries (VTLs) as their
primary backup destinations, with tape as a longer-term, less often
accessed, but more reliable tertiary copy. In either case, I highly
recommend looking at robotic tape libraries with at least 2-3 times the
total projected data capacity of the online pool, and preferably with
multiple drives in the library. This will allow the scheduling of long
archiving jobs while also allowing regular backup or retrieval jobs and
will also facilitate easy rotation of tape sets out to "cold storage."

Also, as a note, I would not anticipate great compression performance
for audio and video data and the speed/computational hit that you take
to increase your tape capacity is probably not worth it.

The second, and much larger issue is asset management of the materials
in the archive. Most of the standard backup and tape library management
software is highly focused on data management for the IT industry and
does a lousy job of handling search/retrieval of sets of audio and video
files associated with particular projects. If you're simply looking
to have a snapshot of your entire disk pool on tape for catastrophic
failure recovery, that might be acceptable, but if you anticipate a need
of moving particular groups of assets to and from tape, you'll probably
want something more tailored to a media library environment. There are
a couple of vendors on the market (the ones I'm aware of are primarily
focused on broadcast, and unfortunately carry the associated price tag)
that handle media asset archive management and then usually integrate
with other "live" asset management systems. However, if you already have
a system in place, your vendor may have, or be able to recommend, an
archive component.

Finally, whatever backup/archiving software is being used really
should support cryptographic verification of file integrity as data
is being written data out to the VTL and to tape. Barring that, I
would recommend manual comparisons on the archive sets destined for
off-site storage. Many backup packages will rely on the underlying
operating system to report successful copying of files. The OS then
relies on the underlying hardware to report success of each block
written. Unfortunately, neither of these mechanisms are 100% reliable.
(I was reading an interesting article on this in the April 2008 issue of
;login titled "Don't Blame Disks for Every Storage Subsystem Failure"
<http://www.usenix.org/publications/login/2008-06/index.html>).
Needless to say, introducing random data errors into your media archives
is undesirable and very hard to catch by eye/ear until it is too late to
deal with the problem.

I'd be more than happy to talk in more detail with anyone who is
interested in such a project.

Cheers,

Dan


On Thu, Nov 20, 2008 at 01:25:45PM -0600, anthony_tadey wrote:

> We are looking to steeping up our Digital file preservation system.
> Currently we are at the multiple server and NAS level.  We are growing
> at a large rate and we think it may be time to start looking at other
> options.  We are looking for an In-house managed system.  We are
> strongly considering adding an LTO or other tape backup format to the
> system.  Is anyone who has experience with LTOs or other tape backup
> formats willing to share there knowledge with us?  Please contact off
> list, I will be passing you along to our collections manager/ IT /
> programmer.
>
> If there are others who have interest in what we decided to do, I will
> be happy to write a little something up about what we finally do and
> about our reasoning.
>
>
> Thank you in advance.
>
>
> Tony Tadey Audio Specialist - Electronic Library Baylor University

-- 

A boast of "I have beens,"   | Daniel G. Epstein
quoted from foolscap tomes,  | Owner/Systems Engineer
is a shadow brushed away     |
by an acorn from an oak tree | Rootlike Technologies, Inc.
or a salmon in a pool.       | http://www.rootlike.com/

GnuPG public keys available from http://pgp.mit.edu/