<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.5pt;
font-family:Consolas;}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:Consolas;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 92.4pt 1.0in 92.4pt;}
div.Section1
{page:Section1;}
/* List Definitions */
@list l0
{mso-list-id:29190700;
mso-list-type:hybrid;
mso-list-template-ids:-1365496078 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1
{mso-list-id:697587718;
mso-list-type:hybrid;
mso-list-template-ids:1821169010 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l1:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2
{mso-list-id:941647708;
mso-list-type:hybrid;
mso-list-template-ids:-310475362 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l2:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoPlainText>> wouldn't a concatenation of mirrors imply that you
write sequentially</p>
<p class=MsoPlainText>> to a mirror until full before using the next
mirror? where's the</p>
<p class=MsoPlainText>> benefit in that? io to my raid10 is striped
thank you very much.</p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>I'm so glad you asked. :-) I've done a bunch
of work on this recently, specifically as it applies to ZFS. Some of this
info I thought was common knowledge and have been surprised to discover really
isn't. So forgive me if the following is too basic at first - then you
can just skip to the juicy stuff at the end.<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>Originally, hard drives were addressed by Cyllinder,
Head, Sector. CHS. Before long, they invented the concept of
Logical Block Address, LBA, in which the physical geometry of the disk doesn't
matter; each block gets a sequential logical address. So the drive didn't
need to be thought of as a drive anymore; it could be thought of as simply a
random access data stream.<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>Then they invented all sorts of different types of RAID,
in particular I'm talking about Striping and Concatenation. <o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>In Striping, the logical blocks are distributed amongst a
bunch of equally sized disks, as follows:<o:p></o:p></p>
<p class=MsoPlainText> Disk0 Disk1 Disk2 Disk3<o:p></o:p></p>
<p class=MsoPlainText> LB0 LB1 LB2 LB3<o:p></o:p></p>
<p class=MsoPlainText> LB4 LB5 LB6 LB7 <o:p></o:p></p>
<p class=MsoPlainText> ... And so on.<o:p></o:p></p>
<p class=MsoPlainText>This way, if you have a large sequential read or write,
each of the disks can contribute to the data stream, and you get n-times the
sustainable throughput of a single drive.<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>In Concatenation, the logical blocks sequentially fill up
each disk before going into the next disk. Suppose each disk has a size
of 100 blocks (unrealistic, but easy for me to diagram), as follows:<o:p></o:p></p>
<p class=MsoPlainText> Disk0 Disk1 Disk2 Disk3<o:p></o:p></p>
<p class=MsoPlainText> LB0 LB100 LB200 LB300<o:p></o:p></p>
<p class=MsoPlainText> ... ... ... ...<o:p></o:p></p>
<p class=MsoPlainText> LB99 LB199 LB299 LB399<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>One more piece of background information: Gone are
the days when filesystems would attempt to position all the files close
together at the beginning of the physical device. For the last several
years, all modern filesystems distribute the files throughout the physical
device, to attempt maximizing empty space between files and minimize file internal
fragmentation.<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>Now - Some of the fundamental differences between Striping
and Concatenation:<o:p></o:p></p>
<p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l1 level1 lfo1'><![if !supportLists]><span
style='mso-list:Ignore'>1.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>In
Striping, the disk sizes must match each other. <o:p></o:p></p>
<p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l1 level1 lfo1'><![if !supportLists]><span
style='mso-list:Ignore'>2.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>In
Striping, you cannot expand by adding disks.<o:p></o:p></p>
<p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l1 level1 lfo1'><![if !supportLists]><span
style='mso-list:Ignore'>3.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>In
Striping, you get a performance benefit for large sequential IO by distributing
the physical blocks across multiple devices.<o:p></o:p></p>
<p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l1 level1 lfo1'><![if !supportLists]><span
style='mso-list:Ignore'>4.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>In
Striping, you get a performance reduction for files which are larger than 1
block and smaller than n-blocks, because the time to read 1 block is smaller
than the time to seek the head to the block.<o:p></o:p></p>
<p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l1 level1 lfo1'><![if !supportLists]><span
style='mso-list:Ignore'>5.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>In
Concatenation, you can expand as you wish, using new disks of any size.<o:p></o:p></p>
<p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l1 level1 lfo1'><![if !supportLists]><span
style='mso-list:Ignore'>6.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>In
Concatenation, you get a performance benefit for small files, because the files
are scattered about within the filesystem, so you can simultaneously allow
n-disks to be seeking.<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>And now the juicy stuff – <o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>If you have a filesystem such as ZFS, which is aware that
it is using a concatenated disk set, the filesystem itself can gain the large
sequential IO benefits of striping, by fragmenting the file and writing
sequential blocks of the file to non-sequential logical blocks in such a way
that the logical blocks are distributed among multiple physical devices.
In this way, you can use a concatenation set, with an intelligent filesystem,
to gain all the benefits of both striping and concatenation:<o:p></o:p></p>
<p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo2'><![if !supportLists]><span
style='mso-list:Ignore'>1.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>The
disk sizes don’t need to match each other<o:p></o:p></p>
<p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo2'><![if !supportLists]><span
style='mso-list:Ignore'>2.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>You
can add any size disk to an existing volume<o:p></o:p></p>
<p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo2'><![if !supportLists]><span
style='mso-list:Ignore'>3.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>You
have optimal performance for small files<o:p></o:p></p>
<p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo2'><![if !supportLists]><span
style='mso-list:Ignore'>4.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>You
have optimal performance for large files<o:p></o:p></p>
<p class=MsoPlainText>For this reason, ZFS does not offer striping as an
option. Many people mistakenly say “striping” when talking
about ZFS concatenation. It’s a common misnomer.<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>The only ingredient which is missing is redundancy.
The way to solve this while maintaining optimal performance is to use a
concatenation of mirrors. <o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>Comparing a concatenation of mirrors versus raidz or
raidz2 (assuming you have a hotspare):<o:p></o:p></p>
<p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l2 level1 lfo3'><![if !supportLists]><span
style='mso-list:Ignore'>1.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Concatenation
of mirrors achieves optimal performance using 2n+1 disks<o:p></o:p></p>
<p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l2 level1 lfo3'><![if !supportLists]><span
style='mso-list:Ignore'>2.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Raidz
achieves somewhat less performance using n+2 disks<o:p></o:p></p>
<p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l2 level1 lfo3'><![if !supportLists]><span
style='mso-list:Ignore'>3.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Raidz2
achieves still lower performance using n+3 disks<o:p></o:p></p>
<p class=MsoPlainText>Using the concatenation of mirrors, you get maximum
performance and flexibility, but you must buy a higher number of disks.
The only reason to use raidz or raidz2 instead is to limit the number of disks
required.<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
</div>
</body>
</html>