{"video_id":"wQS-IhjkBSA","title":"100TB for $2,850??? - Are Archive Drives Useless?","channel":"Linus Tech Tips","show":"Linus Tech Tips","published_at":"2017-05-06T14:53:29Z","duration_s":501,"segments":[{"start_s":0.4,"end_s":6.96,"text":"Seagate archive drives. These things are","speaker":null,"is_sponsor":0},{"start_s":3.84,"end_s":9.76,"text":"freaking cheap for how much capacity you","speaker":null,"is_sponsor":0},{"start_s":6.96,"end_s":14.24,"text":"get. I can actually link my drive cost calculator spreadsheet that I used to","speaker":null,"is_sponsor":0},{"start_s":12.16,"end_s":18.96,"text":"make this chart under the video, by the way. But when I started looking into","speaker":null,"is_sponsor":0},{"start_s":16.48,"end_s":25.199,"text":"picking up some of these drives for our long-term storage NAS, I heard the","speaker":null,"is_sponsor":0},{"start_s":22.32,"end_s":30.48,"text":"performance totally sucked. So, I asked Seagate to send a few of them over, and","speaker":null,"is_sponsor":0},{"start_s":27.359,"end_s":33.12,"text":"I went on a mission to figure out if","speaker":null,"is_sponsor":0},{"start_s":30.48,"end_s":39.2,"text":"there's a way to mask their performance penalty while still getting the cost","speaker":null,"is_sponsor":0},{"start_s":35.68,"end_s":42.559,"text":"benefit to build the cheapest 100 TBTE","speaker":null,"is_sponsor":0},{"start_s":39.2,"end_s":44.879,"text":"storage box possible. Well, my original","speaker":null,"is_sponsor":0},{"start_s":42.559,"end_s":48.879,"text":"concept ended up totally not working. That's a new one, right? But I learned a","speaker":null,"is_sponsor":0},{"start_s":47.28,"end_s":60.239,"text":"bunch of interesting stuff in the process, and here it is.","speaker":null,"is_sponsor":0},{"start_s":60.239,"end_s":67.68,"text":"Cooler Master's Mastercase Maker 5 features their free form modular system,","speaker":null,"is_sponsor":0},{"start_s":65.119,"end_s":72.0,"text":"allowing you to customize, adjust, and upgrade. Make it yours at the link in","speaker":null,"is_sponsor":0},{"start_s":69.92,"end_s":78.08,"text":"the video description. Now, before I can explain why archive","speaker":null,"is_sponsor":0},{"start_s":74.96,"end_s":81.28,"text":"drives are so cheap and at the same time","speaker":null,"is_sponsor":0},{"start_s":78.08,"end_s":83.52,"text":"why their performance is less than ideal","speaker":null,"is_sponsor":0},{"start_s":81.28,"end_s":88.32,"text":"for certain applications, we need a little bit of background. Without","speaker":null,"is_sponsor":0},{"start_s":85.439,"end_s":94.079,"text":"getting into too much grimy detail, data is stored on hard drives by arranging","speaker":null,"is_sponsor":0},{"start_s":90.56,"end_s":96.96,"text":"the polarity of the tiny magnets that","speaker":null,"is_sponsor":0},{"start_s":94.079,"end_s":101.119,"text":"cover the hard discshaped thing inside called a platter according to the","speaker":null,"is_sponsor":0},{"start_s":99.2,"end_s":106.88,"text":"instructions given by your operating system. A magnetized bit is interpreted","speaker":null,"is_sponsor":0},{"start_s":104.079,"end_s":112.32,"text":"as a one and a non-magnetized bit is interpreted as a zero. So you lay down a","speaker":null,"is_sponsor":0},{"start_s":109.92,"end_s":116.72,"text":"few billion ones and zeros in the right order, read them back, and boom, next","speaker":null,"is_sponsor":0},{"start_s":114.56,"end_s":122.399,"text":"thing you know, you're playing Crisis 3. Okay, then. So, traditionally, these","speaker":null,"is_sponsor":0},{"start_s":119.04,"end_s":125.52,"text":"little magnets were arranged laying flat","speaker":null,"is_sponsor":0},{"start_s":122.399,"end_s":128.319,"text":"in concentric circles on the platter.","speaker":null,"is_sponsor":0},{"start_s":125.52,"end_s":134.8,"text":"This is called longitudinal magnetic recording. It's easier. But eventually,","speaker":null,"is_sponsor":0},{"start_s":131.599,"end_s":137.36,"text":"hard drive manufacturers ran out of room","speaker":null,"is_sponsor":0},{"start_s":134.8,"end_s":142.239,"text":"and couldn't increase capacity anymore without making their platters so big","speaker":null,"is_sponsor":0},{"start_s":139.84,"end_s":146.64,"text":"that the latency penalty of moving the read and write heads around would be too","speaker":null,"is_sponsor":0},{"start_s":144.64,"end_s":151.2,"text":"high. Not to mention that I'm pretty sure that no one wants a 10 terbte disc","speaker":null,"is_sponsor":0},{"start_s":149.12,"end_s":156.64,"text":"in their laptop if it has to be the size of a vinyl freaking record. So the first","speaker":null,"is_sponsor":0},{"start_s":153.84,"end_s":162.239,"text":"solution then was perpendicular magnetic recording. standing those magnets up","speaker":null,"is_sponsor":0},{"start_s":159.84,"end_s":166.879,"text":"instead of laying them down. This required more complex read and write","speaker":null,"is_sponsor":0},{"start_s":164.4,"end_s":171.04,"text":"heads, the uh the record needle type ARM that moves around and makes that ticking","speaker":null,"is_sponsor":0},{"start_s":169.519,"end_s":178.0,"text":"noise whenever your drive is working hard, but has gotten us all the way to","speaker":null,"is_sponsor":0},{"start_s":174.64,"end_s":179.68,"text":"10 terabytes so far with maybe a little","speaker":null,"is_sponsor":0},{"start_s":178.0,"end_s":185.12,"text":"bit more headroom left before the magnets again just can't get any","speaker":null,"is_sponsor":0},{"start_s":182.4,"end_s":190.64,"text":"smaller, which is where shingled magnetic recording comes in. Now the","speaker":null,"is_sponsor":0},{"start_s":188.159,"end_s":196.239,"text":"read component of the head, remember the record needle thing, is narrower than","speaker":null,"is_sponsor":0},{"start_s":193.44,"end_s":202.0,"text":"the right component. So by layering the magnetic tracks half on top of each","speaker":null,"is_sponsor":0},{"start_s":199.2,"end_s":207.36,"text":"other, like the shingles on a roof, much more data can be stored without moving","speaker":null,"is_sponsor":0},{"start_s":204.48,"end_s":212.879,"text":"to more exotic materials to make the magnets smaller or even drastically","speaker":null,"is_sponsor":0},{"start_s":209.92,"end_s":218.159,"text":"redesigning the heads. Unfortunately, this means that while you can read at","speaker":null,"is_sponsor":0},{"start_s":215.44,"end_s":222.56,"text":"pretty much full speed, the 8 TBTE archive drives that we used for our test","speaker":null,"is_sponsor":0},{"start_s":220.159,"end_s":226.959,"text":"are rated at 190 megabytes per second reads, way more than enough for the","speaker":null,"is_sponsor":0},{"start_s":225.12,"end_s":232.239,"text":"gigabit networks that most home and small office users are running. Write","speaker":null,"is_sponsor":0},{"start_s":229.36,"end_s":238.4,"text":"speeds can be devastatingly slow, especially when they're random. You see,","speaker":null,"is_sponsor":0},{"start_s":235.12,"end_s":241.36,"text":"the right head is so wide that it would","speaker":null,"is_sponsor":0},{"start_s":238.4,"end_s":247.439,"text":"actually overwrite both the intended track and the next one over on the","speaker":null,"is_sponsor":0},{"start_s":244.56,"end_s":252.159,"text":"drive. So, it has to read the data that it's going to accidentally overwrite.","speaker":null,"is_sponsor":0},{"start_s":249.92,"end_s":256.639,"text":"store that somewhere else, either in a solid state cache or in a reserved part","speaker":null,"is_sponsor":0},{"start_s":254.56,"end_s":261.44,"text":"of the disk platter somewhere else, organize it, and then finally","speaker":null,"is_sponsor":0},{"start_s":259.04,"end_s":266.0,"text":"sequentially write back both the data it's supposed to be writing in the first","speaker":null,"is_sponsor":0},{"start_s":262.96,"end_s":268.56,"text":"place and that data it had to shuffle.","speaker":null,"is_sponsor":0},{"start_s":266.0,"end_s":275.68,"text":"This is called a read modify write and it can be slow as all hell. So, let's","speaker":null,"is_sponsor":0},{"start_s":272.24,"end_s":278.32,"text":"talk then about my idea. I wanted to use","speaker":null,"is_sponsor":0},{"start_s":275.68,"end_s":285.36,"text":"the reasonable read speeds, the low cost, and the 247 operation ratings of","speaker":null,"is_sponsor":0},{"start_s":281.84,"end_s":288.479,"text":"archive drives in one of my lime unrade","speaker":null,"is_sponsor":0},{"start_s":285.36,"end_s":291.28,"text":"systems. I wanted to combine that with","speaker":null,"is_sponsor":0},{"start_s":288.479,"end_s":296.16,"text":"the reliability and all-around high performance of enterprise capacity","speaker":null,"is_sponsor":0},{"start_s":293.52,"end_s":301.52,"text":"drives to get the best of both worlds. So the way unrade works is that your","speaker":null,"is_sponsor":0},{"start_s":298.639,"end_s":305.12,"text":"data is actually written directly to the individual discs in the array which is","speaker":null,"is_sponsor":0},{"start_s":303.84,"end_s":309.6,"text":"great because in the event of a catastrophic failure let's say you lose","speaker":null,"is_sponsor":0},{"start_s":307.039,"end_s":314.08,"text":"two drives simultaneously at least anything written to the rest of the","speaker":null,"is_sponsor":0},{"start_s":310.96,"end_s":317.199,"text":"drives is still there and an additional","speaker":null,"is_sponsor":0},{"start_s":314.08,"end_s":320.32,"text":"drive or two drives acts as a parody","speaker":null,"is_sponsor":0},{"start_s":317.199,"end_s":322.24,"text":"disc that lets data from a single or two","speaker":null,"is_sponsor":0},{"start_s":320.32,"end_s":328.56,"text":"depending how many parody discs you have failed discs be rebuilt in the event of","speaker":null,"is_sponsor":0},{"start_s":325.12,"end_s":331.28,"text":"a less catastrophic failure. The problem","speaker":null,"is_sponsor":0},{"start_s":328.56,"end_s":337.039,"text":"is that while archive drives seem to be okay as standalone individual discs, the","speaker":null,"is_sponsor":0},{"start_s":334.4,"end_s":341.919,"text":"worst use case I could find for them was in parody protected RAID arrays with","speaker":null,"is_sponsor":0},{"start_s":339.919,"end_s":347.919,"text":"their poor random performance being pointed to as an unnecessary risk during","speaker":null,"is_sponsor":0},{"start_s":345.28,"end_s":352.96,"text":"a rebuild operation. So the data rebuilding process actually puts more","speaker":null,"is_sponsor":0},{"start_s":350.8,"end_s":358.32,"text":"strain than normal on the rest of the drives. And so the data across all the","speaker":null,"is_sponsor":0},{"start_s":355.759,"end_s":363.6,"text":"discs is in jeopardy until the corrupted or failed drives data has been rebuilt.","speaker":null,"is_sponsor":0},{"start_s":361.68,"end_s":369.44,"text":"So now we're 70% of the way through the video and we finally come to my idea. I","speaker":null,"is_sponsor":0},{"start_s":366.88,"end_s":375.759,"text":"figured by using archive drives in the array and an enterprise drive for parody","speaker":null,"is_sponsor":0},{"start_s":373.52,"end_s":381.68,"text":"and to replace any failed archive drives, I could mask both the poor","speaker":null,"is_sponsor":0},{"start_s":378.639,"end_s":385.28,"text":"random write performance and the slow","speaker":null,"is_sponsor":0},{"start_s":381.68,"end_s":387.199,"text":"rebuild times of the archive drives. And","speaker":null,"is_sponsor":0},{"start_s":385.28,"end_s":391.84,"text":"as you'll see from these performance numbers, it didn't work out that way at","speaker":null,"is_sponsor":0},{"start_s":389.52,"end_s":397.199,"text":"all. So uh my heterogeneous drive mixture configuration had worse","speaker":null,"is_sponsor":0},{"start_s":394.16,"end_s":401.039,"text":"performance than both all enterprise","speaker":null,"is_sponsor":0},{"start_s":397.199,"end_s":404.08,"text":"capacity drives which I expected and","speaker":null,"is_sponsor":0},{"start_s":401.039,"end_s":406.8,"text":"worse than a pure archive drive setup","speaker":null,"is_sponsor":0},{"start_s":404.08,"end_s":410.72,"text":"which I suspect is due to the mismatched disc spindle speed. So, that's kind of a","speaker":null,"is_sponsor":0},{"start_s":409.12,"end_s":415.6,"text":"drag, I guess. But there's some good news here for me anyway. And that is","speaker":null,"is_sponsor":0},{"start_s":413.12,"end_s":420.16,"text":"that in an unrade environment, I can either settle for 50 megabyte per second","speaker":null,"is_sponsor":0},{"start_s":418.319,"end_s":424.08,"text":"write speeds, about half of what a gigabit network can handle, in the","speaker":null,"is_sponsor":0},{"start_s":422.0,"end_s":428.0,"text":"default configuration where it spins up only the disc to which it's writing","speaker":null,"is_sponsor":0},{"start_s":425.759,"end_s":431.12,"text":"directly, and the parody disc to reduce power consumption and discare at the","speaker":null,"is_sponsor":0},{"start_s":429.52,"end_s":435.52,"text":"cost of performing read modify write operations all the time. Or if I use","speaker":null,"is_sponsor":0},{"start_s":433.199,"end_s":441.44,"text":"their turborite mode that spins all the discs during access, allowing for much","speaker":null,"is_sponsor":0},{"start_s":438.08,"end_s":443.759,"text":"faster reconstruct writes, I can still,","speaker":null,"is_sponsor":0},{"start_s":441.44,"end_s":448.72,"text":"even with the cheapest drives I could find that are rated for 247 operation,","speaker":null,"is_sponsor":0},{"start_s":446.56,"end_s":454.0,"text":"get my 100 megabytes per second since I'm not striping data the way that I","speaker":null,"is_sponsor":0},{"start_s":451.52,"end_s":458.56,"text":"would in a more traditional RAID, which to be clear, archive drives still are","speaker":null,"is_sponsor":0},{"start_s":456.56,"end_s":461.199,"text":"not recommended for. So, thanks for watching, guys. If this video sucked,","speaker":null,"is_sponsor":0},{"start_s":460.0,"end_s":465.759,"text":"you know what to do. But if it was awesome, get subscribed, hit that like button, or maybe even check out the link","speaker":null,"is_sponsor":0},{"start_s":464.479,"end_s":470.08,"text":"to where to buy the stuff that we featured at Amazon. In the video","speaker":null,"is_sponsor":0},{"start_s":467.759,"end_s":474.639,"text":"description, I have my full hard drive like NAS capacity and price calculator","speaker":null,"is_sponsor":0},{"start_s":472.639,"end_s":477.68,"text":"Excel sheet down there, which you can, you're more than welcome to try out.","speaker":null,"is_sponsor":0},{"start_s":476.319,"end_s":483.039,"text":"Also linked in the description is our merch store, which has cool shirts like this one, and our community forum, which","speaker":null,"is_sponsor":0},{"start_s":481.039,"end_s":486.72,"text":"you should totally join. Now that you're done doing all that stuff, you're probably wondering what to watch next.","speaker":null,"is_sponsor":0},{"start_s":485.28,"end_s":492.08,"text":"So, click that little button in the top right corner to check out our video from","speaker":null,"is_sponsor":0},{"start_s":489.52,"end_s":495.84,"text":"last year, which inspired a lot of this storage server stuff that I've been","speaker":null,"is_sponsor":0},{"start_s":493.52,"end_s":500.4,"text":"doing, where we lost pretty much all of our data","speaker":null,"is_sponsor":0},{"start_s":497.68,"end_s":502.8,"text":"temporarily. Or did I ruin the suspense? I don't know.","speaker":null,"is_sponsor":0}],"full_text":"Seagate archive drives. These things are freaking cheap for how much capacity you get. I can actually link my drive cost calculator spreadsheet that I used to make this chart under the video, by the way. But when I started looking into picking up some of these drives for our long-term storage NAS, I heard the performance totally sucked. So, I asked Seagate to send a few of them over, and I went on a mission to figure out if there's a way to mask their performance penalty while still getting the cost benefit to build the cheapest 100 TBTE storage box possible. Well, my original concept ended up totally not working. That's a new one, right? But I learned a bunch of interesting stuff in the process, and here it is. Cooler Master's Mastercase Maker 5 features their free form modular system, allowing you to customize, adjust, and upgrade. Make it yours at the link in the video description. Now, before I can explain why archive drives are so cheap and at the same time why their performance is less than ideal for certain applications, we need a little bit of background. Without getting into too much grimy detail, data is stored on hard drives by arranging the polarity of the tiny magnets that cover the hard discshaped thing inside called a platter according to the instructions given by your operating system. A magnetized bit is interpreted as a one and a non-magnetized bit is interpreted as a zero. So you lay down a few billion ones and zeros in the right order, read them back, and boom, next thing you know, you're playing Crisis 3. Okay, then. So, traditionally, these little magnets were arranged laying flat in concentric circles on the platter. This is called longitudinal magnetic recording. It's easier. But eventually, hard drive manufacturers ran out of room and couldn't increase capacity anymore without making their platters so big that the latency penalty of moving the read and write heads around would be too high. Not to mention that I'm pretty sure that no one wants a 10 terbte disc in their laptop if it has to be the size of a vinyl freaking record. So the first solution then was perpendicular magnetic recording. standing those magnets up instead of laying them down. This required more complex read and write heads, the uh the record needle type ARM that moves around and makes that ticking noise whenever your drive is working hard, but has gotten us all the way to 10 terabytes so far with maybe a little bit more headroom left before the magnets again just can't get any smaller, which is where shingled magnetic recording comes in. Now the read component of the head, remember the record needle thing, is narrower than the right component. So by layering the magnetic tracks half on top of each other, like the shingles on a roof, much more data can be stored without moving to more exotic materials to make the magnets smaller or even drastically redesigning the heads. Unfortunately, this means that while you can read at pretty much full speed, the 8 TBTE archive drives that we used for our test are rated at 190 megabytes per second reads, way more than enough for the gigabit networks that most home and small office users are running. Write speeds can be devastatingly slow, especially when they're random. You see, the right head is so wide that it would actually overwrite both the intended track and the next one over on the drive. So, it has to read the data that it's going to accidentally overwrite. store that somewhere else, either in a solid state cache or in a reserved part of the disk platter somewhere else, organize it, and then finally sequentially write back both the data it's supposed to be writing in the first place and that data it had to shuffle. This is called a read modify write and it can be slow as all hell. So, let's talk then about my idea. I wanted to use the reasonable read speeds, the low cost, and the 247 operation ratings of archive drives in one of my lime unrade systems. I wanted to combine that with the reliability and all-around high performance of enterprise capacity drives to get the best of both worlds. So the way unrade works is that your data is actually written directly to the individual discs in the array which is great because in the event of a catastrophic failure let's say you lose two drives simultaneously at least anything written to the rest of the drives is still there and an additional drive or two drives acts as a parody disc that lets data from a single or two depending how many parody discs you have failed discs be rebuilt in the event of a less catastrophic failure. The problem is that while archive drives seem to be okay as standalone individual discs, the worst use case I could find for them was in parody protected RAID arrays with their poor random performance being pointed to as an unnecessary risk during a rebuild operation. So the data rebuilding process actually puts more strain than normal on the rest of the drives. And so the data across all the discs is in jeopardy until the corrupted or failed drives data has been rebuilt. So now we're 70% of the way through the video and we finally come to my idea. I figured by using archive drives in the array and an enterprise drive for parody and to replace any failed archive drives, I could mask both the poor random write performance and the slow rebuild times of the archive drives. And as you'll see from these performance numbers, it didn't work out that way at all. So uh my heterogeneous drive mixture configuration had worse performance than both all enterprise capacity drives which I expected and worse than a pure archive drive setup which I suspect is due to the mismatched disc spindle speed. So, that's kind of a drag, I guess. But there's some good news here for me anyway. And that is that in an unrade environment, I can either settle for 50 megabyte per second write speeds, about half of what a gigabit network can handle, in the default configuration where it spins up only the disc to which it's writing directly, and the parody disc to reduce power consumption and discare at the cost of performing read modify write operations all the time. Or if I use their turborite mode that spins all the discs during access, allowing for much faster reconstruct writes, I can still, even with the cheapest drives I could find that are rated for 247 operation, get my 100 megabytes per second since I'm not striping data the way that I would in a more traditional RAID, which to be clear, archive drives still are not recommended for. So, thanks for watching, guys. If this video sucked, you know what to do. But if it was awesome, get subscribed, hit that like button, or maybe even check out the link to where to buy the stuff that we featured at Amazon. In the video description, I have my full hard drive like NAS capacity and price calculator Excel sheet down there, which you can, you're more than welcome to try out. Also linked in the description is our merch store, which has cool shirts like this one, and our community forum, which you should totally join. Now that you're done doing all that stuff, you're probably wondering what to watch next. So, click that little button in the top right corner to check out our video from last year, which inspired a lot of this storage server stuff that I've been doing, where we lost pretty much all of our data temporarily. Or did I ruin the suspense? I don't know."}