{"video_id":"lzzavO5a4OQ","title":"Most INSANE SSD RAID Setup – IT BOOTS!","channel":"Linus Tech Tips","show":"Linus Tech Tips","published_at":"2018-05-06T14:53:29Z","duration_s":482,"segments":[{"start_s":0.4,"end_s":9.519,"text":"one of the big stories at computex this year was about removing the bottlenecks","speaker":null,"is_sponsor":0},{"start_s":5.6,"end_s":12.24,"text":"from high-speed m.2 NVMe ssds pretty","speaker":null,"is_sponsor":0},{"start_s":9.519,"end_s":18.56,"text":"much every motherboard was bristling with mounts and partners like ASUS even","speaker":null,"is_sponsor":0},{"start_s":15.28,"end_s":21.84,"text":"showed off this crazy card that could","speaker":null,"is_sponsor":0},{"start_s":18.56,"end_s":25.439,"text":"hold four drives at a time for a","speaker":null,"is_sponsor":0},{"start_s":21.84,"end_s":26.88,"text":"theoretical 16 gigabytes per second of","speaker":null,"is_sponsor":0},{"start_s":25.439,"end_s":32.64,"text":"throughput so when samsung approached us to sponsor","speaker":null,"is_sponsor":0},{"start_s":29.439,"end_s":36.0,"text":"a video showing off their flagship 960","speaker":null,"is_sponsor":0},{"start_s":32.64,"end_s":39.52,"text":"pro we had plenty of inspiration for how","speaker":null,"is_sponsor":0},{"start_s":36.0,"end_s":42.32,"text":"to build the awesomest dream SSD raid","speaker":null,"is_sponsor":0},{"start_s":39.52,"end_s":44.96,"text":"setup unfortunately getting it to actually work","speaker":null,"is_sponsor":0},{"start_s":43.68,"end_s":50.86,"text":"has been well an adventure yeah","speaker":null,"is_sponsor":0},{"start_s":48.48,"end_s":55.84,"text":"let's put it that way","speaker":null,"is_sponsor":0},{"start_s":58.96,"end_s":66.479,"text":"now as the de facto choice for speed freaks for the entire past year","speaker":null,"is_sponsor":0},{"start_s":63.039,"end_s":68.08,"text":"samsung's 960 pro needs no introduction","speaker":null,"is_sponsor":0},{"start_s":66.479,"end_s":75.04,"text":"but they sponsored this video so we're gonna do it anyway each of our four","speaker":null,"is_sponsor":0},{"start_s":70.88,"end_s":77.84,"text":"drives has 512 gigs of vnan storage is","speaker":null,"is_sponsor":0},{"start_s":75.04,"end_s":83.04,"text":"rated to a blazing fast 3 500 megabytes per second reads and 2100 megabytes per","speaker":null,"is_sponsor":0},{"start_s":80.479,"end_s":88.479,"text":"second writes has a custom five core samsung processor on board and a","speaker":null,"is_sponsor":0},{"start_s":85.84,"end_s":93.04,"text":"five-year warranty and somehow the whole thing is the size of a stick of gum","speaker":null,"is_sponsor":0},{"start_s":92.24,"end_s":98.84,"text":"so then let's go let's plug them all in and","speaker":null,"is_sponsor":0},{"start_s":96.32,"end_s":104.96,"text":"rip up some benchmark scores right wrong finding a board with enough slots","speaker":null,"is_sponsor":0},{"start_s":102.32,"end_s":109.84,"text":"was the first challenge we had to solve i thought i saw one at computex that had","speaker":null,"is_sponsor":0},{"start_s":107.6,"end_s":115.6,"text":"five but that turned out to be a figment of my imagination so we settled then on","speaker":null,"is_sponsor":0},{"start_s":112.399,"end_s":118.079,"text":"the ASUS prime x299 deluxe with a","speaker":null,"is_sponsor":0},{"start_s":115.6,"end_s":124.56,"text":"separate pci express card to handle the last m.2 but one small problem","speaker":null,"is_sponsor":0},{"start_s":121.68,"end_s":129.52,"text":"as cool as it is to run Windows software raid for eye-watering sequential","speaker":null,"is_sponsor":0},{"start_s":126.64,"end_s":134.12,"text":"performance numbers that has been done to death","speaker":null,"is_sponsor":0},{"start_s":130.879,"end_s":138.16,"text":"and besides we wanted to actually","speaker":null,"is_sponsor":0},{"start_s":134.12,"end_s":141.04,"text":"experience the speed in day-to-day use","speaker":null,"is_sponsor":0},{"start_s":138.16,"end_s":146.879,"text":"so that means that we need to boot Windows from our array","speaker":null,"is_sponsor":0},{"start_s":143.92,"end_s":152.16,"text":"let's talk about Intel's virtual radon CPU or vroc","speaker":null,"is_sponsor":0},{"start_s":149.84,"end_s":157.28,"text":"since the latest skylakex high-end desktop processors share much of their","speaker":null,"is_sponsor":0},{"start_s":154.4,"end_s":163.04,"text":"pedigree with server level xeon chips they actually have three what are called","speaker":null,"is_sponsor":0},{"start_s":160.239,"end_s":170.4,"text":"volume management devices built in each of which can give up to four PCIe","speaker":null,"is_sponsor":0},{"start_s":166.48,"end_s":173.76,"text":"4x devices direct access to the CPU for","speaker":null,"is_sponsor":0},{"start_s":170.4,"end_s":174.64,"text":"high bandwidth low latency performance","speaker":null,"is_sponsor":0},{"start_s":173.76,"end_s":180.56,"text":"cool the issue though is that today","speaker":null,"is_sponsor":0},{"start_s":177.84,"end_s":185.44,"text":"it is not yet officially launched and making matters worse Intel is rumored to","speaker":null,"is_sponsor":0},{"start_s":183.28,"end_s":191.04,"text":"be planning to enable the feature with a hardware key that will only be available","speaker":null,"is_sponsor":0},{"start_s":188.08,"end_s":197.36,"text":"through bundles with x299 motherboards or ssds thankfully though AMD decided to","speaker":null,"is_sponsor":0},{"start_s":194.959,"end_s":203.2,"text":"ride in on a white steed and save the day by announcing a free","speaker":null,"is_sponsor":0},{"start_s":200.0,"end_s":205.04,"text":"driver update and sadly yes their","speaker":null,"is_sponsor":0},{"start_s":203.2,"end_s":208.56,"text":"marketing materials specifically emphasize the free part that's the world","speaker":null,"is_sponsor":0},{"start_s":207.2,"end_s":215.28,"text":"we live in now which would enable bootable NVMe raid on","speaker":null,"is_sponsor":0},{"start_s":211.599,"end_s":218.159,"text":"the x 399 threadripper platform","speaker":null,"is_sponsor":0},{"start_s":215.28,"end_s":222.959,"text":"so a quick swap to the ASUS zenith extreme and we are ready to","speaker":null,"is_sponsor":0},{"start_s":220.72,"end_s":227.84,"text":"cry on the desk because AMD pulled the software almost","speaker":null,"is_sponsor":0},{"start_s":225.84,"end_s":233.12,"text":"immediately after the announcement due to compatibility issues","speaker":null,"is_sponsor":0},{"start_s":230.64,"end_s":237.68,"text":"once it was finally re-released we then had to bungle our way through the","speaker":null,"is_sponsor":0},{"start_s":234.879,"end_s":244.4,"text":"incredibly poorly documented process of a loading the AMD NVMe controller driver","speaker":null,"is_sponsor":0},{"start_s":241.12,"end_s":247.2,"text":"which allows b the AMD raid controller","speaker":null,"is_sponsor":0},{"start_s":244.4,"end_s":252.0,"text":"driver to be installed which allows c the AMD raid configurator driver to be","speaker":null,"is_sponsor":0},{"start_s":250.48,"end_s":259.68,"text":"installed then even when we were in Windows running on","speaker":null,"is_sponsor":0},{"start_s":255.599,"end_s":262.079,"text":"four ssds we hit another roadblock","speaker":null,"is_sponsor":0},{"start_s":259.68,"end_s":265.919,"text":"our iometer benchmark results were nowhere near what AMD posted on their","speaker":null,"is_sponsor":0},{"start_s":264.88,"end_s":270.96,"text":"blog so after still more research","speaker":null,"is_sponsor":0},{"start_s":268.8,"end_s":273.84,"text":"mad props to gary from ASUS by the way man","speaker":null,"is_sponsor":0},{"start_s":271.84,"end_s":280.72,"text":"we figured out that because a threadripper CPU is technically two","speaker":null,"is_sponsor":0},{"start_s":277.04,"end_s":282.56,"text":"separate dies linked by AMD's infinity","speaker":null,"is_sponsor":0},{"start_s":280.72,"end_s":287.919,"text":"fabric interconnect considerations must be made to ensure","speaker":null,"is_sponsor":0},{"start_s":285.12,"end_s":292.4,"text":"that the NVMe load is balanced between the dies","speaker":null,"is_sponsor":0},{"start_s":289.199,"end_s":295.12,"text":"otherwise this 11 Gigabyte per second","speaker":null,"is_sponsor":0},{"start_s":292.4,"end_s":297.919,"text":"link as fast as it is could actually become a bottleneck","speaker":null,"is_sponsor":0},{"start_s":297.04,"end_s":303.6,"text":"so armed with a detailed diagram of PCIe","speaker":null,"is_sponsor":0},{"start_s":300.639,"end_s":308.16,"text":"lane allocations and a pre-release BIOS we pushed onward meanwhile though we","speaker":null,"is_sponsor":0},{"start_s":306.08,"end_s":312.24,"text":"actually hadn't given up entirely on v-rock","speaker":null,"is_sponsor":0},{"start_s":309.44,"end_s":316.08,"text":"now while i wouldn't recommend getting your Intel drivers from russian download","speaker":null,"is_sponsor":0},{"start_s":314.96,"end_s":322.0,"text":"mirrors we did find one that claimed to enable","speaker":null,"is_sponsor":0},{"start_s":318.8,"end_s":323.6,"text":"v-rock without a raid key and actually","speaker":null,"is_sponsor":0},{"start_s":322.0,"end_s":327.759,"text":"it technically worked but our performance numbers were way off","speaker":null,"is_sponsor":0},{"start_s":325.52,"end_s":333.36,"text":"compared to the guys at pc perspective who also got v-rock working via what we","speaker":null,"is_sponsor":0},{"start_s":330.08,"end_s":336.0,"text":"think was a similar method and anyway","speaker":null,"is_sponsor":0},{"start_s":333.36,"end_s":339.68,"text":"neither of us could get it to boot with non-Intel ssds","speaker":null,"is_sponsor":0},{"start_s":338.72,"end_s":345.6,"text":"so back to AMD threadripper then which for better or for worse has bootable raid","speaker":null,"is_sponsor":0},{"start_s":344.32,"end_s":349.759,"text":"today AMD got back to us with a preset for","speaker":null,"is_sponsor":0},{"start_s":347.6,"end_s":353.36,"text":"iometer and some guidance to help us replicate their results","speaker":null,"is_sponsor":0},{"start_s":351.6,"end_s":357.84,"text":"not while actually booted from it mind you but with a raw as an unformatted","speaker":null,"is_sponsor":0},{"start_s":356.0,"end_s":361.919,"text":"file system and we implemented everything meaning that it is finally","speaker":null,"is_sponsor":0},{"start_s":359.759,"end_s":367.919,"text":"time to do this so after physically balancing the four","speaker":null,"is_sponsor":0},{"start_s":364.639,"end_s":370.96,"text":"samsung 960 pros between the dies using","speaker":null,"is_sponsor":0},{"start_s":367.919,"end_s":374.319,"text":"ASUS's dim.2 for two of them and their","speaker":null,"is_sponsor":0},{"start_s":370.96,"end_s":377.28,"text":"hyper m.2 x 16 card for the other two we","speaker":null,"is_sponsor":0},{"start_s":374.319,"end_s":380.639,"text":"booted in pure UEFI mode and configured our array using the built-in raid expert","speaker":null,"is_sponsor":0},{"start_s":379.36,"end_s":387.12,"text":"2 utility we then f6 all three drivers in order","speaker":null,"is_sponsor":0},{"start_s":384.479,"end_s":391.6,"text":"and saw our 2 terabyte array as available for os installation","speaker":null,"is_sponsor":0},{"start_s":389.919,"end_s":396.96,"text":"once booted into Windows we launched diameter with one megabyte reads and","speaker":null,"is_sponsor":0},{"start_s":393.759,"end_s":399.84,"text":"writes at 32q depth and had to rub our","speaker":null,"is_sponsor":0},{"start_s":396.96,"end_s":405.68,"text":"eyes at the results 12 gigabytes per second on reads and 7.3 gigabytes per","speaker":null,"is_sponsor":0},{"start_s":403.199,"end_s":413.12,"text":"second on rights and that is without AMD's bs raw file system stuff we are","speaker":null,"is_sponsor":0},{"start_s":409.44,"end_s":416.56,"text":"talking a fully operational ntfs","speaker":null,"is_sponsor":0},{"start_s":413.12,"end_s":418.16,"text":"formatted bootable array with those","speaker":null,"is_sponsor":0},{"start_s":416.56,"end_s":422.8,"text":"kinds of numbers for comparison a single 960 pro","speaker":null,"is_sponsor":0},{"start_s":420.72,"end_s":428.319,"text":"delivered three and a half gigabytes per second on reads and two gigabytes per","speaker":null,"is_sponsor":0},{"start_s":425.28,"end_s":430.319,"text":"second on writes in the same machine now","speaker":null,"is_sponsor":0},{"start_s":428.319,"end_s":434.08,"text":"crystaldiskmark didn't scale quite as well on reads but","speaker":null,"is_sponsor":0},{"start_s":432.08,"end_s":439.28,"text":"we do have to keep in mind that this technology is still in its infancy at","speaker":null,"is_sponsor":0},{"start_s":437.039,"end_s":444.8,"text":"least on the driver's side and we might actually see more of the raw hardware's","speaker":null,"is_sponsor":0},{"start_s":441.84,"end_s":449.68,"text":"potential unlocked in the future so thanks to samsung for sponsoring this","speaker":null,"is_sponsor":0},{"start_s":446.88,"end_s":455.44,"text":"exploration of high-end bootable raid on the desktop i don't think this crazy","speaker":null,"is_sponsor":0},{"start_s":452.639,"end_s":459.36,"text":"ride is quite over yet but we're gonna take a little breather and then maybe","speaker":null,"is_sponsor":0},{"start_s":457.199,"end_s":464.8,"text":"we'll revisit it once things have settled down particularly on the Intel","speaker":null,"is_sponsor":0},{"start_s":461.52,"end_s":466.16,"text":"side over the next few months so thanks","speaker":null,"is_sponsor":0},{"start_s":464.8,"end_s":471.12,"text":"you guys for watching if you disliked this video you can hit that button but if you liked it hit like get subscribed","speaker":null,"is_sponsor":0},{"start_s":469.68,"end_s":475.12,"text":"maybe consider checking out where to buy the stuff we featured at the link in the","speaker":null,"is_sponsor":0},{"start_s":472.96,"end_s":478.72,"text":"video description also down there is our merch store which has cool shirts like","speaker":null,"is_sponsor":0},{"start_s":476.479,"end_s":482.68,"text":"this one as well as our community forum which you should totally join","speaker":null,"is_sponsor":0}],"full_text":"one of the big stories at computex this year was about removing the bottlenecks from high-speed m.2 NVMe ssds pretty much every motherboard was bristling with mounts and partners like ASUS even showed off this crazy card that could hold four drives at a time for a theoretical 16 gigabytes per second of throughput so when samsung approached us to sponsor a video showing off their flagship 960 pro we had plenty of inspiration for how to build the awesomest dream SSD raid setup unfortunately getting it to actually work has been well an adventure yeah let's put it that way now as the de facto choice for speed freaks for the entire past year samsung's 960 pro needs no introduction but they sponsored this video so we're gonna do it anyway each of our four drives has 512 gigs of vnan storage is rated to a blazing fast 3 500 megabytes per second reads and 2100 megabytes per second writes has a custom five core samsung processor on board and a five-year warranty and somehow the whole thing is the size of a stick of gum so then let's go let's plug them all in and rip up some benchmark scores right wrong finding a board with enough slots was the first challenge we had to solve i thought i saw one at computex that had five but that turned out to be a figment of my imagination so we settled then on the ASUS prime x299 deluxe with a separate pci express card to handle the last m.2 but one small problem as cool as it is to run Windows software raid for eye-watering sequential performance numbers that has been done to death and besides we wanted to actually experience the speed in day-to-day use so that means that we need to boot Windows from our array let's talk about Intel's virtual radon CPU or vroc since the latest skylakex high-end desktop processors share much of their pedigree with server level xeon chips they actually have three what are called volume management devices built in each of which can give up to four PCIe 4x devices direct access to the CPU for high bandwidth low latency performance cool the issue though is that today it is not yet officially launched and making matters worse Intel is rumored to be planning to enable the feature with a hardware key that will only be available through bundles with x299 motherboards or ssds thankfully though AMD decided to ride in on a white steed and save the day by announcing a free driver update and sadly yes their marketing materials specifically emphasize the free part that's the world we live in now which would enable bootable NVMe raid on the x 399 threadripper platform so a quick swap to the ASUS zenith extreme and we are ready to cry on the desk because AMD pulled the software almost immediately after the announcement due to compatibility issues once it was finally re-released we then had to bungle our way through the incredibly poorly documented process of a loading the AMD NVMe controller driver which allows b the AMD raid controller driver to be installed which allows c the AMD raid configurator driver to be installed then even when we were in Windows running on four ssds we hit another roadblock our iometer benchmark results were nowhere near what AMD posted on their blog so after still more research mad props to gary from ASUS by the way man we figured out that because a threadripper CPU is technically two separate dies linked by AMD's infinity fabric interconnect considerations must be made to ensure that the NVMe load is balanced between the dies otherwise this 11 Gigabyte per second link as fast as it is could actually become a bottleneck so armed with a detailed diagram of PCIe lane allocations and a pre-release BIOS we pushed onward meanwhile though we actually hadn't given up entirely on v-rock now while i wouldn't recommend getting your Intel drivers from russian download mirrors we did find one that claimed to enable v-rock without a raid key and actually it technically worked but our performance numbers were way off compared to the guys at pc perspective who also got v-rock working via what we think was a similar method and anyway neither of us could get it to boot with non-Intel ssds so back to AMD threadripper then which for better or for worse has bootable raid today AMD got back to us with a preset for iometer and some guidance to help us replicate their results not while actually booted from it mind you but with a raw as an unformatted file system and we implemented everything meaning that it is finally time to do this so after physically balancing the four samsung 960 pros between the dies using ASUS's dim.2 for two of them and their hyper m.2 x 16 card for the other two we booted in pure UEFI mode and configured our array using the built-in raid expert 2 utility we then f6 all three drivers in order and saw our 2 terabyte array as available for os installation once booted into Windows we launched diameter with one megabyte reads and writes at 32q depth and had to rub our eyes at the results 12 gigabytes per second on reads and 7.3 gigabytes per second on rights and that is without AMD's bs raw file system stuff we are talking a fully operational ntfs formatted bootable array with those kinds of numbers for comparison a single 960 pro delivered three and a half gigabytes per second on reads and two gigabytes per second on writes in the same machine now crystaldiskmark didn't scale quite as well on reads but we do have to keep in mind that this technology is still in its infancy at least on the driver's side and we might actually see more of the raw hardware's potential unlocked in the future so thanks to samsung for sponsoring this exploration of high-end bootable raid on the desktop i don't think this crazy ride is quite over yet but we're gonna take a little breather and then maybe we'll revisit it once things have settled down particularly on the Intel side over the next few months so thanks you guys for watching if you disliked this video you can hit that button but if you liked it hit like get subscribed maybe consider checking out where to buy the stuff we featured at the link in the video description also down there is our merch store which has cool shirts like this one as well as our community forum which you should totally join"}