Most INSANE SSD RAID Setup – IT BOOTS!
Linus Tech Tips
·Linus Tech Tips
·2018-05-06
·
1,161 words · ~5 min read
0:00
one of the big stories at computex this year was about removing the bottlenecks
0:05
from high-speed m.2 NVMe ssds pretty
0:09
much every motherboard was bristling with mounts and partners like ASUS even
0:15
showed off this crazy card that could
0:18
hold four drives at a time for a
0:21
theoretical 16 gigabytes per second of
0:25
throughput so when samsung approached us to sponsor
0:29
a video showing off their flagship 960
0:32
pro we had plenty of inspiration for how
0:36
to build the awesomest dream SSD raid
0:39
setup unfortunately getting it to actually work
0:43
has been well an adventure yeah
0:48
let's put it that way
0:58
now as the de facto choice for speed freaks for the entire past year
1:03
samsung's 960 pro needs no introduction
1:06
but they sponsored this video so we're gonna do it anyway each of our four
1:10
drives has 512 gigs of vnan storage is
1:15
rated to a blazing fast 3 500 megabytes per second reads and 2100 megabytes per
1:20
second writes has a custom five core samsung processor on board and a
1:25
five-year warranty and somehow the whole thing is the size of a stick of gum
1:32
so then let's go let's plug them all in and
1:36
rip up some benchmark scores right wrong finding a board with enough slots
1:42
was the first challenge we had to solve i thought i saw one at computex that had
1:47
five but that turned out to be a figment of my imagination so we settled then on
1:52
the ASUS prime x299 deluxe with a
1:55
separate pci express card to handle the last m.2 but one small problem
2:01
as cool as it is to run Windows software raid for eye-watering sequential
2:06
performance numbers that has been done to death
2:10
and besides we wanted to actually
2:14
experience the speed in day-to-day use
2:18
so that means that we need to boot Windows from our array
2:23
let's talk about Intel's virtual radon CPU or vroc
2:29
since the latest skylakex high-end desktop processors share much of their
2:34
pedigree with server level xeon chips they actually have three what are called
2:40
volume management devices built in each of which can give up to four PCIe
2:46
4x devices direct access to the CPU for
2:50
high bandwidth low latency performance
2:53
cool the issue though is that today
2:57
it is not yet officially launched and making matters worse Intel is rumored to
3:03
be planning to enable the feature with a hardware key that will only be available
3:08
through bundles with x299 motherboards or ssds thankfully though AMD decided to
3:14
ride in on a white steed and save the day by announcing a free
3:20
driver update and sadly yes their
3:23
marketing materials specifically emphasize the free part that's the world
3:27
we live in now which would enable bootable NVMe raid on
3:31
the x 399 threadripper platform
3:35
so a quick swap to the ASUS zenith extreme and we are ready to
3:40
cry on the desk because AMD pulled the software almost
3:45
immediately after the announcement due to compatibility issues
3:50
once it was finally re-released we then had to bungle our way through the
3:54
incredibly poorly documented process of a loading the AMD NVMe controller driver
4:01
which allows b the AMD raid controller
4:04
driver to be installed which allows c the AMD raid configurator driver to be
4:10
installed then even when we were in Windows running on
4:15
four ssds we hit another roadblock
4:19
our iometer benchmark results were nowhere near what AMD posted on their
4:24
blog so after still more research
4:28
mad props to gary from ASUS by the way man
4:31
we figured out that because a threadripper CPU is technically two
4:37
separate dies linked by AMD's infinity
4:40
fabric interconnect considerations must be made to ensure
4:45
that the NVMe load is balanced between the dies
4:49
otherwise this 11 Gigabyte per second
4:52
link as fast as it is could actually become a bottleneck
4:57
so armed with a detailed diagram of PCIe
5:00
lane allocations and a pre-release BIOS we pushed onward meanwhile though we
5:06
actually hadn't given up entirely on v-rock
5:09
now while i wouldn't recommend getting your Intel drivers from russian download
5:14
mirrors we did find one that claimed to enable
5:18
v-rock without a raid key and actually
5:22
it technically worked but our performance numbers were way off
5:25
compared to the guys at pc perspective who also got v-rock working via what we
5:30
think was a similar method and anyway
5:33
neither of us could get it to boot with non-Intel ssds
5:38
so back to AMD threadripper then which for better or for worse has bootable raid
5:44
today AMD got back to us with a preset for
5:47
iometer and some guidance to help us replicate their results
5:51
not while actually booted from it mind you but with a raw as an unformatted
5:56
file system and we implemented everything meaning that it is finally
5:59
time to do this so after physically balancing the four
6:04
samsung 960 pros between the dies using
6:07
ASUS's dim.2 for two of them and their
6:10
hyper m.2 x 16 card for the other two we
6:14
booted in pure UEFI mode and configured our array using the built-in raid expert
6:19
2 utility we then f6 all three drivers in order
6:24
and saw our 2 terabyte array as available for os installation
6:29
once booted into Windows we launched diameter with one megabyte reads and
6:33
writes at 32q depth and had to rub our
6:36
eyes at the results 12 gigabytes per second on reads and 7.3 gigabytes per
6:43
second on rights and that is without AMD's bs raw file system stuff we are
6:49
talking a fully operational ntfs
6:53
formatted bootable array with those
6:56
kinds of numbers for comparison a single 960 pro
7:00
delivered three and a half gigabytes per second on reads and two gigabytes per
7:05
second on writes in the same machine now
7:08
crystaldiskmark didn't scale quite as well on reads but
7:12
we do have to keep in mind that this technology is still in its infancy at
7:17
least on the driver's side and we might actually see more of the raw hardware's
7:21
potential unlocked in the future so thanks to samsung for sponsoring this
7:26
exploration of high-end bootable raid on the desktop i don't think this crazy
7:32
ride is quite over yet but we're gonna take a little breather and then maybe
7:37
we'll revisit it once things have settled down particularly on the Intel
7:41
side over the next few months so thanks
7:44
you guys for watching if you disliked this video you can hit that button but if you liked it hit like get subscribed
7:49
maybe consider checking out where to buy the stuff we featured at the link in the
7:52
video description also down there is our merch store which has cool shirts like
7:56
this one as well as our community forum which you should totally join