oh is this muronic this is not i know that della's new pneumonic yes well no
the dell is you called it new pneumonic
confirmed maybe i'm just a sucker for punishment but after this thing kicked
my ass for over a year when i was trying to deploy it as our main video editing
nas at the office it's back baby
new new one wanted to but this time i'm
gonna be deploying it at home thanks to our friends at keoxia who sponsored this
video sending over that's right the Intel drives are out and these keoxia
cd6 drives are in they sent over 12 of
their 4 terabyte cd6s which is gonna make this the ballinest naz on the block
not four eight eight they sent over eight terabyte drives
well seven point six i think holy this
is gonna be like a hundred terabyte NVMe nest
for my house how did i get this out of the office
the first thing we're going to need to do today is downgrade this server because as
because as much as i would have loved to take off with 256 gigs of ecc ddr4 memory i think Jake
would have crapped a brick when all of a sudden he goes to like perform
maintenance on a server upgrade it and all of our high spec memory is gone
so let's get all that out of here and while we're at it our 32 core epic 7502p
that's gonna have to go too the good news though is that the lowest end chip
that we had to replace it with sitting in the office was a 7402p
which is still really freaking fast and 24 course i think yeah
nice epic rome is such an amazing platform
because even the lowest end chips still have 128 lanes of pci express gen 4.
that means that even if i populated all 24 bays on this chassis could have
access to the full bandwidth of a pci express gen 4x4 link now on this
particular board four of my bays are gonna be running at pci express gen
three speed i think it might be the ones connected here yeah it's this card right
here that's in a gen three slot but that's not a limitation of the CPU these
things are absolutely nothing fuzz honestly by the time we populated all 24
of those bays with gen 4 NVMe drives we would run into memory bend with
bottlenecks just trying to read or write to them so it's a non-issue for us
especially because i'm not even using 3200 megahertz memory which is the
maximum spec for epic rom i ended up with these just about the most pinner
ddr4 ecc that you can get these are
eight gig 2666 modules they're from
crucial so i expect them to be reliable but they are not going to be fast next
order of business finally is an actual upgrade one of the things that my server
needs to do that most storage servers wouldn't is transcode video on the fly
i've been pretty upfront about the fact that i have an extensive blu-ray collection but i consider the act of
grabbing a physical disk and putting it into a drive to be a very last decade
kind of thing to do so in spite of the fact that ripping your own blu-rays is
kind of a legal gray area i consider it to be morally okay and besides like what
am i what am i gonna do have a bunch of blu-ray players all over the house like
physically go and get the disc to put in the like come on all of it is going to be
served from here but because not every device is capable of playing back a full
quality blu-ray that's where your GPU comes in so this
is just a pretty basic gtx 1050 from MSI
that happens to be low profile so it'll fit in our case and this is gonna handle
down scaling our original like 4k HDR or
whatever to something that's a little more
smartphone friendly or whatever it is that you need of course in order to get it in here
we're going to have to do a little bit of reconjiguity with step one being yep
removing the fans now this might seem like a bad idea but
given that we're installing this card in a server and these heatsink fins are
actually oriented the correct way i would expect this card to run just fine
even without the fans on it so we're going to go ahead and pop it in here and
obviously we'll find out if we're wrong about that later next we've got to
upgrade the onboard networking because believe it or not this server only comes
with gigabit Ethernet but that's perfectly normal because while gigabit
could ship it with onboard 10 gig for example
anyone rolling a server like this is going to be running 100 gig 200 gig or
even higher in order to take advantage of the insane speeds of the drives that
you can hook up to it so those 10 gig chip sets would be
completely wasted cost because they wouldn't be used even for us this 10 gig
card is going to be a placeholder because we still don't know exactly what
kind of switch or realistically switches we're gonna use so we haven't bothered
sourcing any fiber optic transceivers yet now one quirk of this particular
motherboard is that this top slot is labeled optional in the manual and as
far as we can tell in its current configuration it is not working we think
that's because this mezzanine slot here takes the bandwidth that would otherwise
be allocated to it so four of my slots at the front of the
case are actually not going to be usable so i'm just going to electrical tape the
connectors here and tuck them away in the chassis but that's okay because
remember how i said that if we actually populated all 24 of these bays with the
drives that we're using we'd run into memory bandwidth bottlenecks before we
actually were able to use up the full speed well that's true so there's no point
filling all the bays anyway of course what we end up using depends on what we
decide to do with the storage in this server because the sky is the freaking
limit now you know blu-rays are 50 gigs they're big but that is nothing for an
NVMe 1.4 compliant drive that is capable
of throughput in excess of six gigabytes
per second so we could do everything
from use it as a storage server which obviously i'll do and i can stream my
plex movies to as many devices as i freaking want but i could do more than
that for example all of the computers in the house instead of actually having
drives installed on them they could all network boot off of this array so that
all of their storage is safe and centrally stored in one place that'd be
really cool because not only are the drives built to an enterprise standard
that means full power loss protection a variety of different security options
available and depending on whether you go for the regular cd6 series or the
read intensive cd6 series anywhere from three to one drive full full drive rate
per day endurance but because we're going to be running them in a zfs array we get access to all
kinds of cool zfs features like for example the ability to really quickly
and easily create snapshots say if i wanted to back up some or all of my data
to a server at the office i could totally do that or file system level
compression which would allow me to effectively stretch the i think it's
approximately like 90 something terabytes of raw flash storage that's a
lot but hey if you can have more and it costs you literally nothing then
that's definitely the way to go you ready
let's power this puppy on these blowing metron fans they're gonna
go just give them a second
there they go see that's why we weren't worried about
taking the fans off of the GPU because they're going there's your GPU
fan look at this baller with this like 100
builds bridge wall actually no but we should mention lttstore.com hey got this
new CPU reflective design a new lanyard whoa new colors of lanyards
ltteststore.com while the cli side of things might sound a little daunting
fortunately for people like Linus you can get zfs on Linux really easily with
the ui plus it's actually available as a package in the community app store so
all we gotta do is search up unread community apps and you can copy the url paste it into unraid in
the plugin installer give that a few seconds
literally a few seconds it's already done yep and then bam you have a new
apps tab now there's a lot of really cool
docker stuff there's a lot of really cool you know unread plugins in here but
for us all we got to do zfs return
hit install and now we have zfs
just that easy it's downloaded still actually
give it a second
okay it's done because of the limitation of unraid we still do need to have an
array so we just grabbed a couple of old SATA drives that are like i wouldn't put
anything important on them they have reallocated nan flash already like
they're just there to exist yeah according to the unraid guys they're
actually gonna fix that in a few versions from now give them you know a
couple months and yeah theoretically this should be a lot easier and there's
going to be a ui for zfs as well so which would be sweet yeah but zfs is
pretty simple actually if we if we go into our command prompt we can just type
zfs list see no datasets z pool list no data
sets we have to make our data set yeah are we just going to do a single v dev
so that's where zfs can get a little complicated for some people how do you
want to configure it i'm probably just going to go single vw i'm thinking a
single v dev with raid z2 so that would give us two parity drives yeah yeah so
we're gonna lose 15 terabytes or whatever one parity drive is probably
fine you think so literally like data
center drives and i'm gonna back this up to the office anyway
fine fine okay we're going to call our z-pool lambo because cause it's
so we're gonna go down to raid z1 which means one parody drive yeah and mount
slash lambo is where it's gonna be stored
if you don't know what it means to have one parity drive it means that one of
these 12 drives could outright fail which is pretty unlikely and all the
data would still be completely intact in fact the speed of the array would
probably be not even degraded to the point where it would be affected you
probably wouldn't notice there we go 83.8 terabytes raw so sex horrific but
after parody it'll be a little bit less than that you get 72 terabytes that's
crazy so that's a lot of sports yeah what was my old one okay while he does
that we're gonna set up a few other things so one of the nice things about
zfs is we can create multiple data sets
within our pool of storage and those are useful for a number of reasons primarily
that we can define different settings for different data sets they're almost
like folders but i can say my movie folder doesn't get compression because
video doesn't compress very well but then i could say my vm storage folder or
data set in this example does get compression so my old on raid server was
64 terabytes of spinning hard drives and
i had uh two terabytes in raid one as a cache oh
that's pretty nice okay there we go so those are all our data sets
now we have to do the not so fun part of moving all of the stock on raid things
over to the zfs bolts some of it we could probably leave on those crappy
drives like isos come on you could but now that we've already made the thing we
might as well just change it it's not actually that hard we just have to go in
to vm manager turn it off
you know what the most fun part of building something totally overkill like
this is trying to find a use for it because you
end up no i'm serious though you end up exploring all these cool new use cases
that you would never have had any reason to explore yeah like the just the idea
of having all the storage for all the computers in the house just on this one
why would anyone do that once we figure out how to do it you don't have to do it
with such crazy overkill hardware yeah well that's a whole other video so get
subscribed so you don't miss that cool so now we should see some usage on
our z pool used six megabytes nice
zfs is not really designed for NVMe
storage it works but there are some caveats and things you kind of have to
do to make sure it plays nicely the people that develop zfs never had in
mind that your storage was going to be as fast as the memory on the system like
storage now is probably faster than cash was on a CPU when zfs was created so
there's a few things we need to do for one the arc cache in zfs which is really
great for accelerating hard drives we're going to set that to be metadata only so
it doesn't store any actual files on it just the metadata of the files if you
use arc with NVMe it's probably going to hurt your array especially at this speed
these drives are what you might use as an arc cache
on a hard drive array yeah yeah so level one arc would be RAM but like Jake was
saying even RAM is not yeah it doesn't that much faster enough another thing
that's important we have to enable auto trim i think a lot of the times now zfs
is smart enough to do it on itself but we're going to do it just in case if the
drive isn't trimmed there's a lot of wasted extra rights that can happen it's
called right amplification with every right so you want to keep your house in
order effectively we're also going to disable access time which is something
you might use for very specific use cases for us doesn't matter at all
that's going to save us some reading rights you know what i'm just going to set compression on
across the entire array because the nice thing about lz4 compression on unraid is
if it's got a big file and it's not compressing it will just give up so
chances are it's not really going to hurt our performance at all if there's
problems down the road we can set it to off for say the plex library and then on
for the vms like i said before yeah i still can't believe that i ended up with
what was supposed to be new new wanak at the office no new wanted not new new new
monique no just oh is this nuonic this is new and you want it yes no the dell
is you called it new pneumonic
confirmed oh cool do you see him yeah it's just
everything's there zfs dockers efs media system copy a file over there shall we
you got a big file i got a big file i got a 100 gig screen cap that i
accidentally forgot to stop recording so here you go there's a there's your big
big media file oh i need permission ah oh yes i got a
whoops so normally this would be managed through the unread gui but well it would
it would just work by default but mount slash lambo i think we have the
tone nobody users
let's just let's just say everything in this folder sure cool that's right all
right try again
hey there it goes not bad wow man what a
big improvement this is gonna be oh well
there's your limiting factor it's probably smb actually yeah
so we're it looks like we're single thread limited on here either way this
is way faster than you have right now we
should find out how much performance we still have left on the table here just
by doing a quick this benchmark oh i've got about a minute left transferring
these files and then you can go ahead and benchmark it what if i just do it right now anyways sure
i mean i guess that's what we want to know right i mean we're writing at like 15 to
17 gigabytes a second is that fast enough for you i don't even feel it you
don't even feel it yeah well it went down to like 10 12
12 14. that's literally a hundred times
faster than what i have now 100 times
i'm gonna need 100 gigabit networking at home well it's a little late we're not going
to do that we could do that we're not going to do that 25 should we do that
now obviously 10 to 15 gigabytes a second is a lot less than the sum of the
speed of all of these drives in a normal data center environment you might be
able to get a lot closer to raw speed from these drives depending on how
they're deployed somewhere in the neighborhood of more like 60 to 70
gigabytes a second one of the big challenges though one of the best
benchmarks of how good a plex server is is of course whether you can enable
subtitles growing uniquely i'm on the outdoor access point now we're playing
original quality with subtitles baked in so i'm just gonna leave this here
right epic gives zero f's about this workload
we could conceivably have a dozen people doing the same thing at the same time
basically and the best part of all of this zero funky behavior between these drives
and this server match freaking made in heaven which leaves no not that one
i was gonna pull the one that had the nice cd6 logo on it anyway Jake is doing
the demo where we show that video playback is completely uninterrupted
even by the complete loss of one of our drives which is very unlikely to happen
there we go look unavailable degraded but it works look it did have a
little hiccup but hiccup get it i get it but look now
they're working again i think i might be in love just like i love telling you
about our sponsor cheoxia you can get all the details on their cd6 series
enterprise drives at the link in the video description but the main things
you need to know high quality nand lightning fast pci express gen 4
interface and of course their long time
proven track record for delivering reliable enterprise grade drives if you
guys are looking for another video to watch you can maybe check out the last time we tried to deploy this server with
someone else's drives when it didn't it did not go well
that's a bit of an understatement