when dream needed a server that could store unlimited gameplay footage with a
matching budget he knew there was only one place to go yo
and we're going to be building it in true dream fashion that's right we're
going to be speed running the creation of a 1.2 petabyte hard drive storage
server with a 200 terabyte all NVMe SSD
server all we need is a little bit of help
all right i think we're going for a world record here guys kyogsi has
sponsored the SSD so that's taken care of 45 drives practically built the
server for us and you guys are all here to witness it starting with the big face
reveal you ready okay it's more of a faceplate but look
it's a really cool one it has dream on
let's go jakey oh jesus oh server to build your voice oh this is a lot of
drives to shuck because the servers for dream some of the hardware did end up
getting sent to us but the hard drives were not one of those things meaning
that yes my friends dream actually did
shell out for 60 the full
60 20 terabyte hard drives that's going to
be a total of 1.2 petabytes of raw
storage which should work out to somewhere in the neighborhood of one
petabyte of usable storage when we're done here i think we're going to play
this one a little bit safer i'll put some hot spare so it might be more like 900 okay sure that's fair enough i don't
want to get a tech support call realistically from like wherever the
crap dream is from the server hardware itself was provided by 45 drives and
this is their turbo tier server it's got a 26 core xeon gold 512 gigs of ddr4
memory which we'll be using as a zfs read cache we've got four lsi hbas which
are allowing us to hook up all of these freaking drives and
we actually got them to send a dual port
25 gigabit per second network card which
um sorry 45 drives we're actually going to steal from this server and put in the
SSD server and there is a reason for this
the fancy switch hey i need more drives Jake hold on shut up world record the
fancy switch that ubiquity sent happens to have
two 25 gig ports so i'm thinking we put them together aggregate 50 gig to the
storage server wow it's probably excessive but we're gonna do it anyways honestly though
outside of our own server room he is getting the
finest storage setup out of probably
anyone else on youtube or twitch by the way Jake can i just shout you out for
putting these in exactly the correct orientation for me to just am i slot
them right in are you being sarcastic are you not trying to no no i was
actually no it's like the most ergonomic thing ever i just grabbed them and put
them in okay i'll be sure to do it the other way now oh you piece of crap
why did i say anything i was not trying at all it just happened to be how they
were getting pulled out that's it are we done
1.2 petabytes you know it's not that fun when it's
this easy what i mean it's i'm now i'm
still into it we haven't found out if it works yet so
we're replacing this dual 25 gigabit nick with this dual 10 gigabit nick and
the reason we're doing that is that i mean they're hard drives realistically
if we manage to get you know what 20 gigabit trunks so two gigabytes a second
out of it that'll be fine i mean it'll just it'll do a lot more than that but
you're just using this for archives so yes well yes and no i think it's possible
that they could end up editing partially off of it if they were working on a
project it'll have plenty for that called back from you know a previous
video or something along those lines come on
Jake what what was the ergonomic rationale behind
putting this server here i'm not saying you're wrong i'm just saying i don't see
how you're right are you complaining because you're short no
i'm complaining because tall people don't consider my short person needs
that's what i'm complaining about right right very important distinction
unlike the petabyte of hard drive based storage this one actually
is going to have a ton of room for expansion and that is thanks to
high density drives like the cm6 from
kyogre these puppies are available in up to what 15.3 terabytes of top this is a
30 terabyte what shut up it's not not only is it a 30 terabyte SSD it's a 30
terabyte tlc SSD it's not even qlc
these are crazy kiosks just sent these for dream where's mine well they sent 15
for dream and i was like yeah we'll just give them like seven oh really yeah it's
not like he paid for them yeah that still means he's gonna end up with 120
terabytes of usable space with two parity drives and a hot spare
wrap your brain around it right this is the state of the art i mean i
think they're looking at what 22 to 26 in the next year and those aren't even
conventional those are shingles exactly this has 50 more capacity than this
in a fraction of the size mind you it's probably
quite literally 20 times the cost not to mention this is PCIe gen 4
these will do almost 7 gigabytes a second read on a single drive
and that's why we're keeping some yes you should okay so we're gonna do a four
node actually wait the what oh my oh my
god oh my god how do you do that you
just literally holy
oh my god oh my god oh my god did i put
it back are you kidding me
dude what's the actual is wrong with you
oh my god oh oh i want to throw up i really
oh do you think it still works no
did you hear the noise in me well it's a pretty
it's a pretty bouncy floor okay the video's done i'm leaving
goodbye it might still work
okay lunch says it still works
you know it might actually still work here's the thing i've actually talked to
seagate engineers about the kind of drop testing that they do on these things
there's a solid chance that thing still works
what happened what nothing you talking no no i was just saying that uh you're a
fine young man and i'm really proud of the way you've turned out
i'm not proud of the way you've turned out this is pretty gorgeous this is the new
version of the one we have right yeah that's specifically designed for epic
milan so that's a 32 core 75 43p so
single single processor it boosts to 3.8
gigahertz i think so not quite as good as the 75 f3 yeah but that's pretty
freaking incredible this thing is gonna rip are you putting the RAM in are you
gonna talk about the RAM at all i'm just gonna put it in really we're speed running okay well it's gonna be 512 gigs
of RAM nope what 128 128 that's it they're eight gig dimms these are from
the one million dollar pc that we didn't use right microns sent us literally like
flats 96. flats
of these eight gig ecc 3200 modules
they're fast yep they're exactly what you want just not the capacity that you
want but for this for you it's perfect because we're not using RAM caching
since they're NVMe drives you don't need that for zfs but he's gonna get 128 gigs
anyway so we just want a nice fast CPU which for epic high speed memory is
really important and then we're basically not going to
hit this at all yeah i mean stuff will go through it but that's it that's
pretty much it 46 47 48 49 50 51 52 53
54 55 36 57 58 59
that's it shut up it does not there's only 59 yeah because it starts at zero
oh hold on there might be 60 actually hold on it starts at zero doesn't it it
might start at zero might be okay that it does are you screen recording there's
the moment yeah i'm not screen recording but it does start at zero there is 60.
that doesn't necessarily mean one of them isn't completely buggered okay
cooler time yeah wow this is a beefy cooler for a two years yeah the
nice thing about a 2u is you get double the cooler capacity which means
your fans get to spin a lot slower yeah which is going to be good for at your
house yeah one user like we're using a similar
machine to this at my place right yeah okay just a less cool one oh really so
dream's going to have a better server than me yeah i'm feeling pretty
inadequate right now i love these tool-less sleds man when i built the
first NVMe wanik it took frickin forever
and i didn't even put all the screws in oh these are so much easier to deal i
can't believe that every one of these is 30 freaking
terabytes i just installed 60 terabytes of SSD storage hold on here comes number
90 90. like freaking what i mean we do have
more we can put more they're dense yeah they're i was going to grab one of the
cd6s which are like the not enterprise ones um they are substantially heavier
these things are dense as here david catch
i don't know i know hold on a second no i want to i want to open it what no yeah
no yeah chill jesus just relax what's the worst that can happen i lose the
screws and we can't put it back together
it's got a multi pcb design oh my god
of course it does oh my god these caps though that's gonna be the difference
these are their actual enterprise capacity they're willing to say it's
enterprise this is for power loss protection these
capacitors ensure that if any data is
sitting in a cache or in a dram cache for example it can be flushed to nand
before the drive actually powers down even in the event of a sudden power loss
to the server that's freaking cool and those are
huge look at the little holes wait they have cooling for the caps
that's amazing wow i'm so happy right now that's so
much nan how much nand is that man hold on we gotta find more nand we're going
on a we're going on a voyage oh there's a clip there's a clip here no hold on
hold on oh my god this is so cool oh i
never get a chance to play with stuff like this god damn it
there's another layer hold on so they got thermal pads between
oh my god Jake i think it's three layers
Jake no stop please stop hold on please
please relay i can't do this i think it's three
legs on how do we get this layer my heart there it goes
holy balls it's got
four dram cash dies what about that
controller that thing is huge
hold on here it goes here it goes here it goes oh
ain't wait there's more
gas well that's the thing is the higher
capacity you go for your nan flash the
more dram cash you need and if you're gonna have 30 freaking terabytes demand
you're going to need a lot of dram eight gigabit one g by eight
so it's one gig so it'd be eight gigs i guess eight gigs of flippin RAM
this SSD has as much as most people's
gaming machines there's a new update i brought out the unify switch we were
kind of originally thinking we might build a whole rack and send it to him
but then apparently his like installers are doing a bunch of crap so we're just
gonna give him the switch yeah this switch has two 25 gig ports that we're
going to aggregate together to be 50 gig for NVMe server totally probably
overkill for what he's doing and then we're going to do two 10 gigs aggregated
together for 20 gig to the hard drives then all these other ports are 10 gig to
the rest of the house so that means if for whatever reason they expanded and
had like a whole team of editors there they could have five people concurrently
hitting this boy for 50 gigabits a second and this boy for 20 gigabits a
second that is a lot of throughput for game play and if you really wanted you
could switch this to a 4x10 card and then you could have 40 gig each editor
will get its own 10 gigabits it's pretty sick it's a nice little switch ah you
were gonna show me something cool there's a new view yeah hey look so they added this new port view so now instead
of having to edit a port one by one you can go hey i wanna edit that one that one that
one that one and maybe this one over here oh that's cool and you can set
bam i only want this now security-minded folks might put Mac Address like limits
so only that one Mac Address can connect to that port it's a nice security thing
you can also see what device is connected to a specific port a lot
easier so i can just click on it so i plug the bmc into that one port and then
i just click on that there's the ip here's a cool feature for ubiquity it would be
really nice to be able to take this go yes that's the device that i want and
just set a rule that only that Mac Address
can connect to that port just with one click here yeah if you just go like a
limit yeah lock Mac Address lock that would be cool i bet they'll do it
they're probably gonna watch this and they're probably going to do it yeah they might do it hey guess what it works
now i got a vga cable because i thought the ipmi was broken turns out they
screwed up i love you Gigabyte but they sent a server that doesn't support milan
chips no it's a roam server it's a roam server do
we have any roam chips to we do so we had a 75 or 70 402 p which is a 24 core
still going to be plenty for this yeah for the
cut down drive configuration that we put in here anyway yeah
okay it's gonna be great okay so now it'll actually be great yeah
we're in truenast now on both of them we're gonna be using trinas core as much
as i like scale core is a little more production ready yeah and we want this
to just work and him not to have to screw with it and this is actually
shockingly user friendly to use these days back in the early frenzy oh boy you
had to kind of be nerdy enough that you could have basically done it yourself in
Linux anyway now man this gui is so
pretty okay linem ti which one do you want to start
with i want to start with the one that definitely has a working drive in this
slot right here i will admit there are 60 functioning
drives in the server presently
i will not admit that i don't think or
that didn't make sense i will not admit that i don't think
i think that drive will die faster than the other ones
that's nice but that wasn't the bet
i'll buy you lunch whatever yes
why would you do that it's fine it's fine don't touch it yet it's still
spinning i'm letting it still spinning it's letting it spin wiggle wobbling
barely i can't believe you bought a petabyte i
can't believe you threw that hard drive can we just cut back to that for a second
holy crap i can't believe you did that okay
so we want some pretty chunk-tastic raid zenis i'm figuring maybe not
15-drive v-dabs like we do i was thinking to do 10-drive z2s or i mean
for like performance apparently the optimal is six drives but that's a lot
of capacity loss yeah we'd have to do z1s at that point and i really don't
want to do that so raid z2 is kind of like raid 6 where you can
lose two of your drives before you actually lose any data but it means that
two drives out of whatever the v dev size you have are lost in terms of
capacity so raid z1
sets aside only one of those drives for parity data but that means you can only
sustain one loss before your next loss is going to cost you data so for dream
you're thinking 10 well hold on i just realized we want to have some hot spares
we don't want ten why don't we do this why don't we do four hot spares one for
each bank and then and then we can do seven drive v dams
upgrade z2 we're gonna lose a lot of capacity
why don't we go 14 drive v devs then
raid z3 so that would allow three drives lost
per one of these rows sure with a spare okay so if we do raid
z3s 14 drives we get 800 terabytes usable i'm about to blow your mind how
much is each of these drives worth uh like 600 six six hundred that's like six
hundred bucks six hundred dollars okay let's do some quick math real quick here
okay six hundred twelve hundred twenty four hundred okay
five thousand ten thousand ten thousand
dollars worth of drives is sitting doing nothing there will be no data actually
stored on them okay they're just holding data just in case these ones fail
or standing there ready to go in case these fail 10 grand that level of
redundancy costs in a server like this look at all this
big data big data you're going to set it to be an smb share
we're gonna go into advance i'm gonna do z standard compression because it's
really great record size of one mag because it's all
footage bam now we can store stuff big
data so it works out to be 740 tera tibi
bytes usable after formatting that's a big data all right let's do the NVMe
we're going to do it pretty similar to the petabyte server where we have a hot spare but we're just doing one and it's
a raid z2 which means we have two disks of parity that is so overkill it is with
a hot spirit do you want to get a phone call hold on the problem with ssds hear
me out the problem with ssds they're not like hard drives where they kind of
staggered fail usually when you have like 20 ssds and they're all from the
same batch they'll just start failing at the same time because they're so much more dependent on like right endurance
it's better to be safer he's got 100 tv
bites that's so much that's that's more than we have i know but this is like
90 terabytes of SSD storage just
not being used well it's being used Jake i think it's fine raid z1 and a hot
spare i know their enterprise drives he's barely
going to touch these compared to the workloads they're designed for you're
you know i'm looking around i see wendell like through the camera
hey there mr wilson it's me Linus the menace
well Jake and i are having a bit of a debate right now we've got seven of
these 30 terabyte keyoxia drives and i'm not going to tell you who is in which
position so that you're unbiased but there are two potential configurations
for them in in true nas here um are you willing to weigh in on on which one you
think is proper one of us wants to do
a raid z2 with a hot spare and one of us
wants to do a raid z1 with a hot spare i
would probably do everything honestly
a little bit more it's a little bit more dangerous
um but those drives are so fast
and if you're on top of it it's probably okay but what if they're
not on top of it if you're not on top of it then you gotta go
do you really think it's likely that we're gonna hit these drives hard enough
that we should be concerned about their lifespan if you get a hot spare on the
flash side i would probably just make it raise d2 instead of having a hot
spare only because flash wears differently than mechanical hard drives now okay so
what about a raid z2 and a hot spare oh my god Jake okay all right let's just go
raid z2 then with seven drives we need to put some footage on this thing nice
it's going through my workstation so it's not going to be that fast wow even
through your workstation it's doing 400 plus megabytes a second
you know what i should go jump onto mine
and hit it at the same time and see if that even goes down i bet it won't
the bottleneck is going to be your your station not this machine okay well
i'm still copying at 400 to 450 megabytes a second how about
you okay you ready for me to press go oh you haven't pressed go yet okay i
already did yeah that's what i figured
yeah the one thing to consider is because we're copying like a folder full
of a bunch of little files Windows will never really get there because it's like
starting stopping starting zapping starting stopping but that's still super
awesome this is really important just because we have
redundancy okay so we have that extra
couple of drives that can fail doesn't mean that we have backup
so if this is where they're going to store their active projects then if
something were to go wrong let's say uh a power supply blue i mean
theoretically those are also redundant so you could just you
could just yank one in the middle of operation but let's say it blew in such
a catastrophic way that it lit on fire okay the server is dead but somehow the
fire doesn't spread outside the steel chassis and light the other server on
fire we need a backup that backup is gonna go automatically
from this server to this one unfortunately eating up some of our
capacity but not that much it's gonna be incremental so but each time it backs up
it's it's keeping what's already on there and just adding the new stuff yes
and it will cycle automatically so we can set it so it keeps like a daily
backup uh for two weeks and it automatically deletes the old ones when
it makes the new ones and doesn't store a bunch on here
and then as soon as they delete something from here that will eventually
fade out of those daily or weekly backups until the only copy that exists
is the archival copy which like us dream has said is not
super essential it's a nice to have not a need to have because you can always go
re-download it off of youtube if it comes to that that's your off-site
backup essentially thanks youtube okay i
started the backup it's backing up from this server
over to this server and it's running at 400 megabytes a second i mean that's a
lot of data you could probably get it to go faster
by disabling like secure ciphers on ssh but right now we just have it in the
default configuration just to show that it works and since by default this is
only going to be a daily backup it's got 24 hours to move whatever is on here
that's plenty enough if you guys enjoyed this video check out kyoccia's products
they've got everything from enterprise to consumer grade SSD products we're
gonna have them linked down below also major shout out to Gigabyte who sent
over the wrong server but they did send a server so kudos for that micron for
sending over the RAM AMD for providing the CPU that we couldn't use thanks
Gigabyte um and also 45 drives for
sending over this epic storinator that
happens to color match the green tabs on oh wow and with the dream front plate
they can do like literally anything custom you want yeah it looks amazing
if you guys enjoyed this video go check out uh i don't know one of the other
times Jake and i worked on a server thing he'll find something petabyte of flash yeah
oh yeah we need to get back going we gotta finish that but Jake is too busy
helping me with my house he moved in early he's an in-demand kind of boy