come here guys come on this
is 1.2
petabytes and the last time we built a
petabyte storage solution it had to be spread out over multiple servers not
anymore with seagate's new 20 terabyte
exos x20s we are going to be building a
single for you server that can hold over
a petabyte of raw data it's going to be
flipping crazy and truthfully
it's hard to be that excited because the real reason we're building this is
because of a data loss in our previous petabyte project that is making it so
that we have to build a new server to copy all of the data off that server
while we attempt to recover it so I'm going to be having fun but in a
all of this is very sad and stressful kind of way what I'm never sad about is
telling you about our sponsor thanks to Signal RGB for sponsoring this video
with signal RGB you can control and sync your favorite RGB devices all from one
app best of all it's free download signal RGB in the link below
all right I mean it's mind-blowing if
you think about it 20 terabytes of storage
in a single Drive the original wonex server which was hard drive based by the
way had about 20 terabytes of storage except that it was an entire 4u chassis
like this one full of one terabyte drives Man One X server was 20 terabytes
up until like two years ago one freaking drive and the craziest part is last time
we did a petabyte project we wanted to get a full petabyte of addressable
storage into a single chassis but we fell short because I wanted a 75 Drive
version of the chassis so it would be actually five of these rows but there
was a bit of a snafu and we didn't end up getting it done the crazy thing is
since then we've gone from 16 terabytes maximum capacity to 20. so we are going
to have exactly what we were targeting except in 60 Bays instead of 75. now I
couldn't help noticing Jake that none of the hardware is in this well
it's kind of a long story does it involve these bins of Hardware
over here yes yes so in there is actually the original Hardware if you
look there so I think it's a 26 core Xeon
gold I think 512 gigs of RAM it would
have had four hbas in there at one point but in between the process of This
Server being built and it arriving to us I kind of decided I wanted our cash
drives to be NVMe not SATA and unfortunately there's not really a lot
of PCIe on here there's probably enough we probably could have put like some
Jank four splitter bifurcation thing in
there but uh I figured we should go epic instead
we're going epic ladies and gentlemen need that PCIe it's kind of crazy how
like as an Enthusiast or a gamer you don't really think about that as AMD's
advantage over Intel in the server space you kind of think of it in terms of huge
performance yeah or core count I've heard that's important eight channel
memory right like these are all things that as Gamers we understand but 128
PCIe Lanes when you're plugging this kind of shiz into your CPU matters a lot
yeah what we ended up with is actually the xl60s turbo variant so it's got a
little bit bigger CPU than the base one but if you wanted like us you can
probably ask 45 drives and they'd maybe make you an epic system it's just not
something that they show on their website you probably don't need to as
long as you're just running mechanical drives but because we want to put a
bunch of NVMe drives in we do need the extra PCIe now at this stage we've got a
couple of options we can work on things together or we can divide and conquer
but the one thing I'm not flexible on is we need to take this thing and put it on
one of the those carts because if we load it up with hard drives and we have
to move it again I'm gonna lose my I already brought it
downstairs it was fine no no but full of drives yeah brought downstairs full
drives also it's better to carry it than put it on a car cart's bath it was IBM
study look it up you gotta have that was across a like
parking parking still there's lots of bumps and stuff oh
it's gonna be fine look I get it you're kind of a small guy
you know it's okay it's gonna be okay if you don't want to carry it we got other
people for that you know you employ like 50 people
it's not it's not saying anything it's just putting drives on I think he's given up should we maybe wait hold on
hold on should we oh now he doesn't want it to be heavy oh no I'm sorry Jake I'm
sorry Jake is this not how you wanted the hard drives to go in oh oh oh shoot
oh man oh I'm sorry I think we should put the motherboard in and make sure
that's working oh they just keep going in there they just slide in so nice
love 45 drives mechanism look at this 20 40 60 80 100 terabytes just like that oh
yeah but let's actually talk a little bit about the drives that Seagate sent
over these are their Data Center Enterprise grade drives they run at 7200
RPM they've got 256 megabytes of cash
each and a SATA 6 gigabit per second interface they're going to actually be
pretty similar to the 20 terabyte ironwolf Pro prosumer NAS Drives in
terms of performance but these have double the rated mean time between
failure and almost double the rated annual workload so they're just designed
to run a little heavier they're also 20 more expensive at MSRP but a funny thing
that we've noticed in the past is that these EXO strives often end up being
less expensive once they've been on the market for a little bit so these ones
have only been at retail for about a month month and a half so the cheapest
that we could find these specific drives was 571 dollars each from a Newegg
Marketplace seller that puts this at wait
35 000 worth of drives
and the funny thing is they're actually going to send four morphs we have some cold spares so that would put us like
37.50 that's a good thing to have with any storage setup that has more than I
mean realistically a handful of drives realistically we should have a hot spare
Mechanical Devices do fail it's something that happens and it's
something that you have to be ready for holy crap that's sexy this is what this
is like one of my favorite epic boards for sure seven PCIe 16x slots is that a USB
type-c on a server board yeah it's also it's all Gen 4 too we originally got
this board for the sensible jellyfish fryer but I think they originally sent
the OG epic version yeah the non-gen
four one yeah and then this is the Gen 4 one that does Rome and Milan sorry
Yvonne I'm filing the papers this is my new wife
what happens when the new one comes out well then uh more papers off with the
old on with the new Jesus
what kind of Epic are we putting in here okay so it's not a Milan because we
still can't get any of those we have two we finally got two this is not one of
them this is a 7402p it's the previous
generation Rome chip it's 24 cores which is probably excess for this honestly but
it should be pretty good once it's paired up with the 512 gigs of RAM we take from the old board more funny
things if you don't have a CPU Cooler for this that fits in there
we have this one it's a u12 it's a little big we'll just run it hot rod for
now yeah I think we'll just leave the top off can we just take a moment and
talk about how the Molex connector has apparently been doing roids have you
seen these let's plug it back into the right one what is this
saw the boat if I just blew right now no
you wouldn't I wouldn't ruin both of our days so I'm not gonna do
that just a quick
just telling me what a great job I did you mean like what a great job I did
well someone did a great job whose name's on the channel again
okay hey look you strong or you started
it today you 100 started petabytes yourself damn it darn
it done I'm through you started it oh god oh no what
um okay so to use all the ports we want to use we have to move a jumper oh no
so right now they're both on the right two and three yeah two and three two and
three so that's where we want it actually perfect well that was easy I
don't know if I believe that though that's fine we'll find out soon enough worst comes to worst we just have to
switch it to the other side okay because then it would be on one and two one and two okay easy we're hoping that that's
right okay now epic has no built-in raid
functionality so fortunately true Nas has a built-in redundancy function so
that you can just install the OS simultaneously to both drives and it's
not raid but it's a backup which is
better is that a fast mirror is it is oh I didn't even know it was using cool
this does not feel fine oh my God totally fine can you imagine though you
know I cannot imagine I don't think this is built to be tipped like this okay
yeah everything's fine horrifying so this is cool these connectors right here
are super cool they can be used for SAS SATA or PCI Express so it can cause some
confusion because I thought they were u.2 which would be PCIe like ND for NVMe
drives but actually they are are they
so we're just going to connect this little breakout cable and bippity
boppity we'll plug these into our drives here and we're good to go that's cool it
passes through the state of power too you see that yeah
so we actually do need four because story about our NVMe drives they're not
here yet in fact not even two of the
cables we need are here yet the oculink wherever that is to u.2 cables are not
here yet they're supposed to arrive today maybe they're just City and Logistics I don't know anyways
yeah so we're gonna do two eight terabyte SATA drives that are from
Micron and we're gonna do two one terabyte nvmes those are going to be our
intent log for CFS which is almost kind of like a right cache not really and
then our level 2 Arc which is just an extended read cache that adds on to the
RAM cache that ZFS uses by default why didn't you take the
well they are they're there for in case
they're there in case get it this is the case and they were in
it Andy likes it Andy Andy likes everything you pay him
to like it I pay you to like it well I'm just about just one of you is better at
their job oh yeah
put her in the first slot there bud there it goes
oh yeah look at all that PCIe baby we still got two more well one more slot
that works yeah wow you mangled this cooler what the heck I did nothing to
that cooler you know I didn't do anything to that
cooler look at that part I didn't this
is brand new was it really yeah what happened I don't know probably when you
were fumbling it out probably both of us messed it up a bit I'm sure we might
have a big problem Jake I don't know if I can plug in the front i o it's right
angle on this board oh who cares well I can't get it in well who cares okay we
could do it I have a new idea we just take the pins and we just bend them up I
love it okay I'll be back did you just pop that out for no reason no I didn't
know the screw wasn't in yet what do you think I was doing picking my
nose like I mean usually you're not doing too much so
really Mr stands there and we hit it we hit a
cord here boy walks up and unplugs what
I'm working on I did the math and lttstore.com is a great deal check
it out we've got this new hoodie I will be by the time this video is up I think
sure you can finish your thought I don't even remember what I was saying doing
the math oh I did the math so at one Gigabyte a second which I think is a
little uh optimistic optimistic for the old vaults especially in a degraded
state it would take seven and a half days if it sustained that the whole time
to move 700 terabytes 500 megabytes a second they'll take 15.
I want to hear you say it it's gonna take a month you're supposed
to tell me I was right you said it was going to take months I thought it was
going to take everything yeah to do it three times three times twice I don't
want to have new Vault stuff on here I want to move it off and then move it back is that like is that unreasonable
I don't know but I think we're ready to fire this thing up really we ready
are you ready what's the management poured into it I see are you ready wait
see what you were saying you plugged it into management yeah that's what you
wanted to do right yeah okay yeah I wanted to plug the crappier one in time
it was just stupid
smells like fire that's not
is it still postcoding I mean it's a server so like yeah it
could take a minute as rock boards are sort of middle of the road as far as that goes though right
speed I think it's usually pretty quick yeah it's probably got to do memory
training though for 512 gigs of RAM so
oh it's booty nice 170 seconds it's been going
hey look look 63 I think that's right oh wait no
that's not right we're missing some enemy meat so we got our two nvmes
oh the SATA drive 6.99 terabyte yeah
and then we're missing two of these
yeah theoretical bananas
oh I bet you it's those two you suck at
this you put those in this is your fault you distracted me
that those were very out oh I can feel
them they're they're doing stuff now okay I don't know why this boy is not
working though okay 65 we're at 65. okay we have all our hard drives now we're
just missing okay so tell me if that's the bad one I don't know yes we could
have checked the cereal I actually know if it'll oh yeah it could also be that
this cable is bad these cables suck I've seen like tons of them fail another one
of them serial number mismatch it says
so it did pick up the drive then it says SATA link is three gigabit per second oh
no it picked it up Micron 4300 okay try it plug the other one in for a second
yeah that cable might be bunk oh it couldn't be the cable line this
must be a bad Micron Enterprise Drive yeah for sure
you know with all the crap that has taken place
that did not make it into videos it's probably just as well the Vault gets
white once in a while we back up look at that 66 baby nice so
we got 900 900 those are two NVMe's let me just
switch this to 100. we got 6.99 6.99
and the rest okay now we have to make a pool okay let's do it
uh Salve Oasis sure let's call it Oasis it's the data Oasis
Good gravy you have to select them one by one yeah I mean think of how often
champagne problem I think you can also hold control I'm just you know whatever
raid Z2 gives us two Drive failures per
V Dev before we actually sustain any permanent data loss which
from our recent Adventure we found is
not necessarily the be-all and end-all but
because it doesn't save you from bit rot well look trunes sets up a scrub for all
Pools by default monthly yeah it just does it automatically so s had new such
functionality can't screw this up you can but
yeah all right anyways um
fine I want to listen to them initializing formatting
they're already initialized the server's on what do you mean you're not doing anything oh they're going now
that sounds terrifying I know it's normal
we are doing data things wow these are ones are getting kind of warm actually
maybe we should put the top on yeah we probably should hey look at that okay so we have
available 895 Tibby bites
984 terabytes okay so should we remake
the array it just raids E1 so you can see more than a petabyte of usable no
it's okay rape Z2 is bare minimum for this minute
we still have 1.2 petabytes of raw storage plus 32 terabytes of NVMe once
it's done so one point of clarification each V Dev so each grouping of 15 drives
is running in raid Z2 so that means we
could actually theoretically lose up to eight drives before sustaining any kind
of permanent data loss but you'd have to get really lucky I'm going to call it
Vault three okay new new Vault no I'm not doing it I have
power in this manner and it's Vault
three I don't care one day I'm gonna come in on a weekend and just change it
all you know network interface look these
two are 100 gigs sweet one thing that's really smart to
do on your storage service especially one with this many drives is you can
actually set up a scheduled smart task like on all of your drives do a quick
smart test like once a month and you can usually tell like if a drive is going to
die yeah just based on that it'll like email you the results that's one nice
thing about mechanical drives is they tend to spit out some warnings before
they actually completely get up again it's not always though yeah
anything in here that's bad I don't hope
so spam log pickle Linus
how's it going pickle one I was making a
thumbnail for one of my just like random troll streams on Floatplane so I just
put that up there I killed him
oh there we go there it is 8.95 chibi bites I think
anyways the moral of the story is holy sh that's a lot of storage is this
a holy yeah it could be now it's time for us to drop it in the server
room and start a weeks long data transfer it's probably gonna be like two
weeks in a use case where you need more iops
that is operations per second maybe if you're video editing or you have a
database running on this you might want to do a setup that has more vdas because
typically it means you're gonna have better iops performance in our case
basically everything going to this is sequential AF we're moving like 500 to a
terabyte of footage of just straight footage so it doesn't matter for us if
anything we just want more capacity while still being safe and this is our
our Middle Ground actually our it's not really middle it's like on our side
ground you know what I mean thanks to
privacy.com for sponsoring this video Valentine's shopping is well underway
and privacy.com is here to help with the growth of technology and ever increasing
online fraud keeping your banking information safe and secure is becoming
more and more of a concern and privacy.com lets you shop online even at
unfamiliar stores with virtual credit cards that offer way more security and
control than conventional cards you never have to worry about being billed
twice getting upgraded to another service without your consent or having
to change your card information in the event of a hack privacy.com also allows
you to set spend limits so that you can take control of your finances and stick
to your gift budgets their free service offers you up to 12 cards and by
upgrading to Pro you'll be able to get 36 cards more security features and even
one percent cash back as a special bonus if you sign up today you'll get five
bucks it's five bucks so check it out today at privacy.com
forward slash Linus that's privacy.com forward slash Linus if you guys like
this video maybe go watch the video on the original petabyte project uh it was
a cluster we actually needed a gluster cluster two different machines in order
to reach this same capacity it's come a long way