it's actually kind of amazing how many wires got crossed on this project but we
are finally doing it we are building
petabyte project number two
and not a moment too soon we may not even have enough space left on our
servers to offload the footage that we are recording right now and Linus you
might say you could just stop being such a digital hoarder and oh i don't know
delete some bloody data but
i actually have the perfect counter argument to that
you sound like my wife just let me have my fun
and we're gonna have some fun today ladies and gentlemen because um i
accidentally have over three petabytes of hard drives
smart deploy makes it easy to handle daily it tasks like Windows imaging
patching updating apps and migrating user data you can do it all over your
existing network or the cloud without leaving your desk get your free offer at
smartdeploy.com Linus
alright so the first problem was entirely my fault actually i told
seagate that our goal was to show off one petabyte of usable space in a single
4u enclosure instead of doing it in two enclosures like we did last time and i
told them that to do that i would need 75 of their 16 terabyte hard drives to
account for the space that we'd lose to formatting overhead and parity data
so that's true in five zfs raid z2 arrays we would be
able to lose up to two drives per v dev so that's up
to a maximum of 10 of our 75 drives before we would
actually lose any data and that would
still yield over 950 terabytes of accessible space one
small problem though the custom 75 drive chassis that i
thought i asked 45 drives for they were like yeah yeah there bud your
server's in the mailbox you're welcome bud they're eastern canadian they really
do sound like that it's amazing and i was like so is it the 75 drive custom
one and they're like what are you talking about there bud i apparently
never clarified i needed 75 bays so it has
60. so it looks like we're going to have about
750 terabytes of usable space but hold on hold on guys the title is not
clickbait i am still going to have one petabyte of raw capacity in here the
difference is that we're going to make up some of that shortfall with solid
state all right so let's take a look at the drives that seagate sent over here
so there's actually more boxes here than i expected
which is interesting so we're all we're all learning things
today um this can go
here so these are the ones that are right these
are the ones that we can do first this
is so hilarious oh my god there's so many of them that this is actually like
like this will actually build up pretty high
i'm not quite sure how it happened but this was error number two seagate's iron
wolf nas drives are designed for network attached storage use the respect for a
million hours mean time between failure and 180 terabytes of access per year
they've got a three year warranty and they feature seagate's agile array a
combination of hardware and firmware features that make them perform better
in raid arrays they've got rv sensors and better vibration tolerance to
improve performance and reliability in multi-drive arrays and a combination of
solid performance and power consumption across a wide variety of workloads
including video editing which is our primary concern around here we actually
end up editing video directly off of the vault more often than you'd probably
think because whether it's a big project and wanik's server has no room or
because a sponsor wants to change after the fact or whatever the case may be so
thank you seagate appreciate you fam these are great drives and we've
recommended them loads of times except for one small problem i really don't
know where the communication wires got crossed but these
are rated for use in enclosures of up to
eight drives at a time okay then
so seagate then sent over a few boxes of
their iron wolf pro drives putting us up to a total of over two
petabytes of storage but those are also only meant to have up
to 24 drives in an enclosure i mean
honestly speaking i would have been perfectly comfortable with the ironwolf
pros they've got an extra two years of warranty compared to the regular
ironwolf they've got included data rescue service and they've got a greater
rating for both their per year use and mean time between failure but thing is
we're supposed to be setting a good example for you guys and when i
clarified hey guys so the plan is actually to put all the drives into one
system they sent over the big dogs
meet the exos 16 in its top current capacity
of 16 terabytes
each of these is rated for a massive
550 terabytes per year of access and two
and a half million hours meantime between failure with all of the
vibration sensing and mitigation technology at seagate's disposal to rate
them then for an unlimited number of drives per enclosure
oh let's add them to the pile right
i have never seen this much storage in one place in my
life this is over three
raw petabytes of storage
225 drives times 16 terabytes each the bad
news is seagate says that i have to use the earlier shipments of our drives for
other stuff or send them back so make sure you're subscribed so you don't miss
some more nas building collabs with other youtubers that's one of the ideas
that i had anyway let's use this opportunity to take a
closer look at our enclosure now i am wicked excited about this server
so this is an early model this is a prototype of
their next generation storinator
there we go
now from the outside this looks like a regular old plane storinator with 45
drives typical all sheet metal construction all that
good stuff but
oh that's different
they have stepped up their game so they actually went from a cable-based
backplane system where every single port was individually wired in to these pcb
backplanes so this dramatically simplifies the cabling since they're
just running one quad port sas cable for each of the four bays and it also should
theoretically improve reliability they've also put in some logic to
stagger the spin-ups of the drive so you don't get that same kind of power surge
when you first turn on a storinator and all up to 60 drives are like
start like ramping up very cool
i'm not ready to actually build this thing up yet though because
there is another surprise now i don't know how much of this came
about because of my request or how much they were working on already
but what's this then 45 drives
has finally joined team red
that's right so we've got an AMD epic
processor in here i'm actually not a hundred percent sure exactly what the
model number is and then it's equipped with
what are we looking at here just shy of
128 gigs of RAM now there was a bit of a
mishap on our unit and this is like an engineering sample board the Gigabyte
provided so i couldn't really get a new one two of these memory slots are dead
but this is a hard drive based storage
system so i'm not actually too worried about the extra couple of channels of
memory killing our system performance like it did in our all NVMe nas
with that said i did allude to needing some ssds in order to make up the
difference in capacity between the 60 drive one and the 75 drive one that i
thought 45 drives was working on that is
where these come in and i need to actually find out what the devil they are
all right here we go so there's a feature of zfs called adaptive
replacement cache or arc and essentially
what it does is it takes the most frequently used data from your hard
drives and then stores a second copy on your system memory so that you don't
have to go all the way out to your spinning disks in order to access it and
that's especially important for something like running vms or a database
where a lot of the lookups are going to be to the same few entries we're already
using that on the existing petabyte project you know the one with the two
bays what we are not using is something
called l2r or level two arc thing is
sure you can add hundreds of gigabytes or even terabytes of memory to a system
now but the cost can be quite prohibitive so that
is where ssds come into play so this right here is a 7.68
terabyte SSD so nearly 8 terabyte SSD
from micron that's yeah it's only SATA 6
gigabit per second but even though that's obviously a lot slower than
system memory it's much faster than spinning disks and easily enough to
saturate our 10 gigabit or even a 40 gigabit network connection and then
importantly it is much cheaper than just chucking more RAM into the system
now something to bear in mind here is that
we ended up putting six of these in here in order to get our
raw capacity up to the petabyte that we promised but l2 arc
actually does not scale especially well with a ton of capacity
so it's possible that what we'll end up doing is only using some of it cheating
a little bit but for the time being hey we got a petabyte of capacity also
just like arc it only has a copy of the data on it so it doesn't actually count
towards your total capacity but these are just minor details like you know
which way the drive goes in for
example i know i'm gonna get judged so hard for this but
while i could put a dual 10 gigabit mezzanine cart in right here in this
open gap in the back i have actually decided
to use one of our ancient connectx 2 40
gig infiniband cards because i had a slot that didn't have a
cover on it and i don't know what else to do with this ancient thing anyway and
it's i mean it's 40 gig even though it's an older card it's plenty for
hard drives so that's in there with its 3d printed
bracket from that video a long time ago this is a really nice feature so i've
actually done this the janky way just like hot gluing or double sided taping a
fan on top of my hba cards so these are
the controller cards for all these drives we're going to plug in but hey
now there's a you know an officially sanctioned way to do it
sweet so we've got a cooling fan to take care of all of our add-in cards down
here very nice wow now that they've got so much space here they could definitely
modify the chassis and do a third fan here if they really wanted to that's
freaking awesome uh this is an example of a really old storynator this is from
about five years ago you can see there's a lot of oh there's a lot of things
about it that are a lot worse so this horrible horrible mounting system
for the drives with like these rods holding the thing
down that wasn't great you can see there's a lot less space in here oh right i
haven't even talked about the new mounting system for the drives
okay we're gonna do that but first i want you guys to check out this is how
they used to cable it up what a nightmare we've actually done a
swap for a dead port on one of these things and it was not a lot of fun
this right here is apparently a 3d printed like kind of friction mount and
then that works with the spring mount that they already had on some of the
newer chassis along with that pcb backplane which apparently makes it
easier to align the slots perfectly so that it's easier to put the drives in
and out so let's see if that actually worked out
all right so in used to be fine anyway but oh oh okay
that's not bad so all that remains now
is to install 60 drives well 58 i already did two
you can tell this one is very prototype it's got lots of scratches and dings and
stuff i believe this really was their working sample of it
before they sent it over to me you know you could call it used or you could call
it pre-tested
home stretch and
900 60 terabytes of raw spinning storage
along with 46 terabytes of SSD storage
for a total of one petabyte
in a single chassis but i'm not quite done yet right now the
one thing that's missing here is a slog device so i already talked about read
caching which doesn't really have any dangers associated with it because
you're just making copies of this data to put on your RAM or on your l2 arc but
right caching write caching is something you can do with zfs and you can use your
memory for it but the problem is that in the event of a sudden power loss which
who knows could happen any in-flight data that's sitting in RAM but hasn't
been committed to your hard drives yet will be lost so it might be worthwhile
adding something like an octane SSD to one of our pci express lots over here to
handle caching data that is being written so that we won't lose it in the
event that it's sitting in RAM limbo and hasn't been
committed to persistent storage all that though is going to be reserved for part
two where Anthony and i are going to team up to get this thing up and running
on the network so we can start offloading some of the data from the
original vault to the new consolidated
vault yes my friends the capacity of 2
volts is now 1 volt isn't technology amazing
speaking of amazing technology pulseway is a real-time remote monitoring and
management software that helps you fix problems on the go by sending commands
from any mobile device it's compatible with Windows mac and Linux and pulseface
single app gives you remote desktop functionality you can get real-time
status system resources logged in users
network performance you can manage Windows updates and more with pulseway
you can create and deploy custom scripts to automate your it task and you can
scan install things update all your systems on the go it's super cool and
super powerful and you can try it for free at pulseway.com or through our link
in the video description so thanks for watching guys if you're looking for something else to watch why
not check out the epic saga that was our NVMe storage server upgrade yeah we're
basically replacing the whole server room right now if you guys didn't sort
of pick up on that