we make a lot of videos which is why over the years we've built out over 2
and a half pedabytes of bulk storage but
somehow it's not enough It's never
enough and that's especially true when you consider all of our upcoming
projects the videos just keep coming the badminton gaming center needs capacity
and if I am ever going to deliver on that free storage backups as an employee
perk concept I'm going to need a lot
more storage like at least another pedy worth in this economy you're right
storage ain't cheap but what if I told you that I could get a pedite for half
the price I'm listening well we can in
our quest to find the most cost-effective storage upgrade we
stumbled upon these they are recertified
drives but the recertification is done by Seagate themselves and they come in
at just about half the price of the same drives brand new so like any other same
person would I immediately handed eBay my credit card information and ordered
$10,000 worth of them that's 60 drives
so you got to be wondering lonus what's the catch come closer and I'm going to
tell you it's our sponsor MSI they've got
laptop deals on Deck this December from the sleek and slim modern 15h aai to the
gaming Powerhouse Katana a15 AI you can
save big by clicking our Link in the description
okay the real answer to what's the catch is we don't know yet that's kind of the
point of this adventure you see the best advice that we've ever gotten and given
is to avoid secondhand storage so this
is kind of untreed territory for us but the deal it was too sweet to pass up now
it's not like we were worried about getting outright scammed eBay has pretty
good buyer protections and the main sellers of these recertified drives seem
to have excellent ratings with most of the drives even coming with some kind of
warranty but there's still a lot of questions like what does recertified
even mean are they used it's hard to say
I mean the eBay listings for these drives say that they have extremely low
power on hours and no bad sectors which would seem to suggest that they're
basically brand new but if they were recertified by Seagate the manufacturer
then in theory they could just reset the power on hours the next thing the script
says is unbox them but there's not a whole lot of unboxing to do like is this
how they came well it's it's not like when you buy an individual drive it's
not like they're all going to ship in like Amazon packaging when you're buying
60 they come in trads and that makes
sense okay so how used do these look I
would say they look pretty not used to me not at all there's some fingerprints
but you know our inventory people definitely put asset tags on them not on
this one oh it's freaking clean we gloves even the screw holes I don't
think these have ever been mounted to anything look at this 22 2022 are they
all like that 2020
2021 so these were manufactured anytime
over the last few years I mean these drives didn't come out until like the
end of 2021 like December oh yeah that one's 2020 so that must be a very early
one where did they come from server Parts deals ah yes on eBay so in instead
of buying our drives from seagate's own store we chose server part deals they
sell them either Factory sealed still recertified but sealed or they actually
crack them open and pre-b bur in test them for up to 50 hours according to
server part deals the chance of getting a DOA Drive is actually much lower with
those Burnin ones so we opted for that
when we picked these up last December sometimes it takes us a while to catch
up to our production ideas we found that
The Sweet Spot was these 20 terab drives
for $180 a piece and looking on eBay
seems like we actually got a pretty sweet deal they're closer to $220 now
but even that is a huge discount compared to new and you could probably
save a bit by buying from them directly rather than buying through eBay
Disclosure by the way server part deals is one of our sponsors now but not on
this video and they actually weren't when we bought these drives from them no
we we paid retail for these should we plug them in yeah I want to see if the
hours are correct okay we got Cate SE tools we've got one of our drives here
yeah Drive details smart 21 okay that
checks out that's like nothing essentially yeah great now we just need
a machine to put them in since our goal is to Save A Buck we're going to reuse
as much Hardware as possible starting with this 45 Drive storinator XL 60 that
we decommissioned a few years ago it still fits 60 drives just like it did
when it was part of our original paby cluster but now it has double the
capacity cuz we're using 20 TB drives man I remember when we built that
so many people were like you're crazy dude just just use the cloud bro news
flash the cloud is expensive bro
and even when it isn't getting your data back from it is really expensive it's
actually been pretty validating seeing how many folks have been migrating back
to having their servers on Prem now to be clear we do use back Blaze backups
but only for backup and only for our active projects now obviously a home or
a small business user is unlikely to have one of these kicking around but
everything that we're doing could be scaled down to just a handful of drives
in a Nas chassis or even a single Drive in a recycled oplex like that one that
Jake has right next to him which is the same one that we showed off in our hexos
announcement video there's actually even people selling 3D printed Nas cases that
look pretty cool the point is you don't need to buy dozens of drives in a fancy
old server to get a great deal or at least what we hope is a really great
deal you sound so confident oh I'm
confident all right I'm confident it's time to rip these out of here oh it's
almost as old as me it is not on the
subject of our server being kind of old there are some Choice upgrades we want
to make to this old girl before we throw her back in the rack oh yeah if our
inventory records are correct the CPU in that machine is over 8 years old and
perform Sim Illy to a first gen ryzen 5 I mean look don't get me wrong there is
absolutely no shame in driving the car you got and for home use realistically
your network speed is going to be a bottleneck before your CPU but we
happened to have some newer Hardware sitting on the shelf specifically this
ASRock rack Rome d8 2T with an epic
7402p 24 core processor according to
PassMark it is nearly five times faster
than the old8 core that it's replacing now one important tip if you are going
to be reusing old gear like our old board make sure to update the BIOS to
get the latest security patches and definitely replace any dried out thermal
past this one should be good I I redid that not that long ago okay board's in
but I forgot to check the memory it looks like yeah we got 256 gigs of RAM
in here which means we should probably put in some more the rule of thumb is
about 1 gig of RAM per terabyte of storage for read caching but our
workload is actually pretty read light so I think we're just going to leave
this for now what we're not not going to leave are these hbas these high point
750s were end of life years ago but more importantly they just kind of suck So
the plan is to replace them with some LSI 9200 series cards now these are also
very old like PCI Express Gen 2 a but
for us that's a feature since they happen to have the right connectors for
the cabling in our machine so specifically we're going to be using a
pair of genuine LSI 8 drive cards and
then three of these 16 Drive cards two
of which are genuine and one of which is an eBay knockoff now we had a whole
video planned on these things back in the day which is actually why we have
this I think this one's the fake one yeah it doesn't say LSI on the front
whereas the other ones do but the thing is counterfeit silicon is pretty unusual
and I suspect this is just old chips
from maybe failed boards in some other way that have been resolder onto new
boards so should be fine I guess that's part of
the adventure I've had some friends who've had these like cheap cards work
for years without issues and others who've had them die in a few months we
don't have much choice either way since the legit ones are really hard to find
I'm almost done wiring up all the drives then all I got to do is Chuck a little m.2 boot Drive in there and then in our
last slot that's not it uh I'm going to
be using a melanox there we go Invidia connectx 6 dual Port 100 Gig network
card oh you want to put that in too we're also going to use this cool
adapter do dad to connect this 2 and 1/2 in Dell Gen 4 NVMe SSD that we had lying
around and then we can use that to help with read caching or level two Arc it
can hang out let screw it in first what I don't need to screw nothing in you
screw it in look it fell out that's because you didn't screw it in way to go
Jake didn't say drop it on the be careful that's from startech they kindly
sent that to us thanks
startech now to keep everything in this area cool I'm going to throw in a noctua
industrial 3000 RPM 140 mil fan that way
we won't have any issues with overheating on our network card hbas or
the NVMe SSD that's in there now to find out if it
boots I don't actually know like it
probably
should n my optimism is steadily
decreasing if you have a situation like this where you're not getting a display
signal from a computer you're trying to troubleshoot a good way to check if it's
actually working is to use caps lock or numb lock which you can see when I click
caps lock it is turning on and off so clearly
this system is posting it's just not outputting the display signal for some
reason or potentially this cable or the monitor is not working I'm going to try
the ipmi and see if that works hey okay
it's working sick with that out of the way let's talk software there are
obviously a lot of options but our go-to for a while has been trass scale it's
free which is always great and most importantly the ZF FS file system it
uses is perfect for large setups like this and super resilient so if we have
any issues with these drives which I hope we don't our data should be fine if
you're a home gamer you might also want to check out hexos the userfriendly nas
software Linus invested in once it's out of Early Access but if you're not
feeling adventurous there's always unraid whose main claim to fame is the
ability to expand its array one drive at a time or there's open media Vault we've
got trass installed now everything seems to be working and we can finally put
some hard drives in heavy as a full train it's heavy so I've recruited
help it'll take a sec for them all to get like detected and imported but let's
see how many we have so far ooh is she working I mean it's on but I haven't
looked at the drives yet so I guess let's see how we doing
61 wow they're all working well they all show up well yeah I mean we haven't made
an array yet should we run a smart test on all of them first can you do that all
at once oh yeah nice wow well that's great so they all pass a short smart
test if you're getting new drives you should do more than that you should
usually people recommend zeroing them all out not because you need to clear
the data but as a burn in to make sure that all the sectors are good and
everything is happy we're not going to do that right now but it's not a bad
thing to try and in theory server part deals has already done that for us cool
so we can make a pool now oh boy what do you want to call it swimming what so
it's a swimm swimming pool no I don't like that I'm going to call it ocean I'm
not salty since this is primarily archival storage and since we have a lot
of discs we're going to group our drives into six 10 Drive raid Z2 vdevs that two
in the raid Z2 means that we have two drives worth of data protection kind of
like raid six so for us to lose any
actual data we would need to have three drives fail and all three would have to
be within the same 10 Drive Group which which is fairly unlikely since
multi-drive failures like that usually happen when you end up with drives from
the same bad batch and since our drives are recertified it's much more likely
that they are from different time periods and different batches and the
script says in theory because we hadn't actually looked at the stamps on the
chassis yet in practice they are very different 2021 2022 they're all over the
place now for a home setup you're probably going to go with something like a four six or eight wide raid Z1 or Z2
depending on how important that data is to you but with the new r z expansion
feature you can start as small as two or even three drives and then expand that
existing group as you want to add more drives that's pretty convenient yeah
well we can click Start now or go yeah do it so we should have 873 tib bytes
what are you for t to TB 960 terab damn
hard drives have just been kind of quietly in the background getting like
kind of awesome kind of thick well I've paid absolutely no attention attention
to the it's like children uh our cash SSD is not detected
most people if they're trying to build something like this at home they're not building with a petabyte of drives
they're not going to have a cache SSD in it so let's have a look something about
it is not happy we will have to figure that out later yep I'm not expecting
like crazy numbers cuz it's really old hbas and we have a very wide vde depth
size I mean but 8 gigabytes a second
that's pretty good or giby bytes oh I'll take it goes up and down and up and down
it's hard drivve so we're not really gonna get much better than that well
dude I'm not gonna complain about that that's sick we're not even GNA be able to realistically do that so yeah that's
great but most of what we're doing on this thing is sequential anyway exactly
that is like five to seven times what we
would even be doing over like a 10 gig connection yeah which is most of the
accesses we do except when we're doing backups and stuff anyways so great
freaking awesome yeah I mean the drives they haven't complained wait this is
great but remember you said you were going to reach out to and find out what's the deal with these drives I did
they didn't respond at least they haven't yet and it's been like 4 days or
something like that okay I did find a press release on their website about
this program with eBay and it makes it sound almost like they're taking drives
that people were going to throw away like from a data center recycler and
refurbing them I don't know it doesn't
say anything conclusively and they look really new they look really new it seems
like our best guess is that they're just customer returns or something like that
okay well bad batches they rehabed one way or another in total we spent
$1,160 on 62 drives which compared to
the 235,000 we would have paid for new ones
seems like a pretty good deal as long as they don't all crap out for now things
look good but it's hard to say what the long-term reliability is going to look
like I'm hopeful though especially given that none of them died immediately so
even if they do the fact that we're using ZFS means we should be able to
resilver and repair the in a pretty safe way yeah I mean most
hard drives either die like in the first day or you know 3 four 5 years later
right server part deals also warranties the drives themselves so should be okay
yeah and full disclosure um I took about
half of these drives for another project and they were running for 6 months and
they still don't have errors okay well that's a good sign yeah and to feed my
data hoarder habits in this economy I actually switched over to recertify
drives at my house and and I've got two more or three more of them actually that
have been running for multiple years now and they've been great now a brand new
exos Drive is going to come with a 5year warranty instead of just two but given
these are around half the price with halfish the warranty it was kind of
worth the gamble for me but let us know in the comments if you've had any
experience with recertified drives either good or bad like I don't know we
get Flack sometimes when we show a good deal online and then it goes out of
stock a few hours later and no one else can get the same deal but that shouldn't
be the case with these there's an entire industry around reusing recertifying and
refurbishing Enterprise it equipment and there seems to be no shortage of storage
devices uh one of our favorites is the those jbod disc shelves oh yeah those
allow you to connect a bunch of hard drives to like an existing system and uh
we did a video on them a while ago you should check it out yeah we also came
across a pretty wild deal on some NVMe ssds when we were researching this
project so uh Happy deal hunting out there for now it's time for me to go
hunting segue season to our sponsor
ground news when you think of the news you think cold hard facts right wrong
now more than ever you're being told the stories the media wants you to hear
ground News is here to help filter out the fluff and let you come to your own conclusions they gather articles on
stories from thousands of sources and take into account political biases and
allow you to compare the coverage and see how the story changes depending on
whose's reporting I'm sure you've all heard that Pat Ginger recently retired
as Intel CEO around when they announced their next generation of graphics cards
you can see on screen right now that reports are not really leaning heavily
left or right but there's a tiny bit of left leaning bias and thanks to their
bias comparison tool as more articles come in you'll see a breakdown of what
aspects of the story are being focused on by each side their blind spot tool is
a great feature that shows you stories that might not be covered by your own
preferred news sources go to ground. news/ LTT to get started and take
advantage of a very limited 50% discount on their Vantage plans for yourself or
maybe as a little gift if you guys like this video why not check out the mother
Vault build hell yeah that was pretty sick that was like the deluxe version of
those Drive shelves yeah it is exactly that yeah and you know I might use some
of these drives as like spares for that