as promised i have brought you guys the
one and only petabyte project
two and the one and only Anthony Anthony our
task today is very simple we had two hours to have this petabyte project
deployed so that the editors and the shooters can offload a bunch of footage
off of the existing vault onto this one so we can load balance things a little
bit easy right easy
so uh go
wave the wand what what do you want to do what you do well i don't know i can plug it i mean
it's not it's not powered i can't like just cast a spell and we're going to do
something really similar to what we've got for existing petabyte project where
we're going to do four 15-drive v-devs in raid z2 so so that
you guys know that allows us to lose up to eight drives total before we'd lose
any data and that has come in handy yeah the re-silvering was a bit of a
nail-biting process let me tell you that it took many weeks i was thinking of
adding an optane pci express card as a slog
so that would be a right cache effectively
that doesn't depend on dram so that in the event of a sudden power loss we
would still retain all the data i don't know how big we did that though 480 gigs
that's the smallest obtained drive we have okay okay i'll go grab one
don't forget to sign it out yay got em oh they break off yeah
for a server chassis that's like hundreds of dollars this is not an
amazing experience but so we ready to power it up
uh sure
it's actually not even crazy loud no i was expecting much louder did they send
the credentials for login well let's try root and password
okay we should probably change that i'm a hacker today boys first of all you
shouldn't be logged in as root well i mean just for like initial
configurations matter yeah do rm-rf
don't do that oh wait you were gonna ruin my stuff
now the original plan was for 45 drives to build me a custom 75 drive unit right
but seagate did send 75 drives
so i was kind of thinking you know what would be cool
is if we took an old decommissioned storinator
and actually created a new cluster with this one and that one i'd have to
upgrade it obviously so we go epic with the other one as well and uh what do you
think no yes good bad
they said not to do it over a petabyte let's do it hold on a second we might
need drivers for that melanox one what are those 10 gigabit cars that are in
there they're just the onboard ones um uh um i
could try another slot okay if you want to throw the power switch to make sure
it doesn't wake back up yep and now
ready that's not a power switch i ask you if
there's a power switch well there is a power switch
that's not hey how's it going i found another 60
drive storinator so now we could have matching ones are they really matching well if i
change out all the hardware in this one they would be somewhat matching yes
somewhat yeah it's the older model though i was kind of thinking while
you're working on this i refurbed this i don't think this should take very long
all we have to do right now is last words
let me see if i've got the melanox card now yeah there we go
oh nice so just moving it to the other slot worked yeah i hope that just
rebooted on its own probably fine
famous last words yeah thanks for that
oh man do you remember when 45 drives used to use non-ferrous screws
me too this is super annoying
i'm going to replace every screw in this case when i rebuild this okay when i
said this probably wasn't going to take very long i didn't realize that we had
to download so many updates so here we are oh failure wait god why
meanwhile i'm extracting this motherboard oh shoot i didn't unplug the
front panel connectors or the 24 pin
we are totally not going to finish this today oh
package g deploy because it's not available in the mirror anymore
what's g deploy i have no idea what the crap
oh boy that is a dusty server
dusty boy ah
go away they're saying to use gluster ansible so
i'll just use that hey there we go
i feel like i'm playing operation don't touch the sides
okay hey is that all lit up yeah the drives are showed up wow yeah it was
just a different command sweet thanks Anthony go nuts
that was our motto at best buy was have fun while being the best how'd that work
out for you we were the worst performing store in canada
yeah like all the contacts are still there if you can get enough mounting
pressure on the pins it should be able to work what if we jammed something in
there that might do something non-conductive that just presses against there i found
an old junk 3d print there's actually a bit on this side too whatever hit that
thing hit it pretty hard somebody just ripped the memory out of it without pulling the tabs first no i think it was
during transport at some point well they're physically in let's see what
happens show me the 128 gigs
deleting intensifies uh oh yeah
we got it we repaired the memories a lot
repaired it's such a dumb fix but it's this is totally gonna work thankfully
hot glue is non-conductive and it melted the filament a little so
didn't think of that hot glue it's it's hot filament it melts our hot glue fix
is applied so this should boot up with 64 gigs of RAM right now
good got it
the scale of the project has changed a little bit so we're gonna cheat
ever so slightly and pull three of our
eight terabyte ssds and put them in this second server
we actually pulled out all of the eight gig sticks of RAM that
were in the servers and we are replacing them with 32 gig sticks so each of them
is going to have 256 gigs of RAM now hey you only live once and uh i think that's
pretty much all that i've changed other than that the last time you guys saw
this it had old hardware in it and now it is running brand spanking new 16 core
epic hardware and i've got uh another 256 gigs of RAM that i can throw in this
puppy um oh
and then oh okay but we got 256 gigs of RAM okay
sweet this is working then theoretically there's nothing wrong with what i'm
doing here but i am officially the jank king this
power supply has only two SATA connectors and i need to plug in five
drives so get this
i'm going SATA male to say to female to
molex male to molex female to quadruple
seda over here that'll power the four uh
regular SATA drives and then i'm using the other one
for the octane drive just kind of jam that in there yeah i mean thankfully
ssds don't take that much power exactly that's what i'm talking about who's
going to say anything all of our viewers
yeah i am just about done building this puppy i've got my SATA cables run just
casually doing some cable management with my LTT Store.com cable test fans
are clear fans are spinning everything's good to go all we got to do is load it up with some
drives oh i guess i need to find all the drive mounting hardware i put away a
panel that has some screws that go with it i put a little magnet on there with
all the screws stuck to it see that free tech tip
day three of this epic shoot what do you think is this just gonna boot up
um i would say 95 yes i mean we haven't
even checked the RAM compatibility yet okay well that's a bad sign go ahead and
cut go ahead and cut go ahead and cut no dice on getting it to boot so i'm going
to start by ripping some extraneous hardware out and just see if at least we
know the CPU and RAM work nope i don't think that's doing
any better go ahead and cut according to the manual i shouldn't need any of that
stuff cut do you want to get a close-up of ah my foot
cut all right we're back where we started and cut now to just do one hba
at a time
one and cut what does this g3 denote so
i can cut go ahead and cut oh weird just cut this
is very boring troubleshooting after a BIOS update and a cmos clear
it's up everything is detected what do you think
i'm just gonna friction mount it no no i'm gonna put some hot glue along the
top of the card edges and i'm gonna stick it down on it don't worry about it it's all good i read about it in a
textbook somewhere these babies ain't going to deliver themselves i i i
literally cannot tell if you're joking people claim they don't love the gore
but they love the gore it's in it's done we can put drives in it now right yeah i
mean theoretically it's hot plug right heck yeah oh right okay so no we should
just do this in the server and that one's a lot harder to remove the drives from we have to remove the drives in
order to transport it so i think that's the extent of the
testing we can do here what about this we're gonna have to remove all the
drives and then put it back together in there
okay do we have to put them back in the same order yeah i think so okay
unfortunately we're out of power plugs the good news is i did buy a second one
that isn't mounted so we can put that in but the reason i didn't bother doing it
was that it was going to be a royal pain in the butt checklist
put in new pdu for more power move albion server up
figure out all the kvm cables and labels
so that everything corresponds again and then add two more servers easy right
easy when you put it like that yeah
okay
wait no missed up a bit good in
i've had worse man am i gonna have to just kind of jam it in there i'm glad
you guys can't see this right now okay we've got power we've got kvm
so we can throw them in plug in network and then we're good to go
all right this is awkward mind labeling that will i put the rails in pull out
just a little there we go should be in easy peasy
fill her up boys
my ARM so strong they don't understand boom box so big carry it with two hands
all i wanna do is sit back and kick it all the cops wanna do is write me a
ticket wait this is four why'd i put it here we gotta move it up one go ahead
and cut oh man now to make transporting this a
little bit easier we're going to take out all the drives put them on this
little cart here and transport this separately
we're gonna make the biggest gluster cluster we can by filling it up with
some of the drives that we're just gonna return later
uh i guess yeah i'll just grab it sure yeah you know what i got it don't worry
about it no problem okay okay yeah boom box so big carry it with two hands yeah
it's fine i'll just do this that's fine i got it oh i thought you were being no
now the main trick is to put these all in the right spots all the storage we'll
ever need ever wow that's a total capacity of like
300 drives i noticed that this one's flashing green
though go ahead and cut let's see so you changed the melanox
card out yep so we got the static ip 44
and we are able to ping rock on now we just needed the same thing to delta
three how long does it take though however
long it takes for me to manually oh my god you have to key those in
yeah so we're picking this up tomorrow
yeah okay yeah
okay we're so close the good news is that Anthony
found a script that makes it easy to import all the drives and it doesn't say
nothing here buds yeah no way
they are the most eastern canadian tech company of all time wait so how do we
have hold on a second um but that doesn't make any sense because
that's whatever that's whatever that's doing i have no idea it's lifeline time
yo brett long story short we tried to use the new script
okay awesome so it's an older chassis and i repurposed it with new hardware so
there's a drive they're just not showing up okay give me a second i'm gonna do
some open heart surgery here moving all the cables is going to be a little dumb
don't fix your hardware to fix your software no big deal okay all right
you guys having fun yet oh yeah it's always fun
so with you just i know it's kind of annoying but change the cables so that
they're in this order and then we'll fix the order of the cards in the d-map
thing and we're home 45 drives never fix
the software problem with hardware also 45 drives can you unplug literally
all the cables and put them in the other way in their defense this is not a
supported configuration so are they just in reverse order on this motherboard
because i think super micro numbers them from bottom to top i think so you're
coming out for ltx right yeah hell yeah okay cool i'll see you then then okay
sweet they're all showing up so it was just a matter of
uh it was just a matter of uh reseeding all the cables in a different order
sort of yeah sure yes so we're ready to pull
them together and then start samba and then we can see over one petabyte
accessible over the network we're ready to
pull them together uh they recommend that a single brick shouldn't be more
than 100 terabytes in size wait so this is good probes are good probes are very
good okay we just probed our pure probe success probe success let me do the
other way around pure probing do you want to see it i do want to see it now
last time we had a petabyte of raw storage
but because of overhead loss to redundancy and whatever
the case may be we ended up with what like
600 something yeah terabytes
this time 1.2
petabytes of storage it's ready would you say that's a new pb
what's a pb personal best yeah definitely so i'm
just gonna take a quick test file here how about this three gig file
we going so yeah it's not quite 10 gig speeds around
730 750 gigabytes a second but
considering that we're writing to magnetic storage i am pretty pleased
oh we never installed pulseway on the server andy just get Anthony installing
pulseway it's up already just like that delta 3
21 hours of uptime don't you wish the gloucester setup was that easy i know
right so thanks for watching guys if you enjoyed this maybe check out our video
rolling out our high speed NVMe server it was
almost as headache inducing as this one actually more so this this was
relatively painless most of the problems were uh caused by me deciding that we
were gonna roll two servers at once instead of just one so yeah
we're gonna have that video linked below thanks andy he's been a trooper he's
been a trooper for this