WEBVTT

00:00:00.240 --> 00:00:06.879
as promised i have brought you guys the

00:00:03.760 --> 00:00:08.080
one and only petabyte project

00:00:06.879 --> 00:00:13.920
two and the one and only Anthony Anthony our

00:00:10.960 --> 00:00:18.800
task today is very simple we had two hours to have this petabyte project

00:00:16.720 --> 00:00:22.960
deployed so that the editors and the shooters can offload a bunch of footage

00:00:20.640 --> 00:00:29.320
off of the existing vault onto this one so we can load balance things a little

00:00:24.560 --> 00:00:29.320
bit easy right easy

00:00:37.280 --> 00:00:42.399
so uh go

00:00:40.879 --> 00:00:47.120
wave the wand what what do you want to do what you do well i don't know i can plug it i mean

00:00:45.200 --> 00:00:51.520
it's not it's not powered i can't like just cast a spell and we're going to do

00:00:49.440 --> 00:00:55.039
something really similar to what we've got for existing petabyte project where

00:00:53.600 --> 00:01:00.800
we're going to do four 15-drive v-devs in raid z2 so so that

00:00:58.320 --> 00:01:05.760
you guys know that allows us to lose up to eight drives total before we'd lose

00:01:03.760 --> 00:01:09.040
any data and that has come in handy yeah the re-silvering was a bit of a

00:01:07.040 --> 00:01:14.159
nail-biting process let me tell you that it took many weeks i was thinking of

00:01:11.520 --> 00:01:17.439
adding an optane pci express card as a slog

00:01:15.280 --> 00:01:21.040
so that would be a right cache effectively

00:01:18.720 --> 00:01:24.640
that doesn't depend on dram so that in the event of a sudden power loss we

00:01:22.799 --> 00:01:28.799
would still retain all the data i don't know how big we did that though 480 gigs

00:01:27.360 --> 00:01:34.000
that's the smallest obtained drive we have okay okay i'll go grab one

00:01:31.280 --> 00:01:38.640
don't forget to sign it out yay got em oh they break off yeah

00:01:36.479 --> 00:01:42.720
for a server chassis that's like hundreds of dollars this is not an

00:01:40.560 --> 00:01:45.990
amazing experience but so we ready to power it up

00:01:44.400 --> 00:01:47.840
uh sure

00:01:47.840 --> 00:01:54.320
it's actually not even crazy loud no i was expecting much louder did they send

00:01:52.240 --> 00:02:00.399
the credentials for login well let's try root and password

00:01:57.520 --> 00:02:04.880
okay we should probably change that i'm a hacker today boys first of all you

00:02:03.280 --> 00:02:11.440
shouldn't be logged in as root well i mean just for like initial

00:02:06.479 --> 00:02:12.800
configurations matter yeah do rm-rf

00:02:11.440 --> 00:02:17.280
don't do that oh wait you were gonna ruin my stuff

00:02:15.200 --> 00:02:22.400
now the original plan was for 45 drives to build me a custom 75 drive unit right

00:02:20.560 --> 00:02:26.080
but seagate did send 75 drives

00:02:24.319 --> 00:02:30.080
so i was kind of thinking you know what would be cool

00:02:27.760 --> 00:02:34.239
is if we took an old decommissioned storinator

00:02:31.440 --> 00:02:37.840
and actually created a new cluster with this one and that one i'd have to

00:02:36.000 --> 00:02:41.120
upgrade it obviously so we go epic with the other one as well and uh what do you

00:02:40.000 --> 00:02:44.879
think no yes good bad

00:02:43.040 --> 00:02:49.840
they said not to do it over a petabyte let's do it hold on a second we might

00:02:47.440 --> 00:02:51.920
need drivers for that melanox one what are those 10 gigabit cars that are in

00:02:51.200 --> 00:02:57.920
there they're just the onboard ones um uh um i

00:02:55.920 --> 00:03:01.280
could try another slot okay if you want to throw the power switch to make sure

00:02:58.959 --> 00:03:05.840
it doesn't wake back up yep and now

00:03:03.360 --> 00:03:08.640
ready that's not a power switch i ask you if

00:03:07.280 --> 00:03:12.400
there's a power switch well there is a power switch

00:03:10.959 --> 00:03:15.920
that's not hey how's it going i found another 60

00:03:14.319 --> 00:03:20.560
drive storinator so now we could have matching ones are they really matching well if i

00:03:19.120 --> 00:03:23.920
change out all the hardware in this one they would be somewhat matching yes

00:03:22.239 --> 00:03:27.440
somewhat yeah it's the older model though i was kind of thinking while

00:03:25.280 --> 00:03:31.200
you're working on this i refurbed this i don't think this should take very long

00:03:29.280 --> 00:03:35.360
all we have to do right now is last words

00:03:33.200 --> 00:03:39.840
let me see if i've got the melanox card now yeah there we go

00:03:38.080 --> 00:03:44.000
oh nice so just moving it to the other slot worked yeah i hope that just

00:03:41.920 --> 00:03:48.959
rebooted on its own probably fine

00:03:45.200 --> 00:03:50.799
famous last words yeah thanks for that

00:03:48.959 --> 00:03:56.159
oh man do you remember when 45 drives used to use non-ferrous screws

00:03:52.879 --> 00:03:57.840
me too this is super annoying

00:03:56.159 --> 00:04:01.680
i'm going to replace every screw in this case when i rebuild this okay when i

00:04:00.319 --> 00:04:06.239
said this probably wasn't going to take very long i didn't realize that we had

00:04:03.519 --> 00:04:11.439
to download so many updates so here we are oh failure wait god why

00:04:11.439 --> 00:04:18.720
meanwhile i'm extracting this motherboard oh shoot i didn't unplug the

00:04:15.519 --> 00:04:20.160
front panel connectors or the 24 pin

00:04:18.720 --> 00:04:23.840
we are totally not going to finish this today oh

00:04:22.240 --> 00:04:29.280
package g deploy because it's not available in the mirror anymore

00:04:26.479 --> 00:04:31.919
what's g deploy i have no idea what the crap

00:04:30.320 --> 00:04:36.880
oh boy that is a dusty server

00:04:35.199 --> 00:04:39.120
dusty boy ah

00:04:39.280 --> 00:04:44.639
go away they're saying to use gluster ansible so

00:04:42.960 --> 00:04:48.320
i'll just use that hey there we go

00:04:46.080 --> 00:04:51.360
i feel like i'm playing operation don't touch the sides

00:04:52.320 --> 00:04:57.360
okay hey is that all lit up yeah the drives are showed up wow yeah it was

00:04:55.759 --> 00:05:00.720
just a different command sweet thanks Anthony go nuts

00:05:01.759 --> 00:05:06.639
that was our motto at best buy was have fun while being the best how'd that work

00:05:05.040 --> 00:05:10.080
out for you we were the worst performing store in canada

00:05:08.400 --> 00:05:13.440
yeah like all the contacts are still there if you can get enough mounting

00:05:11.520 --> 00:05:16.320
pressure on the pins it should be able to work what if we jammed something in

00:05:15.520 --> 00:05:22.560
there that might do something non-conductive that just presses against there i found

00:05:19.600 --> 00:05:26.560
an old junk 3d print there's actually a bit on this side too whatever hit that

00:05:24.960 --> 00:05:31.039
thing hit it pretty hard somebody just ripped the memory out of it without pulling the tabs first no i think it was

00:05:29.360 --> 00:05:34.960
during transport at some point well they're physically in let's see what

00:05:32.720 --> 00:05:39.479
happens show me the 128 gigs

00:05:40.400 --> 00:05:45.680
deleting intensifies uh oh yeah

00:05:44.320 --> 00:05:50.479
we got it we repaired the memories a lot

00:05:47.919 --> 00:05:54.960
repaired it's such a dumb fix but it's this is totally gonna work thankfully

00:05:52.639 --> 00:05:59.280
hot glue is non-conductive and it melted the filament a little so

00:05:57.039 --> 00:06:04.000
didn't think of that hot glue it's it's hot filament it melts our hot glue fix

00:06:02.240 --> 00:06:08.000
is applied so this should boot up with 64 gigs of RAM right now

00:06:06.400 --> 00:06:11.759
good got it

00:06:10.160 --> 00:06:17.680
the scale of the project has changed a little bit so we're gonna cheat

00:06:14.319 --> 00:06:20.080
ever so slightly and pull three of our

00:06:17.680 --> 00:06:24.080
eight terabyte ssds and put them in this second server

00:06:21.759 --> 00:06:27.680
we actually pulled out all of the eight gig sticks of RAM that

00:06:25.840 --> 00:06:32.319
were in the servers and we are replacing them with 32 gig sticks so each of them

00:06:29.919 --> 00:06:36.960
is going to have 256 gigs of RAM now hey you only live once and uh i think that's

00:06:35.039 --> 00:06:40.720
pretty much all that i've changed other than that the last time you guys saw

00:06:38.560 --> 00:06:46.000
this it had old hardware in it and now it is running brand spanking new 16 core

00:06:43.680 --> 00:06:49.919
epic hardware and i've got uh another 256 gigs of RAM that i can throw in this

00:06:48.880 --> 00:06:54.479
puppy um oh

00:06:52.000 --> 00:06:58.479
and then oh okay but we got 256 gigs of RAM okay

00:06:55.919 --> 00:07:01.360
sweet this is working then theoretically there's nothing wrong with what i'm

00:07:00.000 --> 00:07:05.919
doing here but i am officially the jank king this

00:07:04.000 --> 00:07:09.680
power supply has only two SATA connectors and i need to plug in five

00:07:08.319 --> 00:07:14.960
drives so get this

00:07:11.360 --> 00:07:18.080
i'm going SATA male to say to female to

00:07:14.960 --> 00:07:21.120
molex male to molex female to quadruple

00:07:18.080 --> 00:07:23.440
seda over here that'll power the four uh

00:07:21.120 --> 00:07:27.759
regular SATA drives and then i'm using the other one

00:07:25.120 --> 00:07:31.599
for the octane drive just kind of jam that in there yeah i mean thankfully

00:07:29.680 --> 00:07:34.479
ssds don't take that much power exactly that's what i'm talking about who's

00:07:33.039 --> 00:07:39.120
going to say anything all of our viewers

00:07:36.560 --> 00:07:44.080
yeah i am just about done building this puppy i've got my SATA cables run just

00:07:41.840 --> 00:07:48.639
casually doing some cable management with my LTT Store.com cable test fans

00:07:46.800 --> 00:07:53.520
are clear fans are spinning everything's good to go all we got to do is load it up with some

00:07:51.280 --> 00:07:57.680
drives oh i guess i need to find all the drive mounting hardware i put away a

00:07:55.680 --> 00:08:01.840
panel that has some screws that go with it i put a little magnet on there with

00:07:59.919 --> 00:08:04.319
all the screws stuck to it see that free tech tip

00:08:05.199 --> 00:08:12.720
day three of this epic shoot what do you think is this just gonna boot up

00:08:09.680 --> 00:08:14.560
um i would say 95 yes i mean we haven't

00:08:12.720 --> 00:08:17.759
even checked the RAM compatibility yet okay well that's a bad sign go ahead and

00:08:16.080 --> 00:08:20.800
cut go ahead and cut go ahead and cut no dice on getting it to boot so i'm going

00:08:19.280 --> 00:08:24.800
to start by ripping some extraneous hardware out and just see if at least we

00:08:22.879 --> 00:08:29.199
know the CPU and RAM work nope i don't think that's doing

00:08:27.599 --> 00:08:32.479
any better go ahead and cut according to the manual i shouldn't need any of that

00:08:30.800 --> 00:08:35.519
stuff cut do you want to get a close-up of ah my foot

00:08:34.240 --> 00:08:41.479
cut all right we're back where we started and cut now to just do one hba

00:08:38.320 --> 00:08:41.479
at a time

00:08:42.320 --> 00:08:47.360
one and cut what does this g3 denote so

00:08:45.760 --> 00:08:52.160
i can cut go ahead and cut oh weird just cut this

00:08:49.839 --> 00:08:57.279
is very boring troubleshooting after a BIOS update and a cmos clear

00:08:55.920 --> 00:09:01.279
it's up everything is detected what do you think

00:08:59.600 --> 00:09:03.920
i'm just gonna friction mount it no no i'm gonna put some hot glue along the

00:09:02.640 --> 00:09:08.640
top of the card edges and i'm gonna stick it down on it don't worry about it it's all good i read about it in a

00:09:06.720 --> 00:09:13.440
textbook somewhere these babies ain't going to deliver themselves i i i

00:09:11.360 --> 00:09:17.600
literally cannot tell if you're joking people claim they don't love the gore

00:09:14.800 --> 00:09:22.000
but they love the gore it's in it's done we can put drives in it now right yeah i

00:09:20.000 --> 00:09:26.399
mean theoretically it's hot plug right heck yeah oh right okay so no we should

00:09:25.279 --> 00:09:31.440
just do this in the server and that one's a lot harder to remove the drives from we have to remove the drives in

00:09:28.959 --> 00:09:34.480
order to transport it so i think that's the extent of the

00:09:32.959 --> 00:09:38.480
testing we can do here what about this we're gonna have to remove all the

00:09:37.200 --> 00:09:41.519
drives and then put it back together in there

00:09:42.399 --> 00:09:49.440
okay do we have to put them back in the same order yeah i think so okay

00:09:46.640 --> 00:09:54.320
unfortunately we're out of power plugs the good news is i did buy a second one

00:09:51.680 --> 00:09:57.360
that isn't mounted so we can put that in but the reason i didn't bother doing it

00:09:56.000 --> 00:10:02.240
was that it was going to be a royal pain in the butt checklist

00:09:59.600 --> 00:10:08.160
put in new pdu for more power move albion server up

00:10:04.959 --> 00:10:10.800
figure out all the kvm cables and labels

00:10:08.160 --> 00:10:17.440
so that everything corresponds again and then add two more servers easy right

00:10:14.160 --> 00:10:19.839
easy when you put it like that yeah

00:10:17.440 --> 00:10:19.839
okay

00:10:20.480 --> 00:10:26.959
wait no missed up a bit good in

00:10:25.040 --> 00:10:31.120
i've had worse man am i gonna have to just kind of jam it in there i'm glad

00:10:28.880 --> 00:10:35.839
you guys can't see this right now okay we've got power we've got kvm

00:10:34.079 --> 00:10:39.839
so we can throw them in plug in network and then we're good to go

00:10:37.360 --> 00:10:43.279
all right this is awkward mind labeling that will i put the rails in pull out

00:10:41.200 --> 00:10:47.910
just a little there we go should be in easy peasy

00:10:46.000 --> 00:10:51.760
fill her up boys

00:10:51.760 --> 00:10:59.040
my ARM so strong they don't understand boom box so big carry it with two hands

00:10:56.800 --> 00:11:03.120
all i wanna do is sit back and kick it all the cops wanna do is write me a

00:11:01.040 --> 00:11:06.480
ticket wait this is four why'd i put it here we gotta move it up one go ahead

00:11:05.200 --> 00:11:10.160
and cut oh man now to make transporting this a

00:11:08.880 --> 00:11:14.560
little bit easier we're going to take out all the drives put them on this

00:11:12.560 --> 00:11:17.040
little cart here and transport this separately

00:11:17.839 --> 00:11:23.519
we're gonna make the biggest gluster cluster we can by filling it up with

00:11:22.160 --> 00:11:26.160
some of the drives that we're just gonna return later

00:11:27.040 --> 00:11:33.360
uh i guess yeah i'll just grab it sure yeah you know what i got it don't worry

00:11:30.640 --> 00:11:38.240
about it no problem okay okay yeah boom box so big carry it with two hands yeah

00:11:36.000 --> 00:11:42.720
it's fine i'll just do this that's fine i got it oh i thought you were being no

00:11:41.040 --> 00:11:46.480
now the main trick is to put these all in the right spots all the storage we'll

00:11:45.120 --> 00:11:51.200
ever need ever wow that's a total capacity of like

00:11:49.200 --> 00:11:54.160
300 drives i noticed that this one's flashing green

00:11:52.720 --> 00:11:59.200
though go ahead and cut let's see so you changed the melanox

00:11:55.760 --> 00:12:02.079
card out yep so we got the static ip 44

00:11:59.200 --> 00:12:06.160
and we are able to ping rock on now we just needed the same thing to delta

00:12:03.920 --> 00:12:11.200
three how long does it take though however

00:12:08.720 --> 00:12:14.800
long it takes for me to manually oh my god you have to key those in

00:12:13.440 --> 00:12:18.399
yeah so we're picking this up tomorrow

00:12:16.800 --> 00:12:22.839
yeah okay yeah

00:12:20.160 --> 00:12:28.000
okay we're so close the good news is that Anthony

00:12:25.440 --> 00:12:32.800
found a script that makes it easy to import all the drives and it doesn't say

00:12:30.720 --> 00:12:36.560
nothing here buds yeah no way

00:12:33.920 --> 00:12:42.000
they are the most eastern canadian tech company of all time wait so how do we

00:12:39.600 --> 00:12:45.760
have hold on a second um but that doesn't make any sense because

00:12:43.920 --> 00:12:51.360
that's whatever that's whatever that's doing i have no idea it's lifeline time

00:12:48.560 --> 00:12:54.639
yo brett long story short we tried to use the new script

00:12:55.200 --> 00:13:01.839
okay awesome so it's an older chassis and i repurposed it with new hardware so

00:12:59.920 --> 00:13:05.519
there's a drive they're just not showing up okay give me a second i'm gonna do

00:13:03.519 --> 00:13:09.519
some open heart surgery here moving all the cables is going to be a little dumb

00:13:08.240 --> 00:13:14.560
don't fix your hardware to fix your software no big deal okay all right

00:13:12.480 --> 00:13:19.040
you guys having fun yet oh yeah it's always fun

00:13:16.959 --> 00:13:23.120
so with you just i know it's kind of annoying but change the cables so that

00:13:21.200 --> 00:13:28.240
they're in this order and then we'll fix the order of the cards in the d-map

00:13:24.880 --> 00:13:30.560
thing and we're home 45 drives never fix

00:13:28.240 --> 00:13:34.800
the software problem with hardware also 45 drives can you unplug literally

00:13:33.279 --> 00:13:39.440
all the cables and put them in the other way in their defense this is not a

00:13:36.959 --> 00:13:42.800
supported configuration so are they just in reverse order on this motherboard

00:13:41.120 --> 00:13:47.600
because i think super micro numbers them from bottom to top i think so you're

00:13:45.279 --> 00:13:50.639
coming out for ltx right yeah hell yeah okay cool i'll see you then then okay

00:13:49.360 --> 00:13:53.200
sweet they're all showing up so it was just a matter of

00:13:54.000 --> 00:14:00.160
uh it was just a matter of uh reseeding all the cables in a different order

00:13:58.639 --> 00:14:04.639
sort of yeah sure yes so we're ready to pull

00:14:02.639 --> 00:14:08.959
them together and then start samba and then we can see over one petabyte

00:14:06.800 --> 00:14:12.000
accessible over the network we're ready to

00:14:10.240 --> 00:14:15.600
pull them together uh they recommend that a single brick shouldn't be more

00:14:13.440 --> 00:14:20.160
than 100 terabytes in size wait so this is good probes are good probes are very

00:14:17.839 --> 00:14:23.760
good okay we just probed our pure probe success probe success let me do the

00:14:21.760 --> 00:14:27.440
other way around pure probing do you want to see it i do want to see it now

00:14:28.000 --> 00:14:34.639
last time we had a petabyte of raw storage

00:14:32.480 --> 00:14:38.880
but because of overhead loss to redundancy and whatever

00:14:36.959 --> 00:14:43.120
the case may be we ended up with what like

00:14:40.079 --> 00:14:44.560
600 something yeah terabytes

00:14:43.120 --> 00:14:49.600
this time 1.2

00:14:46.800 --> 00:14:53.680
petabytes of storage it's ready would you say that's a new pb

00:14:52.160 --> 00:14:59.120
what's a pb personal best yeah definitely so i'm

00:14:56.320 --> 00:15:03.360
just gonna take a quick test file here how about this three gig file

00:15:02.000 --> 00:15:09.760
we going so yeah it's not quite 10 gig speeds around

00:15:06.639 --> 00:15:11.040
730 750 gigabytes a second but

00:15:09.760 --> 00:15:18.720
considering that we're writing to magnetic storage i am pretty pleased

00:15:16.240 --> 00:15:22.000
oh we never installed pulseway on the server andy just get Anthony installing

00:15:20.880 --> 00:15:25.680
pulseway it's up already just like that delta 3

00:15:24.000 --> 00:15:28.320
21 hours of uptime don't you wish the gloucester setup was that easy i know

00:15:27.440 --> 00:15:34.240
right so thanks for watching guys if you enjoyed this maybe check out our video

00:15:31.760 --> 00:15:37.120
rolling out our high speed NVMe server it was

00:15:35.199 --> 00:15:40.639
almost as headache inducing as this one actually more so this this was

00:15:38.720 --> 00:15:44.959
relatively painless most of the problems were uh caused by me deciding that we

00:15:43.360 --> 00:15:49.920
were gonna roll two servers at once instead of just one so yeah

00:15:48.320 --> 00:15:55.639
we're gonna have that video linked below thanks andy he's been a trooper he's

00:15:51.920 --> 00:15:55.639
been a trooper for this