WEBVTT

00:00:00.160 --> 00:00:06.080
so i've got a problem when it comes to building a storage server there are two

00:00:03.919 --> 00:00:11.920
main options for your drives you can either go with ssds which are compact

00:00:08.960 --> 00:00:16.960
and have amazing performance or you can go with traditional mechanical storage

00:00:14.480 --> 00:00:22.800
which has the benefit of lower prices and outstanding capacity with the

00:00:19.600 --> 00:00:26.240
drawback of being an order of magnitude

00:00:22.800 --> 00:00:28.400
or more slower especially in highly

00:00:26.240 --> 00:00:32.399
random workloads like if they were loaded up with a bunch of virtual

00:00:29.840 --> 00:00:37.840
machines or in our case if they're pulling network storage duty for a whole

00:00:35.200 --> 00:00:44.399
bunch of heavy users who are all trying to edit 8k videos at the same time if

00:00:41.280 --> 00:00:47.680
only there were some way to

00:00:44.399 --> 00:00:50.800
merge the benefits of both

00:00:47.680 --> 00:00:52.800
with the drawbacks of neither

00:00:50.800 --> 00:00:56.399
well there is it's called the apple fusion drive

00:00:54.480 --> 00:01:01.039
and this episode is brought to you by ifixit the marlin screwdriver set from

00:00:58.800 --> 00:01:07.450
ifixit features five specialty precision screwdrivers check it out at ifixit.com

00:01:03.440 --> 00:01:10.499
forward slash Linus at the link below

00:01:13.439 --> 00:01:20.640
the concept of tiered storage is nothing new it's been commonplace in the data

00:01:18.080 --> 00:01:24.720
center for many years and has even made its way down to the consumer level in

00:01:22.640 --> 00:01:32.000
the form of Intel's rapid storage technology or rst which uses a small

00:01:28.320 --> 00:01:34.960
fast SSD to accelerate a larger slower

00:01:32.000 --> 00:01:40.479
mechanical drive and more recently Intel's octane memory technology which

00:01:37.840 --> 00:01:46.640
does pretty much the exact same thing but with even faster optane rather than

00:01:43.439 --> 00:01:49.439
nand ssds the problem that i'm running

00:01:46.640 --> 00:01:55.119
into here is that most of the tiering solutions out there have one of two

00:01:52.240 --> 00:01:59.600
problems they are either inordinately complicated to set up requiring

00:01:57.200 --> 00:02:05.840
extensive knowledge of not just Linux but even broader storage architecture or

00:02:03.520 --> 00:02:09.840
they are expensive because you are paying the folks who have the

00:02:07.520 --> 00:02:14.000
aforementioned knowledge to create a turnkey solution for you

00:02:12.239 --> 00:02:19.599
there are some exceptions to this however and i would like to explore a

00:02:16.720 --> 00:02:24.879
couple of them so the popular freebsd based freenas software from ix systems

00:02:22.720 --> 00:02:30.480
uses your system's RAM and its ssds to cache frequently used

00:02:28.640 --> 00:02:36.800
data that's stored on your mechanical hard drives but the issue is that zfs's

00:02:33.840 --> 00:02:42.319
particular implementation is geared more towards accelerating performance of

00:02:39.440 --> 00:02:46.319
large databases and doesn't look like it's going to be great for our workload

00:02:44.080 --> 00:02:51.920
because the file sizes that we want cached that we're storing on our hard

00:02:48.400 --> 00:02:55.519
drives are so large like these are 8k

00:02:51.920 --> 00:02:59.280
red video files and there are practical

00:02:55.519 --> 00:03:01.280
limits to how much level 2 arc that

00:02:59.280 --> 00:03:06.959
would be the SSD tier i mean there's also practical limits to how much RAM

00:03:03.920 --> 00:03:08.000
you can put in a given system

00:03:06.959 --> 00:03:13.920
so then the one we're gonna try today is the

00:03:10.560 --> 00:03:16.879
built-in tiering mode for storage spaces

00:03:13.920 --> 00:03:21.920
in Windows server and what better way to test this out than to set up a tiered

00:03:19.840 --> 00:03:26.640
storage space on the network so i'm going to use this test server right here

00:03:24.319 --> 00:03:31.400
and then have our team of editors just switch over to it and see what happens

00:03:32.560 --> 00:03:38.239
and it's at this point then that we wondered whether or not optane would

00:03:36.319 --> 00:03:41.440
make a difference for some of the heavy tasks that we deal with around the

00:03:39.840 --> 00:03:49.680
office no no it does not now our regular editing

00:03:44.879 --> 00:03:51.840
server uses 31 1.2 terabyte NVMe drives

00:03:49.680 --> 00:03:55.599
and it was really expensive for us to build and they've been complaining that

00:03:53.840 --> 00:04:01.840
there's not enough space on it because we've only got about 24 terabytes of

00:03:58.720 --> 00:04:03.439
usable space after all the lost space to

00:04:01.840 --> 00:04:08.080
redundancy on it and they were like oh well you could just upgrade it i went yeah okay

00:04:06.799 --> 00:04:14.000
technically i do have another 17 bays that i could

00:04:11.360 --> 00:04:20.560
fill up but that's going to cost me like seven grand eight grand even if i buy

00:04:16.639 --> 00:04:22.800
used drives on ebay so i was

00:04:20.560 --> 00:04:28.800
looking for another way so our test server here the config is

00:04:25.120 --> 00:04:30.880
pretty straightforward i've got four 10

00:04:28.800 --> 00:04:36.720
terabyte drives then because i want to see exactly how much

00:04:34.560 --> 00:04:39.919
cash you need this is what really got me thinking about it

00:04:37.919 --> 00:04:44.360
Intel uses just 32 gigs of cash

00:04:42.479 --> 00:04:49.520
for large multi-terabyte hard drives and we've

00:04:46.800 --> 00:04:56.320
seen real world huge increases in performance so i went okay let's try

00:04:52.320 --> 00:04:58.960
this so i took two of their 900 series

00:04:56.320 --> 00:05:04.000
optane pci express drives and then i've got a total of 40 terabytes of

00:05:01.040 --> 00:05:09.440
mechanical storage and i want to see if for a fraction of the price

00:05:06.400 --> 00:05:11.520
we can get good enough performance that

00:05:09.440 --> 00:05:15.120
the editors aren't gonna run into a bottleneck

00:05:12.960 --> 00:05:21.039
so you can see i've got my four mechanical drives i've got my two ssds

00:05:18.560 --> 00:05:27.120
uh allocation is all gonna be automatic so i go ahead and i try to create a pool

00:05:23.919 --> 00:05:29.840
okay i've created a pool now i try to

00:05:27.120 --> 00:05:35.120
create a virtual disk create storage tiers on this virtual disk it should

00:05:32.560 --> 00:05:40.160
automatically detect which ones are your fast drives and which ones are your

00:05:38.240 --> 00:05:44.320
capacity drives so we're going to use mirroring because

00:05:42.160 --> 00:05:48.479
that way we're protected from a drive failure we're going to use fixed

00:05:46.320 --> 00:05:53.199
provisioning and now we get to specify the size of our faster and our standard

00:05:51.520 --> 00:05:57.840
tier we're going to go ahead and select the maximum sizes failed to create

00:05:55.120 --> 00:06:02.800
virtual disks not supported so the issue is that the gui version of storage

00:05:59.520 --> 00:06:05.600
spaces doesn't have some very important

00:06:02.800 --> 00:06:09.600
options like selecting how many columns you want

00:06:06.400 --> 00:06:11.759
so when you only have two drives and the

00:06:09.600 --> 00:06:17.360
default is three columns it spits out this stupid error how did

00:06:14.080 --> 00:06:18.960
nobody at microsoft notice this for

00:06:17.360 --> 00:06:23.440
years anyway doesn't matter

00:06:20.800 --> 00:06:26.720
i called in a lifeline led eater over on the forum as a guru when it comes to

00:06:25.360 --> 00:06:31.280
this stuff so i'm just going to pull up his dm over there and follow his

00:06:29.600 --> 00:06:34.880
instructions to hopefully do this in powershell so this will be fun you guys

00:06:33.520 --> 00:06:38.639
are going to be learning right along with me i haven't done this before the

00:06:36.639 --> 00:06:42.800
video so i can still use the gui to create my storage pool apparently

00:06:41.120 --> 00:06:49.360
i'm going to go ahead and do that then i need to fire up powershell and

00:06:45.680 --> 00:06:50.400
change the default number of columns

00:06:49.360 --> 00:06:55.600
okay well i don't know what i just did

00:06:52.720 --> 00:06:59.199
but i'm going to change it to mirror

00:06:56.720 --> 00:07:05.599
number of columns default one i think i think we did that let's

00:07:02.720 --> 00:07:09.680
find out so i think the command i just entered

00:07:08.400 --> 00:07:13.360
changed the mirror layout default

00:07:12.479 --> 00:07:18.960
to one column it did not

00:07:17.120 --> 00:07:26.240
okay uh so i'm using powershell ise now

00:07:22.319 --> 00:07:26.240
and this seems to be working better

00:07:26.960 --> 00:07:32.400
mirror number of columns default one

00:07:30.880 --> 00:07:35.680
come on so mad right now

00:07:35.759 --> 00:07:43.039
i just i just want to change a stupid

00:07:39.440 --> 00:07:46.000
setting it's definitely called

00:07:43.039 --> 00:07:50.880
mirror it's definitely called storage pool one like i create

00:07:48.560 --> 00:07:53.440
these commands using their own thing

00:07:52.000 --> 00:08:00.240
we'll figure this out at some point the good news is there's a backup plan

00:07:56.639 --> 00:08:04.030
i swear i had this working at one point

00:08:00.240 --> 00:08:05.599
what is this oh okay oops actually yeah

00:08:05.599 --> 00:08:11.440
oops all right

00:08:09.440 --> 00:08:16.080
here we go so right now we've only got a 10 gig link i can put a faster card in

00:08:14.000 --> 00:08:22.240
here if it ends up being a networking bottleneck but for now what we need to

00:08:18.720 --> 00:08:24.560
do is grab a bunch of

00:08:22.240 --> 00:08:30.400
just like video projects that are actually current and copy them to our

00:08:27.759 --> 00:08:34.880
tiered storage drive here the idea is that i want to bring over more than

00:08:32.719 --> 00:08:39.440
would comfortably fit in the cache and then i want the editors during our test

00:08:37.279 --> 00:08:44.080
to open up first a whole set of completely different projects from each

00:08:40.959 --> 00:08:46.959
other then close those open up a new set

00:08:44.080 --> 00:08:52.000
go back open the other ones and evaluate the performance for themselves

00:08:49.600 --> 00:08:58.080
so we're back and we settled into a steady transfer speed of anywhere from

00:08:54.560 --> 00:09:00.560
200 to 300 plus megabytes a second

00:08:58.080 --> 00:09:04.640
and all of our projects are over on here so

00:09:02.320 --> 00:09:10.720
i guess all that's left for me to do then is go interrupt the editing team

00:09:08.800 --> 00:09:15.120
no no no so i just need to know if you guys all

00:09:12.640 --> 00:09:19.040
hit this and try to edit 8k footage at the same time

00:09:17.040 --> 00:09:22.399
will you notice slow downs because i know some people are

00:09:21.279 --> 00:09:28.160
sensitive to slow downs so here's the idea

00:09:26.720 --> 00:09:32.080
i don't even know what a network share is this is the network share but i don't

00:09:30.480 --> 00:09:36.080
know what that is uh so everyone kind of pick a project and

00:09:34.160 --> 00:09:39.680
open it and start doing video editor things whatever those are work the way

00:09:37.920 --> 00:09:44.160
you would normally work because full is going to be affected by like your system

00:09:41.920 --> 00:09:50.560
as well so the idea

00:09:47.760 --> 00:09:56.160
is that once it arrives it gets cached in the fast tier so i

00:09:53.760 --> 00:10:01.120
just wanted to get a feel for what this is like ed are you opening a project oh

00:09:58.399 --> 00:10:05.680
you're coding wait you're encoding it oh uh exporting probably isn't a very

00:10:03.680 --> 00:10:09.680
stressful test okay what are you doing taren i uh

00:10:07.600 --> 00:10:13.839
i don't know can you try doing some more editory things like pretend to edit this

00:10:11.760 --> 00:10:19.279
i'm moving stuff around i you know keep pretending pretend to do your job

00:10:17.040 --> 00:10:24.560
yes can you go back and play that sequence again

00:10:22.079 --> 00:10:29.839
and see if it's any better how fast would you expect it to be

00:10:28.079 --> 00:10:35.839
like is this normal or it's a little bit faster

00:10:33.839 --> 00:10:40.480
it's not that way okay just hold on a second i haven't i

00:10:37.760 --> 00:10:45.200
haven't given up on saving money yet so the thing i want to find out now is

00:10:42.800 --> 00:10:49.200
if you guys keep doing stuff will it get any better

00:10:47.600 --> 00:10:54.720
so i'm starting to come around to the realization that

00:10:51.200 --> 00:10:56.880
maybe tiered storage solutions

00:10:54.720 --> 00:11:00.480
aren't grab and go because in every environment you'd want

00:10:59.600 --> 00:11:06.240
your tiers to behave in a completely different manner

00:11:04.399 --> 00:11:11.040
like if there was some programmability in storage spaces where i could say okay

00:11:08.720 --> 00:11:16.079
take all the newest files that come in and assume they are important

00:11:13.760 --> 00:11:20.560
until such time as they do not get accessed for

00:11:17.760 --> 00:11:24.000
a very long period or until such time as something new comes in and

00:11:22.240 --> 00:11:28.720
bumps it out then it would be less likely for those

00:11:26.160 --> 00:11:32.480
guys to open up a project and have it chug until

00:11:30.560 --> 00:11:36.480
storage spaces goes oh this is important and can promote it from the slow

00:11:34.320 --> 00:11:42.480
mechanical tier so it's possible that if i had a much much larger fast NVMe tier

00:11:40.880 --> 00:11:47.120
we wouldn't run into this issue because this is really a very heavy load for

00:11:44.240 --> 00:11:51.200
four hard drives and two ssds but the problem is that that would

00:11:48.800 --> 00:11:55.120
require me to basically build that entire server without even knowing that

00:11:53.680 --> 00:11:59.519
it would work so you are dropping i'm dropping 90 of

00:11:57.760 --> 00:12:04.959
my frames can you rate your editing experience out of 10 for me here there's

00:12:01.600 --> 00:12:06.959
like little moments of seven out of ten

00:12:04.959 --> 00:12:11.200
but then overall it's more like a three out of ten sorry you're saying this is

00:12:09.120 --> 00:12:14.720
better than waning a little bit maybe all right everyone just go back to wanik

00:12:13.279 --> 00:12:17.760
unmap the drive throw it all in the garbage

00:12:22.800 --> 00:12:30.880
so that's pretty much it i am

00:12:26.639 --> 00:12:33.760
coming to the the the acceptance stage

00:12:30.880 --> 00:12:38.560
in the in the stages of grief here um

00:12:35.279 --> 00:12:41.200
where i'm realizing that if we want

00:12:38.560 --> 00:12:45.839
more capacity of our high performance network storage

00:12:42.880 --> 00:12:50.720
i'm just going to have to pull out the wallet and make it happen

00:12:48.880 --> 00:12:55.760
thankfully to take some of the sting out of it is our sponsor for this video

00:12:53.360 --> 00:13:00.560
squarespace with squarespace you can make your own beautiful website without

00:12:58.800 --> 00:13:03.839
the hassle they're all in one platform makes it easy to get up and running

00:13:02.399 --> 00:13:08.160
quickly and they've got all kinds of great features from their logo designer

00:13:06.079 --> 00:13:12.000
to their built-in e-commerce to the ability to publish in apple news format

00:13:10.320 --> 00:13:16.800
so that your blog will be available immediately to millions of potential

00:13:14.320 --> 00:13:21.279
readers worldwide and they offer all kinds of support so you can build the

00:13:18.639 --> 00:13:26.399
best website you can build whether it's their seminars or contacting their 24 7

00:13:24.320 --> 00:13:30.079
tech support via live chat and email so don't take my word for it go try out

00:13:28.639 --> 00:13:35.040
squarespace at the link in the video description then when you decide to sign

00:13:32.079 --> 00:13:39.279
up for real go to squarespace.com LTT and you can get 10 off your first

00:13:37.440 --> 00:13:42.399
purchase so thanks for watching guys if you

00:13:40.639 --> 00:13:45.360
disliked this video you can hit that button but if you liked it hit like get

00:13:43.839 --> 00:13:50.160
subscribed or maybe consider checking out where to buy the stuff we featured at the link in the video description

00:13:48.560 --> 00:13:52.959
also down there is our merch store which has cool shirts

00:13:53.200 --> 00:14:00.360
it's winter like this one and our community forum which you should totally

00:13:57.360 --> 00:14:00.360
join