1
00:00:00,160 --> 00:00:06,080
so i've got a problem when it comes to building a storage server there are two

2
00:00:03,919 --> 00:00:11,920
main options for your drives you can either go with ssds which are compact

3
00:00:08,960 --> 00:00:16,960
and have amazing performance or you can go with traditional mechanical storage

4
00:00:14,480 --> 00:00:22,800
which has the benefit of lower prices and outstanding capacity with the

5
00:00:19,600 --> 00:00:26,240
drawback of being an order of magnitude

6
00:00:22,800 --> 00:00:28,400
or more slower especially in highly

7
00:00:26,240 --> 00:00:32,399
random workloads like if they were loaded up with a bunch of virtual

8
00:00:29,840 --> 00:00:37,840
machines or in our case if they're pulling network storage duty for a whole

9
00:00:35,200 --> 00:00:44,399
bunch of heavy users who are all trying to edit 8k videos at the same time if

10
00:00:41,280 --> 00:00:47,680
only there were some way to

11
00:00:44,399 --> 00:00:50,800
merge the benefits of both

12
00:00:47,680 --> 00:00:52,800
with the drawbacks of neither

13
00:00:50,800 --> 00:00:56,399
well there is it's called the apple fusion drive

14
00:00:54,480 --> 00:01:01,039
and this episode is brought to you by ifixit the marlin screwdriver set from

15
00:00:58,800 --> 00:01:07,450
ifixit features five specialty precision screwdrivers check it out at ifixit.com

16
00:01:03,440 --> 00:01:10,499
forward slash Linus at the link below

17
00:01:13,439 --> 00:01:20,640
the concept of tiered storage is nothing new it's been commonplace in the data

18
00:01:18,080 --> 00:01:24,720
center for many years and has even made its way down to the consumer level in

19
00:01:22,640 --> 00:01:32,000
the form of Intel's rapid storage technology or rst which uses a small

20
00:01:28,320 --> 00:01:34,960
fast SSD to accelerate a larger slower

21
00:01:32,000 --> 00:01:40,479
mechanical drive and more recently Intel's octane memory technology which

22
00:01:37,840 --> 00:01:46,640
does pretty much the exact same thing but with even faster optane rather than

23
00:01:43,439 --> 00:01:49,439
nand ssds the problem that i'm running

24
00:01:46,640 --> 00:01:55,119
into here is that most of the tiering solutions out there have one of two

25
00:01:52,240 --> 00:01:59,600
problems they are either inordinately complicated to set up requiring

26
00:01:57,200 --> 00:02:05,840
extensive knowledge of not just Linux but even broader storage architecture or

27
00:02:03,520 --> 00:02:09,840
they are expensive because you are paying the folks who have the

28
00:02:07,520 --> 00:02:14,000
aforementioned knowledge to create a turnkey solution for you

29
00:02:12,239 --> 00:02:19,599
there are some exceptions to this however and i would like to explore a

30
00:02:16,720 --> 00:02:24,879
couple of them so the popular freebsd based freenas software from ix systems

31
00:02:22,720 --> 00:02:30,480
uses your system's RAM and its ssds to cache frequently used

32
00:02:28,640 --> 00:02:36,800
data that's stored on your mechanical hard drives but the issue is that zfs's

33
00:02:33,840 --> 00:02:42,319
particular implementation is geared more towards accelerating performance of

34
00:02:39,440 --> 00:02:46,319
large databases and doesn't look like it's going to be great for our workload

35
00:02:44,080 --> 00:02:51,920
because the file sizes that we want cached that we're storing on our hard

36
00:02:48,400 --> 00:02:55,519
drives are so large like these are 8k

37
00:02:51,920 --> 00:02:59,280
red video files and there are practical

38
00:02:55,519 --> 00:03:01,280
limits to how much level 2 arc that

39
00:02:59,280 --> 00:03:06,959
would be the SSD tier i mean there's also practical limits to how much RAM

40
00:03:03,920 --> 00:03:08,000
you can put in a given system

41
00:03:06,959 --> 00:03:13,920
so then the one we're gonna try today is the

42
00:03:10,560 --> 00:03:16,879
built-in tiering mode for storage spaces

43
00:03:13,920 --> 00:03:21,920
in Windows server and what better way to test this out than to set up a tiered

44
00:03:19,840 --> 00:03:26,640
storage space on the network so i'm going to use this test server right here

45
00:03:24,319 --> 00:03:31,400
and then have our team of editors just switch over to it and see what happens

46
00:03:32,560 --> 00:03:38,239
and it's at this point then that we wondered whether or not optane would

47
00:03:36,319 --> 00:03:41,440
make a difference for some of the heavy tasks that we deal with around the

48
00:03:39,840 --> 00:03:49,680
office no no it does not now our regular editing

49
00:03:44,879 --> 00:03:51,840
server uses 31 1.2 terabyte NVMe drives

50
00:03:49,680 --> 00:03:55,599
and it was really expensive for us to build and they've been complaining that

51
00:03:53,840 --> 00:04:01,840
there's not enough space on it because we've only got about 24 terabytes of

52
00:03:58,720 --> 00:04:03,439
usable space after all the lost space to

53
00:04:01,840 --> 00:04:08,080
redundancy on it and they were like oh well you could just upgrade it i went yeah okay

54
00:04:06,799 --> 00:04:14,000
technically i do have another 17 bays that i could

55
00:04:11,360 --> 00:04:20,560
fill up but that's going to cost me like seven grand eight grand even if i buy

56
00:04:16,639 --> 00:04:22,800
used drives on ebay so i was

57
00:04:20,560 --> 00:04:28,800
looking for another way so our test server here the config is

58
00:04:25,120 --> 00:04:30,880
pretty straightforward i've got four 10

59
00:04:28,800 --> 00:04:36,720
terabyte drives then because i want to see exactly how much

60
00:04:34,560 --> 00:04:39,919
cash you need this is what really got me thinking about it

61
00:04:37,919 --> 00:04:44,360
Intel uses just 32 gigs of cash

62
00:04:42,479 --> 00:04:49,520
for large multi-terabyte hard drives and we've

63
00:04:46,800 --> 00:04:56,320
seen real world huge increases in performance so i went okay let's try

64
00:04:52,320 --> 00:04:58,960
this so i took two of their 900 series

65
00:04:56,320 --> 00:05:04,000
optane pci express drives and then i've got a total of 40 terabytes of

66
00:05:01,040 --> 00:05:09,440
mechanical storage and i want to see if for a fraction of the price

67
00:05:06,400 --> 00:05:11,520
we can get good enough performance that

68
00:05:09,440 --> 00:05:15,120
the editors aren't gonna run into a bottleneck

69
00:05:12,960 --> 00:05:21,039
so you can see i've got my four mechanical drives i've got my two ssds

70
00:05:18,560 --> 00:05:27,120
uh allocation is all gonna be automatic so i go ahead and i try to create a pool

71
00:05:23,919 --> 00:05:29,840
okay i've created a pool now i try to

72
00:05:27,120 --> 00:05:35,120
create a virtual disk create storage tiers on this virtual disk it should

73
00:05:32,560 --> 00:05:40,160
automatically detect which ones are your fast drives and which ones are your

74
00:05:38,240 --> 00:05:44,320
capacity drives so we're going to use mirroring because

75
00:05:42,160 --> 00:05:48,479
that way we're protected from a drive failure we're going to use fixed

76
00:05:46,320 --> 00:05:53,199
provisioning and now we get to specify the size of our faster and our standard

77
00:05:51,520 --> 00:05:57,840
tier we're going to go ahead and select the maximum sizes failed to create

78
00:05:55,120 --> 00:06:02,800
virtual disks not supported so the issue is that the gui version of storage

79
00:05:59,520 --> 00:06:05,600
spaces doesn't have some very important

80
00:06:02,800 --> 00:06:09,600
options like selecting how many columns you want

81
00:06:06,400 --> 00:06:11,759
so when you only have two drives and the

82
00:06:09,600 --> 00:06:17,360
default is three columns it spits out this stupid error how did

83
00:06:14,080 --> 00:06:18,960
nobody at microsoft notice this for

84
00:06:17,360 --> 00:06:23,440
years anyway doesn't matter

85
00:06:20,800 --> 00:06:26,720
i called in a lifeline led eater over on the forum as a guru when it comes to

86
00:06:25,360 --> 00:06:31,280
this stuff so i'm just going to pull up his dm over there and follow his

87
00:06:29,600 --> 00:06:34,880
instructions to hopefully do this in powershell so this will be fun you guys

88
00:06:33,520 --> 00:06:38,639
are going to be learning right along with me i haven't done this before the

89
00:06:36,639 --> 00:06:42,800
video so i can still use the gui to create my storage pool apparently

90
00:06:41,120 --> 00:06:49,360
i'm going to go ahead and do that then i need to fire up powershell and

91
00:06:45,680 --> 00:06:50,400
change the default number of columns

92
00:06:49,360 --> 00:06:55,600
okay well i don't know what i just did

93
00:06:52,720 --> 00:06:59,199
but i'm going to change it to mirror

94
00:06:56,720 --> 00:07:05,599
number of columns default one i think i think we did that let's

95
00:07:02,720 --> 00:07:09,680
find out so i think the command i just entered

96
00:07:08,400 --> 00:07:13,360
changed the mirror layout default

97
00:07:12,479 --> 00:07:18,960
to one column it did not

98
00:07:17,120 --> 00:07:26,240
okay uh so i'm using powershell ise now

99
00:07:22,319 --> 00:07:26,240
and this seems to be working better

100
00:07:26,960 --> 00:07:32,400
mirror number of columns default one

101
00:07:30,880 --> 00:07:35,680
come on so mad right now

102
00:07:35,759 --> 00:07:43,039
i just i just want to change a stupid

103
00:07:39,440 --> 00:07:46,000
setting it's definitely called

104
00:07:43,039 --> 00:07:50,880
mirror it's definitely called storage pool one like i create

105
00:07:48,560 --> 00:07:53,440
these commands using their own thing

106
00:07:52,000 --> 00:08:00,240
we'll figure this out at some point the good news is there's a backup plan

107
00:07:56,639 --> 00:08:04,030
i swear i had this working at one point

108
00:08:00,240 --> 00:08:05,599
what is this oh okay oops actually yeah

109
00:08:05,599 --> 00:08:11,440
oops all right

110
00:08:09,440 --> 00:08:16,080
here we go so right now we've only got a 10 gig link i can put a faster card in

111
00:08:14,000 --> 00:08:22,240
here if it ends up being a networking bottleneck but for now what we need to

112
00:08:18,720 --> 00:08:24,560
do is grab a bunch of

113
00:08:22,240 --> 00:08:30,400
just like video projects that are actually current and copy them to our

114
00:08:27,759 --> 00:08:34,880
tiered storage drive here the idea is that i want to bring over more than

115
00:08:32,719 --> 00:08:39,440
would comfortably fit in the cache and then i want the editors during our test

116
00:08:37,279 --> 00:08:44,080
to open up first a whole set of completely different projects from each

117
00:08:40,959 --> 00:08:46,959
other then close those open up a new set

118
00:08:44,080 --> 00:08:52,000
go back open the other ones and evaluate the performance for themselves

119
00:08:49,600 --> 00:08:58,080
so we're back and we settled into a steady transfer speed of anywhere from

120
00:08:54,560 --> 00:09:00,560
200 to 300 plus megabytes a second

121
00:08:58,080 --> 00:09:04,640
and all of our projects are over on here so

122
00:09:02,320 --> 00:09:10,720
i guess all that's left for me to do then is go interrupt the editing team

123
00:09:08,800 --> 00:09:15,120
no no no so i just need to know if you guys all

124
00:09:12,640 --> 00:09:19,040
hit this and try to edit 8k footage at the same time

125
00:09:17,040 --> 00:09:22,399
will you notice slow downs because i know some people are

126
00:09:21,279 --> 00:09:28,160
sensitive to slow downs so here's the idea

127
00:09:26,720 --> 00:09:32,080
i don't even know what a network share is this is the network share but i don't

128
00:09:30,480 --> 00:09:36,080
know what that is uh so everyone kind of pick a project and

129
00:09:34,160 --> 00:09:39,680
open it and start doing video editor things whatever those are work the way

130
00:09:37,920 --> 00:09:44,160
you would normally work because full is going to be affected by like your system

131
00:09:41,920 --> 00:09:50,560
as well so the idea

132
00:09:47,760 --> 00:09:56,160
is that once it arrives it gets cached in the fast tier so i

133
00:09:53,760 --> 00:10:01,120
just wanted to get a feel for what this is like ed are you opening a project oh

134
00:09:58,399 --> 00:10:05,680
you're coding wait you're encoding it oh uh exporting probably isn't a very

135
00:10:03,680 --> 00:10:09,680
stressful test okay what are you doing taren i uh

136
00:10:07,600 --> 00:10:13,839
i don't know can you try doing some more editory things like pretend to edit this

137
00:10:11,760 --> 00:10:19,279
i'm moving stuff around i you know keep pretending pretend to do your job

138
00:10:17,040 --> 00:10:24,560
yes can you go back and play that sequence again

139
00:10:22,079 --> 00:10:29,839
and see if it's any better how fast would you expect it to be

140
00:10:28,079 --> 00:10:35,839
like is this normal or it's a little bit faster

141
00:10:33,839 --> 00:10:40,480
it's not that way okay just hold on a second i haven't i

142
00:10:37,760 --> 00:10:45,200
haven't given up on saving money yet so the thing i want to find out now is

143
00:10:42,800 --> 00:10:49,200
if you guys keep doing stuff will it get any better

144
00:10:47,600 --> 00:10:54,720
so i'm starting to come around to the realization that

145
00:10:51,200 --> 00:10:56,880
maybe tiered storage solutions

146
00:10:54,720 --> 00:11:00,480
aren't grab and go because in every environment you'd want

147
00:10:59,600 --> 00:11:06,240
your tiers to behave in a completely different manner

148
00:11:04,399 --> 00:11:11,040
like if there was some programmability in storage spaces where i could say okay

149
00:11:08,720 --> 00:11:16,079
take all the newest files that come in and assume they are important

150
00:11:13,760 --> 00:11:20,560
until such time as they do not get accessed for

151
00:11:17,760 --> 00:11:24,000
a very long period or until such time as something new comes in and

152
00:11:22,240 --> 00:11:28,720
bumps it out then it would be less likely for those

153
00:11:26,160 --> 00:11:32,480
guys to open up a project and have it chug until

154
00:11:30,560 --> 00:11:36,480
storage spaces goes oh this is important and can promote it from the slow

155
00:11:34,320 --> 00:11:42,480
mechanical tier so it's possible that if i had a much much larger fast NVMe tier

156
00:11:40,880 --> 00:11:47,120
we wouldn't run into this issue because this is really a very heavy load for

157
00:11:44,240 --> 00:11:51,200
four hard drives and two ssds but the problem is that that would

158
00:11:48,800 --> 00:11:55,120
require me to basically build that entire server without even knowing that

159
00:11:53,680 --> 00:11:59,519
it would work so you are dropping i'm dropping 90 of

160
00:11:57,760 --> 00:12:04,959
my frames can you rate your editing experience out of 10 for me here there's

161
00:12:01,600 --> 00:12:06,959
like little moments of seven out of ten

162
00:12:04,959 --> 00:12:11,200
but then overall it's more like a three out of ten sorry you're saying this is

163
00:12:09,120 --> 00:12:14,720
better than waning a little bit maybe all right everyone just go back to wanik

164
00:12:13,279 --> 00:12:17,760
unmap the drive throw it all in the garbage

165
00:12:22,800 --> 00:12:30,880
so that's pretty much it i am

166
00:12:26,639 --> 00:12:33,760
coming to the the the acceptance stage

167
00:12:30,880 --> 00:12:38,560
in the in the stages of grief here um

168
00:12:35,279 --> 00:12:41,200
where i'm realizing that if we want

169
00:12:38,560 --> 00:12:45,839
more capacity of our high performance network storage

170
00:12:42,880 --> 00:12:50,720
i'm just going to have to pull out the wallet and make it happen

171
00:12:48,880 --> 00:12:55,760
thankfully to take some of the sting out of it is our sponsor for this video

172
00:12:53,360 --> 00:13:00,560
squarespace with squarespace you can make your own beautiful website without

173
00:12:58,800 --> 00:13:03,839
the hassle they're all in one platform makes it easy to get up and running

174
00:13:02,399 --> 00:13:08,160
quickly and they've got all kinds of great features from their logo designer

175
00:13:06,079 --> 00:13:12,000
to their built-in e-commerce to the ability to publish in apple news format

176
00:13:10,320 --> 00:13:16,800
so that your blog will be available immediately to millions of potential

177
00:13:14,320 --> 00:13:21,279
readers worldwide and they offer all kinds of support so you can build the

178
00:13:18,639 --> 00:13:26,399
best website you can build whether it's their seminars or contacting their 24 7

179
00:13:24,320 --> 00:13:30,079
tech support via live chat and email so don't take my word for it go try out

180
00:13:28,639 --> 00:13:35,040
squarespace at the link in the video description then when you decide to sign

181
00:13:32,079 --> 00:13:39,279
up for real go to squarespace.com LTT and you can get 10 off your first

182
00:13:37,440 --> 00:13:42,399
purchase so thanks for watching guys if you

183
00:13:40,639 --> 00:13:45,360
disliked this video you can hit that button but if you liked it hit like get

184
00:13:43,839 --> 00:13:50,160
subscribed or maybe consider checking out where to buy the stuff we featured at the link in the video description

185
00:13:48,560 --> 00:13:52,959
also down there is our merch store which has cool shirts

186
00:13:53,200 --> 00:14:00,360
it's winter like this one and our community forum which you should totally

187
00:13:57,360 --> 00:14:00,360
join