1
00:00:00,400 --> 00:00:06,160
why the heck do you guys need a whole room full of servers

2
00:00:03,919 --> 00:00:09,360
just to make youtube videos we get asked this

3
00:00:07,839 --> 00:00:14,080
and actually a lot of other questions about our data management and our

4
00:00:10,960 --> 00:00:16,080
workflow all the time so for the last

5
00:00:14,080 --> 00:00:20,240
year actually ever since we got our red cameras i've been meaning to do an

6
00:00:17,840 --> 00:00:24,640
update for you guys on how we handle such heavy footage

7
00:00:21,840 --> 00:00:30,000
how we share resources with such a large team and how we do all of that while

8
00:00:27,279 --> 00:00:33,920
maintaining our rigorous daily release schedule so

9
00:00:32,480 --> 00:00:39,120
when jiwin reached out to us asking us to do a

10
00:00:36,320 --> 00:00:42,559
feature on their smooth 4 camera gimbal i thought well hey there's an

11
00:00:41,200 --> 00:00:46,239
opportunity why don't we grab this thing

12
00:00:44,480 --> 00:00:51,760
and take it with us while i show you guys the way that data flows through

13
00:00:48,800 --> 00:00:55,079
Linus media group to become a video on your screen

14
00:01:02,000 --> 00:01:08,880
so while i let you all appreciate the irony of a video about our overly

15
00:01:06,400 --> 00:01:13,360
complicated workflow shot on a cell phone with a simple handheld gimbal i'll

16
00:01:11,520 --> 00:01:17,680
tell you guys a little bit more about the zoom four so it's got a highly

17
00:01:15,439 --> 00:01:22,640
integrated control panel down here with apps for both iOS and Android that allow

18
00:01:20,159 --> 00:01:27,759
you to control your camera it's got focus pull and zoom capabilities it's

19
00:01:24,960 --> 00:01:32,159
got the ability to do vertigo shots with one button through the app or manually

20
00:01:30,240 --> 00:01:36,240
it's got what they call phone go mode for instant scene transitions and it's

21
00:01:34,320 --> 00:01:40,079
got both a quick standby mode and a one-click mode switch

22
00:01:38,079 --> 00:01:43,920
everything that we shot outside of the intro and this outro right now was

23
00:01:41,920 --> 00:01:47,759
filmed on this it's got a note 8 on it right now but it was shot on an iphone

24
00:01:45,520 --> 00:01:50,560
10 and i'll let you guys be the judge of the stabilized footage that comes off of

25
00:01:49,680 --> 00:01:56,240
it so to demonstrate some of the challenges with working with red footage we're

26
00:01:54,000 --> 00:01:59,600
actually going to take that intro that we just shot just now and we're going to

27
00:01:58,240 --> 00:02:05,119
ingest the whole thing to show you guys how we do it now red cameras

28
00:02:02,799 --> 00:02:09,920
are not great for recording audio in fact even with an external preamp it is

29
00:02:08,160 --> 00:02:14,080
really difficult to get anything resembling usable audio out of the

30
00:02:11,840 --> 00:02:18,239
camera directly so recently we finally gave in and resorted to an

31
00:02:16,319 --> 00:02:22,480
external audio recorder it really sucks to synchronize audio

32
00:02:20,800 --> 00:02:26,160
especially if you're stuck using the scratch audio that gets recorded

33
00:02:23,920 --> 00:02:30,560
directly to the camera and then the sd card out of your external recorder and

34
00:02:27,920 --> 00:02:35,120
aligning it manually but we recently got a little doodad called a tentacle sink

35
00:02:32,879 --> 00:02:39,680
you get two pieces here a master that sits on our mix pre-6 and a slave that

36
00:02:37,840 --> 00:02:42,720
sits on our camera you synchronize them at the beginning of the day and what

37
00:02:41,200 --> 00:02:47,680
they do is they inject what's called time code into the mag and into the sd

38
00:02:45,760 --> 00:02:51,440
card so that you can easily synchronize them later on down the line

39
00:02:50,400 --> 00:02:57,040
but that is far from the only problem so the

40
00:02:54,800 --> 00:03:02,640
thing about the weapon 8k so that's our camera recently renamed the

41
00:02:59,319 --> 00:03:04,480
dsmc2 helium or something like that

42
00:03:02,640 --> 00:03:09,200
is that it records footage they're both the same camera at up to 8k

43
00:03:07,040 --> 00:03:16,640
60 frames per second and with the 480 or 960 gig mags that's

44
00:03:13,200 --> 00:03:19,040
up to 300 megabytes per second so to put

45
00:03:16,640 --> 00:03:25,200
that in perspective if we were filming a simple TechLinked episode even though

46
00:03:22,400 --> 00:03:29,360
we will usually use like a 20 to 1 compression ratio so that's about four

47
00:03:28,000 --> 00:03:34,720
times the compression that they would probably use on something like a feature

48
00:03:31,120 --> 00:03:36,560
film that means that a 10 gig clip could

49
00:03:34,720 --> 00:03:41,840
still end up being in the neighborhood of about did i say 10 gig clip

50
00:03:39,360 --> 00:03:47,040
that means that a 10 minute clip could still end up being in the neighborhood

51
00:03:43,519 --> 00:03:48,799
of around 50 gigs and while it's less of

52
00:03:47,040 --> 00:03:53,519
an issue for something like Linus tech tips or Techquickie when you're

53
00:03:50,560 --> 00:03:56,560
recording dailies and you're expected to release the video on the same day that

54
00:03:55,120 --> 00:04:00,799
you film it like with something like a new show

55
00:03:58,560 --> 00:04:05,040
size of files ends up becoming a potential problem so that's why our two

56
00:04:02,959 --> 00:04:09,760
ingestations which you might recognize from one of our tech showdown episodes

57
00:04:07,840 --> 00:04:14,480
are both equipped with 10 gigabit network cards so one of them has this

58
00:04:11,920 --> 00:04:19,919
ASUS one which is basically the last generation version of this action one

59
00:04:17,199 --> 00:04:24,560
they're both using aquantia chips that allow them to be somehow under a hundred

60
00:04:22,400 --> 00:04:27,520
dollars for 10 gig networking there are a number of different ways that we

61
00:04:25,759 --> 00:04:31,440
ingest footage depending on the type of camera we're using so for red you can

62
00:04:29,520 --> 00:04:35,840
see each of our video clips is actually a folder made up of smaller broken up

63
00:04:34,479 --> 00:04:41,440
video clips so we're going to go ahead and grab that and copy the whole thing into our

64
00:04:39,840 --> 00:04:45,040
project folder for this video so that goes in a roll

65
00:04:43,840 --> 00:04:48,720
one what we're looking at right here is

66
00:04:46,880 --> 00:04:52,960
probably some kind of like weird buffering thing or something but this is

67
00:04:50,960 --> 00:04:57,360
about what we'd expect to see in terms of our ingest speeds

68
00:04:54,880 --> 00:05:02,479
it's not the full 10 gigabit because that would be in the neighborhood of one

69
00:04:59,040 --> 00:05:03,840
to 1.1 gigabytes per second but

70
00:05:02,479 --> 00:05:08,240
the reality of it is that you're going to run into bottlenecks elsewhere and in

71
00:05:06,240 --> 00:05:12,880
fact these red megs even though we're using an esata interface for them are

72
00:05:10,479 --> 00:05:18,720
just not that fast and they only read at about 230 megs a second

73
00:05:16,080 --> 00:05:22,800
then what we got to do is grab the audio clip off of our sd card that goes into

74
00:05:20,960 --> 00:05:27,840
our audio folder right here and that's pretty much the whole process

75
00:05:25,919 --> 00:05:32,479
for red footage but for that we wouldn't need powerful

76
00:05:30,160 --> 00:05:37,840
machines and there is a reason that each of these is running 64 gigs of RAM and a

77
00:05:35,120 --> 00:05:43,280
10 core extreme edition processor and that's because for our other cameras

78
00:05:40,160 --> 00:05:46,400
like the a7s for example we actually use

79
00:05:43,280 --> 00:05:48,560
adobe prelude which is an imperfect

80
00:05:46,400 --> 00:05:52,560
piece of software but has its uses

81
00:05:50,080 --> 00:05:56,400
to transcode the footage when we're bringing it in so what we do is we take

82
00:05:54,880 --> 00:06:02,320
whatever project it is and this is generally fast as possible we still shoot that on the a7s we create a

83
00:05:59,919 --> 00:06:06,960
subfolder and we set it to transcode to cineform 4k this improves our timeline

84
00:06:05,360 --> 00:06:10,960
performance when we're scrubbing through the clips in adobe premiere the other

85
00:06:09,360 --> 00:06:15,199
thing that we do is set a second destination for the original files that

86
00:06:13,280 --> 00:06:19,680
we're not transcoding just in case something goes wrong with the transcode

87
00:06:17,600 --> 00:06:23,120
which does happen from time to time and we need to go back and grab the original

88
00:06:22,080 --> 00:06:27,360
files now just to give you some idea even if

89
00:06:25,360 --> 00:06:31,520
we're not using the full potential of this network connection all the time of

90
00:06:29,280 --> 00:06:34,400
what it's capable of let's go ahead and just grab a file

91
00:06:33,280 --> 00:06:40,720
off the desktop and show you just what this puppy can do

92
00:06:37,440 --> 00:06:42,960
so that is saturating the read speeds of

93
00:06:40,720 --> 00:06:48,160
the raid 1 ssds that each of these machines are booted off of so that was a

94
00:06:45,360 --> 00:06:51,919
10 Gigabyte file let's go have a look on the other end of this Ethernet cable at

95
00:06:50,319 --> 00:06:57,199
how exactly that whole thing comes together now i've shown you guys our

96
00:06:53,919 --> 00:06:58,960
server room a fair number of times but i

97
00:06:57,199 --> 00:07:04,880
haven't given you guys an update in quite a while

98
00:07:00,560 --> 00:07:07,039
on how exactly it's working in here

99
00:07:04,880 --> 00:07:11,919
so the main server that everyone is editing off of at the same time

100
00:07:09,520 --> 00:07:17,120
is this one right here this is wanik server and it's running uh 24 plus one

101
00:07:15,120 --> 00:07:24,240
two three four five six seven so it's running 31

102
00:07:19,599 --> 00:07:26,960
Intel 750 series 1.1 terabyte NVMe ssds

103
00:07:24,240 --> 00:07:30,400
and this guy is an absolute monster

104
00:07:28,160 --> 00:07:34,720
so if we fire up performance monitor you can see that even though we've got

105
00:07:32,160 --> 00:07:42,319
four of our editors in office right now and we're seeing each of them doing 150

106
00:07:38,240 --> 00:07:45,840
120 50 megabytes a second of reads this

107
00:07:42,319 --> 00:07:48,160
thing is barely suffering and our

108
00:07:45,840 --> 00:07:55,199
there we go our z disc q depth is only about 0.5 that's the benefit of NVMe is

109
00:07:52,800 --> 00:08:00,479
that nice fast responsiveness and in fact when we were still using our SATA

110
00:07:57,759 --> 00:08:04,319
SSD server for everyone we were starting to run into issues once we had the room

111
00:08:02,319 --> 00:08:09,520
fully staffed over there where adobe premiere would crash but not just crash

112
00:08:07,280 --> 00:08:15,120
everyone's would crash at the same time and we narrowed that down to slow

113
00:08:11,680 --> 00:08:18,720
response times on our nas so this guy's

114
00:08:15,120 --> 00:08:22,479
connected at 40 gigabits per second

115
00:08:18,720 --> 00:08:24,639
this one right here has 24 SATA ssds and

116
00:08:22,479 --> 00:08:31,520
this one only gets used this is called qq server this one only gets used for

117
00:08:28,720 --> 00:08:36,560
large projects that only one person needs to work on at a time it's still

118
00:08:34,080 --> 00:08:39,279
SSD based so we still don't run into any crashes or any other weird issues like

119
00:08:38,399 --> 00:08:44,080
that but it doesn't have quite the snap that

120
00:08:41,519 --> 00:08:48,720
an NVMe machine does now something that we learned

121
00:08:45,920 --> 00:08:51,040
a really valuable lesson about i guess it must have been about two

122
00:08:50,000 --> 00:08:57,200
years ago was real time data replication and

123
00:08:54,320 --> 00:09:01,120
off-site backup and that is where this guy comes in so i want to show you guys

124
00:08:59,519 --> 00:09:04,560
a little trick here i don't know if i've ever actually done

125
00:09:02,720 --> 00:09:08,560
this demo in a video before i've shown people that have come into our office

126
00:09:06,480 --> 00:09:13,440
so i'm going to use my test folder here and i'm going to create a document

127
00:09:10,560 --> 00:09:18,240
called uh test for video right here this is a text file

128
00:09:15,920 --> 00:09:23,120
on our main drive then i'm going to jump over to this hard drive based server

129
00:09:20,399 --> 00:09:28,480
under it so this is running eight eight terabyte drives for a total of 64

130
00:09:25,839 --> 00:09:34,480
terabytes and i'm just going to pull up the

131
00:09:30,800 --> 00:09:34,480
wanik sink folder

132
00:09:34,959 --> 00:09:41,839
get that test folder open test video but

133
00:09:38,959 --> 00:09:46,480
just like that so within about five to ten seconds whether it's a text file or

134
00:09:44,240 --> 00:09:50,800
whether it's a large video file the synchronization software that we're

135
00:09:48,080 --> 00:09:55,600
running here we'll automatically dump it over to here so if this crashes we can

136
00:09:53,120 --> 00:10:00,880
actually continue editing videos within about 10 to 15 minutes of switching over

137
00:09:58,800 --> 00:10:04,399
everyone's map drives and another benefit that it gives us is that

138
00:10:02,959 --> 00:10:10,240
normally on a network drive when you delete a file it's just gone

139
00:10:07,600 --> 00:10:13,680
but instead check this out if i delete test for video i'm actually

140
00:10:12,240 --> 00:10:18,760
going to have to do it from a separate server but if i delete test for video

141
00:10:20,640 --> 00:10:24,720
boop you're gone you're done

142
00:10:25,519 --> 00:10:33,200
and i go into here that file is going to disappear but

143
00:10:30,880 --> 00:10:40,160
what will happen is we can go into the deletions folder and we can rescue it

144
00:10:36,000 --> 00:10:42,160
this has actually saved our butts

145
00:10:40,160 --> 00:10:47,360
more times than i would like to admit because

146
00:10:43,279 --> 00:10:48,800
stuff does get accidentally deleted now

147
00:10:47,360 --> 00:10:53,279
all of this is only for active projects that we are

148
00:10:51,360 --> 00:10:57,440
working on right now once we're done with something it goes

149
00:10:55,279 --> 00:11:01,839
on to petabyte project and we're actually running both phases of it right

150
00:10:59,760 --> 00:11:06,079
now when we originally deployed it we were only using one of them so we could

151
00:11:03,519 --> 00:11:10,480
just save power on hours for all the drives that were in the other one

152
00:11:07,839 --> 00:11:16,320
not the case anymore so petabyte project is now up to

153
00:11:12,839 --> 00:11:19,920
777 terabytes of total space of which

154
00:11:16,320 --> 00:11:22,240
we've consumed 432. this holds all of

155
00:11:19,920 --> 00:11:26,959
our archived projects so that in the event that we want to make a video that

156
00:11:23,920 --> 00:11:29,360
refers back to one of our other videos

157
00:11:26,959 --> 00:11:33,040
we can grab the original quality files rather than downloading off youtube like

158
00:11:31,040 --> 00:11:37,360
a lot of other youtubers do bringing us into this room one of the biggest

159
00:11:34,880 --> 00:11:41,440
challenges is not just working with red footage because the files are so heavy

160
00:11:39,760 --> 00:11:45,760
one of the biggest problems that we have is that we've got multiple people

161
00:11:43,279 --> 00:11:49,839
working off of that same shared nas all at the same time

162
00:11:47,200 --> 00:11:54,320
so this is where the 10 gigabit connections that

163
00:11:51,680 --> 00:11:58,720
our editors are also using comes in when we're scrubbing through footage we can

164
00:11:56,079 --> 00:12:04,160
actually see data rates in excess of two and a half gigabit per second so we

165
00:12:00,399 --> 00:12:06,000
peaked at just shy of four gigabit per

166
00:12:04,160 --> 00:12:12,240
second that my friends is why we've got the bang and

167
00:12:09,120 --> 00:12:14,000
nas with the high speed networking

168
00:12:12,240 --> 00:12:18,480
so once the edit is done taran or one of our other editors will export the entire

169
00:12:16,000 --> 00:12:24,959
project as an mov but those are extremely inefficient

170
00:12:22,639 --> 00:12:28,880
files so they're they're great quality but they're absolutely massive so he's

171
00:12:27,360 --> 00:12:32,800
going to copy that goes into the transcode flow plane yep and then you

172
00:12:31,360 --> 00:12:37,120
paste it here now what's going to happen there is that

173
00:12:35,360 --> 00:12:41,440
water cooled server that i built a while ago the one that's at the very bottom of

174
00:12:38,720 --> 00:12:46,160
the rack is going to see that that file got transferred and then it's going to

175
00:12:43,600 --> 00:12:50,560
spit out the correct formats for all the different platforms that we upload to

176
00:12:47,680 --> 00:12:55,040
whether it be youtube or Floatplane or facebook or whatever the case may be

177
00:12:53,120 --> 00:12:59,920
that way the editor's machine doesn't get tied up with this so this is cool

178
00:12:57,519 --> 00:13:04,480
once that file's done copying it does take a second and media encoder is not

179
00:13:03,120 --> 00:13:09,200
always perfect before it'll pick it up but we use vnc

180
00:13:07,440 --> 00:13:12,639
in order to remote into that machine so everyone can go in at the same time

181
00:13:10,639 --> 00:13:16,480
unlike remote desktop connection and make sure that it has actually picked up

182
00:13:14,320 --> 00:13:20,880
the project and that it's transcoding to the correct format okay cool the system

183
00:13:19,600 --> 00:13:25,680
works so thanks for watching guys if you disliked this video you can hit that

184
00:13:24,160 --> 00:13:30,480
button but if you liked it hit like get subscribed or maybe consider checking out where to buy the stuff we featured

185
00:13:28,639 --> 00:13:33,680
at the link in the video description also down there is our merch store which

186
00:13:32,000 --> 00:13:38,639
has cool shirts like this one and our community forum which you should totally

187
00:13:35,760 --> 00:13:38,639
join