why the heck do you guys need a whole room full of servers
just to make youtube videos we get asked this
and actually a lot of other questions about our data management and our
workflow all the time so for the last
year actually ever since we got our red cameras i've been meaning to do an
update for you guys on how we handle such heavy footage
how we share resources with such a large team and how we do all of that while
maintaining our rigorous daily release schedule so
when jiwin reached out to us asking us to do a
feature on their smooth 4 camera gimbal i thought well hey there's an
opportunity why don't we grab this thing
and take it with us while i show you guys the way that data flows through
Linus media group to become a video on your screen
so while i let you all appreciate the irony of a video about our overly
complicated workflow shot on a cell phone with a simple handheld gimbal i'll
tell you guys a little bit more about the zoom four so it's got a highly
integrated control panel down here with apps for both iOS and Android that allow
you to control your camera it's got focus pull and zoom capabilities it's
got the ability to do vertigo shots with one button through the app or manually
it's got what they call phone go mode for instant scene transitions and it's
got both a quick standby mode and a one-click mode switch
everything that we shot outside of the intro and this outro right now was
filmed on this it's got a note 8 on it right now but it was shot on an iphone
10 and i'll let you guys be the judge of the stabilized footage that comes off of
it so to demonstrate some of the challenges with working with red footage we're
actually going to take that intro that we just shot just now and we're going to
ingest the whole thing to show you guys how we do it now red cameras
are not great for recording audio in fact even with an external preamp it is
really difficult to get anything resembling usable audio out of the
camera directly so recently we finally gave in and resorted to an
external audio recorder it really sucks to synchronize audio
especially if you're stuck using the scratch audio that gets recorded
directly to the camera and then the sd card out of your external recorder and
aligning it manually but we recently got a little doodad called a tentacle sink
you get two pieces here a master that sits on our mix pre-6 and a slave that
sits on our camera you synchronize them at the beginning of the day and what
they do is they inject what's called time code into the mag and into the sd
card so that you can easily synchronize them later on down the line
but that is far from the only problem so the
thing about the weapon 8k so that's our camera recently renamed the
dsmc2 helium or something like that
is that it records footage they're both the same camera at up to 8k
60 frames per second and with the 480 or 960 gig mags that's
up to 300 megabytes per second so to put
that in perspective if we were filming a simple TechLinked episode even though
we will usually use like a 20 to 1 compression ratio so that's about four
times the compression that they would probably use on something like a feature
film that means that a 10 gig clip could
still end up being in the neighborhood of about did i say 10 gig clip
that means that a 10 minute clip could still end up being in the neighborhood
of around 50 gigs and while it's less of
an issue for something like Linus tech tips or Techquickie when you're
recording dailies and you're expected to release the video on the same day that
you film it like with something like a new show
size of files ends up becoming a potential problem so that's why our two
ingestations which you might recognize from one of our tech showdown episodes
are both equipped with 10 gigabit network cards so one of them has this
ASUS one which is basically the last generation version of this action one
they're both using aquantia chips that allow them to be somehow under a hundred
dollars for 10 gig networking there are a number of different ways that we
ingest footage depending on the type of camera we're using so for red you can
see each of our video clips is actually a folder made up of smaller broken up
video clips so we're going to go ahead and grab that and copy the whole thing into our
project folder for this video so that goes in a roll
one what we're looking at right here is
probably some kind of like weird buffering thing or something but this is
about what we'd expect to see in terms of our ingest speeds
it's not the full 10 gigabit because that would be in the neighborhood of one
to 1.1 gigabytes per second but
the reality of it is that you're going to run into bottlenecks elsewhere and in
fact these red megs even though we're using an esata interface for them are
just not that fast and they only read at about 230 megs a second
then what we got to do is grab the audio clip off of our sd card that goes into
our audio folder right here and that's pretty much the whole process
for red footage but for that we wouldn't need powerful
machines and there is a reason that each of these is running 64 gigs of RAM and a
10 core extreme edition processor and that's because for our other cameras
like the a7s for example we actually use
adobe prelude which is an imperfect
piece of software but has its uses
to transcode the footage when we're bringing it in so what we do is we take
whatever project it is and this is generally fast as possible we still shoot that on the a7s we create a
subfolder and we set it to transcode to cineform 4k this improves our timeline
performance when we're scrubbing through the clips in adobe premiere the other
thing that we do is set a second destination for the original files that
we're not transcoding just in case something goes wrong with the transcode
which does happen from time to time and we need to go back and grab the original
files now just to give you some idea even if
we're not using the full potential of this network connection all the time of
what it's capable of let's go ahead and just grab a file
off the desktop and show you just what this puppy can do
so that is saturating the read speeds of
the raid 1 ssds that each of these machines are booted off of so that was a
10 Gigabyte file let's go have a look on the other end of this Ethernet cable at
how exactly that whole thing comes together now i've shown you guys our
server room a fair number of times but i
haven't given you guys an update in quite a while
on how exactly it's working in here
so the main server that everyone is editing off of at the same time
is this one right here this is wanik server and it's running uh 24 plus one
two three four five six seven so it's running 31
Intel 750 series 1.1 terabyte NVMe ssds
and this guy is an absolute monster
so if we fire up performance monitor you can see that even though we've got
four of our editors in office right now and we're seeing each of them doing 150
120 50 megabytes a second of reads this
thing is barely suffering and our
there we go our z disc q depth is only about 0.5 that's the benefit of NVMe is
that nice fast responsiveness and in fact when we were still using our SATA
SSD server for everyone we were starting to run into issues once we had the room
fully staffed over there where adobe premiere would crash but not just crash
everyone's would crash at the same time and we narrowed that down to slow
response times on our nas so this guy's
connected at 40 gigabits per second
this one right here has 24 SATA ssds and
this one only gets used this is called qq server this one only gets used for
large projects that only one person needs to work on at a time it's still
SSD based so we still don't run into any crashes or any other weird issues like
that but it doesn't have quite the snap that
an NVMe machine does now something that we learned
a really valuable lesson about i guess it must have been about two
years ago was real time data replication and
off-site backup and that is where this guy comes in so i want to show you guys
a little trick here i don't know if i've ever actually done
this demo in a video before i've shown people that have come into our office
so i'm going to use my test folder here and i'm going to create a document
called uh test for video right here this is a text file
on our main drive then i'm going to jump over to this hard drive based server
under it so this is running eight eight terabyte drives for a total of 64
terabytes and i'm just going to pull up the
wanik sink folder
get that test folder open test video but
just like that so within about five to ten seconds whether it's a text file or
whether it's a large video file the synchronization software that we're
running here we'll automatically dump it over to here so if this crashes we can
actually continue editing videos within about 10 to 15 minutes of switching over
everyone's map drives and another benefit that it gives us is that
normally on a network drive when you delete a file it's just gone
but instead check this out if i delete test for video i'm actually
going to have to do it from a separate server but if i delete test for video
boop you're gone you're done
and i go into here that file is going to disappear but
what will happen is we can go into the deletions folder and we can rescue it
this has actually saved our butts
more times than i would like to admit because
stuff does get accidentally deleted now
all of this is only for active projects that we are
working on right now once we're done with something it goes
on to petabyte project and we're actually running both phases of it right
now when we originally deployed it we were only using one of them so we could
just save power on hours for all the drives that were in the other one
not the case anymore so petabyte project is now up to
777 terabytes of total space of which
we've consumed 432. this holds all of
our archived projects so that in the event that we want to make a video that
refers back to one of our other videos
we can grab the original quality files rather than downloading off youtube like
a lot of other youtubers do bringing us into this room one of the biggest
challenges is not just working with red footage because the files are so heavy
one of the biggest problems that we have is that we've got multiple people
working off of that same shared nas all at the same time
so this is where the 10 gigabit connections that
our editors are also using comes in when we're scrubbing through footage we can
actually see data rates in excess of two and a half gigabit per second so we
peaked at just shy of four gigabit per
second that my friends is why we've got the bang and
nas with the high speed networking
so once the edit is done taran or one of our other editors will export the entire
project as an mov but those are extremely inefficient
files so they're they're great quality but they're absolutely massive so he's
going to copy that goes into the transcode flow plane yep and then you
paste it here now what's going to happen there is that
water cooled server that i built a while ago the one that's at the very bottom of
the rack is going to see that that file got transferred and then it's going to
spit out the correct formats for all the different platforms that we upload to
whether it be youtube or Floatplane or facebook or whatever the case may be
that way the editor's machine doesn't get tied up with this so this is cool
once that file's done copying it does take a second and media encoder is not
always perfect before it'll pick it up but we use vnc
in order to remote into that machine so everyone can go in at the same time
unlike remote desktop connection and make sure that it has actually picked up
the project and that it's transcoding to the correct format okay cool the system
works so thanks for watching guys if you disliked this video you can hit that
button but if you liked it hit like get subscribed or maybe consider checking out where to buy the stuff we featured
at the link in the video description also down there is our merch store which
has cool shirts like this one and our community forum which you should totally
join