today is the day we are finally finishing the 100 000-ish
dollar six editors one CPU machine
it's been a freaking long time coming oh come on so we won't make you guys wait
that's all looking peachy
any longer now this should work and it should show up over here hey
to hear this word from our sponsor pebble host are you in the market for a
hosting provider try pebble host they balance price and performance with vps
hosting starting at 275 a gig and game hosting at one dollar a gig use code LTT
at the link below to save 15 percent
at this point those of you who have been keeping up with the 100 000 pc saga
have probably seen this machine put together and then torn apart again
four or five times so it's safe to say
that we've already dry fitted everything and we know it all pretty much works
together so just like the last video in this series we've stuck with the god tier tie
end s7 100 ex motherboard
yes it's a little ugly because aesthetics and RGB aren't exactly top
priorities for server manufacturers yet
you give it time but it's got it where it counts because the
thing just works speaking of working the original cpus
that Intel supplied for this build are of course no longer working
i dropped one of them and we weren't able to source a matching retail xeon
8180 platinum replacement without spending nearly 10 000 us dollars
remember guys you need to use two of the exact same model in most dual socket
systems so we turned to ebay and obtained a pair of
qs or qualification sample 8180s for
less than half of what we'd have spent for a single retail one now qs chips
aren't perfect but unlike engineering
samples which may be less reliable or have significantly lower clock speeds
qs chips are the ones that are sent out to oems just before a particular model
launches for their final validation and testing
what that means is that in many instances they are practically speaking
or even literally exactly the same as a
retail chip save for the labeling on the integrated heat spreader
so we're hoping that these should work perfectly for our little experiment
especially with an ek annihilator CPU block keeping each one of them cool
man those are industrial looking for our RAM we stuck with the 384 gigs
of ecc ddr4 memory running in a six
channel configuration that our friends from kingston sent over
almost a year ago so this is going to give us about 60 gigs of memory for each
of our six virtualized 8k video editing
workstations with a little bit of leftover RAM for our hypervisor
now our gpus are of course the same six
NVIDIA titan v 12 gig hbm2 cards that
we've been working with since earlier in the project
each one of our virtual editing stations is going to get one of these fully
dedicated to it now because we're using GPU passthrough as far as our editor is
concerned it's like that card is installed directly into their own
computer now for this video we've done something special with our cards
each one of them was outfitted with one of these beautiful acetal slash nickel
ek water blocks and then made it to a
custom six GPU terminal which might give you guys some of those
sweet nostalgia feels from all the way back in seven gamers one CPU
massive thanks to ek by the way for going out of their way to make this
thing happen because dang this looks awesome
now to dissipate the up to 2
000 watts of heat that our cpus and gpus
will kick out under full load and remember that's without overclocking we
settled on a full complement of ek waterblox's ultra thick class radiators
we've got quad 140 millimeter units for both the top and the bottom as well as a
triple fan model on the front now we did have a bit of trouble mounting our 140
millimeter Noctua fans as the clearances are just slightly off with these
radiators but thanks to some space under the top panel we made it work now even
with this configuration we're not a hundred percent sure that we've got
enough cooling capacity problem is we kind of had to roll with
it since case Labs the company that makes this monster chassis
uh excuse me i misspoke i should have said made this monster chassis because
they went out of business while we were working on this project
so we really can't get any more parts for it
not that i think there would have been anywhere to mount them because i think
we've managed to build one of the few machines in existence that truly needs a
case of this magnitude to find out why let's have a look at the crown jewel of
our build this pci express expansion daughter board courtesy of one stop
systems so this bad boy or girl whatever it is
is gonna allow us to take the bottom PCIe 16x slot on our motherboard and
then break it out into eight more slots
for a total capacity of up to 14
expansion cards take that mac pro
what this gives us is two individually hot pluggable usb ports for each of our
editing machines and a melanox 100 gigabit Ethernet card
to pull all that juicy 8k footage off of
our main editing server which is also equipped with one of melanox's
high-speed network cards now we're only able to run our cards at 40 gigabit
because we don't have a 100 gigabit network switch
or enough pci express bandwidth anyway
but that should still be plenty for six 8k raw streams so the thing is here that
the daughter board wasn't necessary per se it's just that without it we would
have had to use a bunch of usb hubs and assign each usb device individually to
our editors meaning that any changes would require reconfiguration and unraid
and a reboot of the virtual machine so
yeah it's mostly a creature comfort but i think it's super cool because it means
that for all intents and purposes our editors might as well be sitting at a
completely normal computer now mounting this thing was a bit hard
to watch not because Jake doesn't know his way around a dremel or anything but
it's more just that this is a 500 us dollar case
and more importantly you straight up can't even buy them anymore
thankfully it turned out pretty dang clean we actually ended up stealing the
pci express slots from a thermaltake tower 900 and it worked perfectly for
the eight cards that we needed to install
then we mounted the daughter board with some upside down bolts threaded through
the bottom of the case and it was off to the races
now as for the rest of the trunk over here we ended up going with dual evga
1600 watt t2 80 plus titanium power
supplies with some sexy cable mod cables now i'm one of the first people to point
out that one power supply of this magnitude is
totally unnecessary but this is not exactly a normal
computer remember guys even just the
gpus in this system could be too much
for a single power supply 1600 watts or
otherwise so for us then two of these bad boys
fits the bill nicely and should even give us plenty of headroom for capacitor
aging or even overclocking think of it kind of like being that guy at the car
show with the nitrous bottle in their trunk except the nitrous is a second power
supply and rather than being outside they're in
their mom's basement speaking of horsepower adders the eight 10 terabyte
nas drives that we have installed in the front of the case can be used to assign
local bulk storage to our virtual machines and we'll be using our dual
Intel 905p series optane ssds as a right
cache for our magnetic drives as well as
for raw high-speed storage for the Windows installs for each of our editor
now the reason we used optane for this is that when you're hitting a drive with
six users at a time booting up or launching programs even a
high-speed NVMe SSD can struggle well
optane has much faster access times so we're hoping that to our editors this
entire virtualization experience should be transparent
although we haven't actually tested that yet now with the build mostly dry
assembled at this point we could finally get to running our tubes yes we went
with soft tubing so no it's not the
sexiest machine in the world but with a good chunk of ek water blocks
nickel fittings and a little bit of creativity we still think it looks
pretty good not in a blinged out RGB
gaming rig kind of way but in more of like a
this thing does real work kind of way and it has the added benefit of making
it much easier to reconfigure the system if we ever decide to upgrade it in the
future so from there just add water and a little bit of
patience and wa-bam
finished system
of course we're not done yet though i've been out of the office for a few days
while Jake's been getting things together so
why don't we go check it out all right so for our first power on test we're not
trying to go full ham virtual machines everything so we've actually unplugged
our octane drives and our hard drives and just through a simple
basic SATA SSD with a Windows install on it because
we want to make sure it's all still working after the assembly ordeal
we have fan spin hey
do we have display on the other side nice we want to find out if our cooling
solution is actually adequate before we start though
let's do a good old-fashioned cinebench r15 run
ready there it is boys oh my god
and you're done seven thousand so something that a lot of people don't
know is that just because you have two CPU sockets doesn't mean that even a
heavily multi-threaded application like blender you can see it's using the
entire first CPU it doesn't necessarily mean that it can
utilize the second CPU it just doesn't
quite work that way alright so here we are blender's running our gpus are all
sitting at 75 80 utilization our CPU is
at 92 and all we can do is let it cook for a
bit oh wow
like just this chassis being made out of aluminum just like feels like a bad idea
like there's so much flex to it if only it was like a cheese grater
i don't think you could fit that many cores into cheese grater
ouch yeah this is twice as many cores as new
mac pro just saying the highest end mac bro yeah
that means each one of our editing stations effectively has an eight core
CPU 60 gigs of memory a 100 gig optane drive
and its own titan v and wow they actually booted really fast in spite of
you like starting them rapid fire like that
that's flipping awesome and we're good
you know what no for the purposes of today we're just trying to find out if
it can handle a full load thermally and stability wise so let's forget about
premiere let's just start loading up stress tests on these things
what stress test should we do well let's throw like
i do ida and uh fur mark boom there
all right so here comes our first set of stress tests we're going stress fpu on
ida 64 and GPU stress tests at 1600
by 1200 yep that's better if we don't leave a
couple of CPU cores for fur mark to grab
then it throttles the GPU and we don't use up all the power so
time to do number four i'm having fun
it's finally done all right i'm ready to go with stress test number four
not bad is there a reason you didn't put fans in the bottom two here
no no we could we should yeah because that
would look more you know symmetrical ish you gotta cool these optane ssds yo
actually it's not a terrible idea the fan's still not on on the second power
supply okay so we've had the whole thing running for about five minutes now um
our block of GPU blocks
is noticeably warm to the touch not hot
not uncomfortable but it's warm
these fans that are blowing down
like it's kind of like having a space heater blowing at me from under the
computer same thing up here
and we've actually got some temperature results as well
so it looks like our gpus have settled in right around the 55
to 60 degree mark depending on which ones
we're looking at and then Jake you've got CPU temps right 48 49 across the
board now we're not fully loading the CPU
but i mean we're probably over 80 right
you tell me that looks not bad
we're hitting it pretty hard so that's it guys at long last it's finally working
not only that but our cooling solution is adequate good job Jake
and the thing actually seems to be stable so all that's left now then
is to get a nice long 40 gigabit network
cable plug this thing into our switch and get
six editors set up with full stations
around it and to see if we actually can do it
because this has all been theoretical up until now theoretically if we have eight
CPU cores and a titan v and 60 gigs of RAM should be fine
but it's all theoretical up till now let's see if we can do it six times
ak editing workstations off of a single
tower i can't believe it's finally working
speaking of work i've got some work to do too to tell you guys about our
sponsor for today's video massdrop the massdrop hd6xx headphones is one of the
most successful collaborations that drop.com has ever done they've sold over
70 000 units of these things making them
the all-time best seller on massdrop and it's no wonder they've got a balanced
mid-range natural sounding bass they're open back they sound absolutely
fantastic they're actually very similar to my daily driver hd 650s
they've made a couple of tweaks based on community feedback they've got a 1 8th
inch plug for everyday use and a one-quarter inch adapter for professional use and sennheiser backs
these headphones with their support so check out the link below and get your
hd6xx headphones today
so thanks for watching guys if you disliked this video hey it's on now
you can hit that button but if you liked it hit like get subscribed or maybe consider checking out where to buy the
stuff we featured at the link in the video description also down there is our
merch store which has cool shirts like the one i'm wearing and our community
forum which you should totally join i'm a little bit concerned one of our
displays just went blank um
hey okay the monitor just went to sleep it's all good yeah
yeah it's still working good
so 100 000 computer was based on if we were
going to load it up completely with top end quadro cards which didn't end up
happening so we took some artistic
liberties with the naming of the computer