this is the million dooll PC a very
expensive collection of servers that are linked together with an incredible piece
of software called WCA that allows them to act like one giant storage server
with the ability to reach speeds measured in the 100s of gigabytes per
second watch this
and damn boy wa a second that's a little
faster than it used to be that's because the million dooll PC has
glowed up a little bit we've scaled from
six to nine storage machines giving us a
whopping 2.2 pedabytes of raw kokia
powered PCIe Gen 4 storage it's almost
as much storage as we have on hard drives in our mother Vault but orders of
magnitude faster which means it's also
really power hungry which means it's also really hot even with 3 tons of air
conditioning in here it's still 40 flipping de which is terrible for
hardware and terrible for any humans who need to work in here now we could add
more air conditioning but that would be even more expensive than this already is
you know what's not expensive the not one but six APC Tower coolers that we
inherited for free from a local data center that's right folks we are doing
whole server room water cooling and this
time I swear we are going to do it right so where are the plumbers well we're not
we're not really doing it right today we'll do it right later that's fine
right we're just going to DIY that is not what I had in mind when I sold this
to our sponsor build Redux they make it easy to get into the world of PC gaming
just pick your games see the performance and the pros at build Redux will build
the PC for you visit build Redux at the link down below now under normal
circumstances you would run chilled water in the unit in your data center
and use that chilled water to cool the air but chilling the water costs money
and that's what we're trying to avoid right so since we also spend money
heating our warehouse and we have so many of these things kicking around
anyway we were thinking why not cool the
server room and heat the warehouse in the winter turning this one into less of
a cooling tower and more of a heating Tower yeah it was designed for that
right I mean this should work great like in theory this is a mint plan in
practice well I guess we're about to find out yeah in practice we have a
giant cooling tower on a forklift on a not storage rated uh
mezzanine water's not heavy 3 2 1 oh
okay yeah that feels great where do we want it exactly in
theory it blows hot out that side mhm so
probably pointing that way sure that might not be intuitive because this is
obviously the front of the unit so you would think that's where it draws air in
but in a data center deployment these cooling towers sit in between the
servers and servers will typically pull cold air in from the cold aisle in front
of them and then heat up the air and blow it out the back so by having these
towers in between your servers what
happens is that hot air comes out gets sucked into the cooling tower and then
gets blown through this Radiator in the middle blown out the front and sucked
back into the server let's put it in the corner pointing that way Hey Jake yeah
what I figured out what one of the things we thought might be a temp probe
before is what it must have gotten jostled loose at some point but these
guys right here they have little floats on them oh they're the like the water
level they're for an excess condensation meter yeah when you're running chilled
water through them they can start to condensate pretty aggressively we're not
doing that so it should be fine yeah I saw the drip trays before and I also saw
that there was rust everywhere in the bottom of the units but I didn't realize
that that was by Design as cool as it is is though that this unit is more intact
it also means that we haven't modified the fittings yet so well you did such a
great job last time so uh you want to have a chance to try to do such a great
job too oh okay boom Uno
reverse see you later before we solder on those fittings I want to give you
guys a bit of a background the tubing inside the unit is 1 in which is a
pretty standard size which is nice um except what I found out getting 1in
tubing it's not that common and also all
of the fittings are very expensive and since this is like our DIY approach even
though my original plan was to use garden hoses uh I found out garden hoses
are really expensive so we might as well use the stuff we're going to use in the
long term and since we're going to have to take it down anyways we're going to
use shark bites and shark bites are really cool they're basically like you
push the tubing in and that's all you do it's now a fixed it's warrantied for 25
years it's it's sick they're just really really
expensive that's smells delicious it was taking too
long one and a two you got to make sure it's
clean before you solder otherwise the solder doesn't stick at least that's
what Plumbing tick Tok told me naturally this is going to be what you'd call a
closed loop system meaning it's filled with a certain amount of water and
closed off but only to a point when you have Radiant Heating in your house for
instance there is some amount of evaporative loss so you fill the coolant
up and over time slowly you lose some of
it the way that they get around that is by using a check valve that's basically just hooked up to your House's water
pressure which automatically as some evaporates a little bit comes into fill
I didn't really want to do that because I want to do it today and I'm not a plumber but fortunately our mechanical
room is right below us and it has a hose bib so we were like why don't we just
hook the hose bib up to the tank but then if you just turn it on the Tank's
going to overflow and explode and whatever it's like how can we fill a
tank I could like design a custom system
or I could just buy a toilet fill valve we have this tank from McMaster Car
which we're going to add fittings to with these fancy bulkhead fittings but
in the meantime this has got to come out the bottom which means we need to drill
a
hole it just knew the job was done I bet you this stuff is tasty there's Supply
sparingly oh oops we're putting the
threaded ones on here we'll solder those and then twist the female shark bites on
and then the tubing can just go like that in theory a she's mint
Bud it occurs to me we never tested this thing I mean we have spare parts it
would probably be faster just to pull another one up here right Jamie oh yeah
absolutely luckily there just so happens to be the exact plug we need right here
20 amp 120 volt we got some adapters and extension cords from infinite cables
thanks guys pluggy plug hey you hear that
that's the condensation pump running at the start that's not too loud right sounds
great it's actually not that heavy when there's nothing in it as he struggles to
move it what's the worst that could happen I drop the
thing mint let's go over this one more time pump right below the reservoir that
way it gets fed directly we had a problem trying to Prime in at Linus's house with the pool trying to avoid that
then it will go down to the cooler come all the way back up blah blah blah into
our heater what I'm calling it and then
out the heater into a filter before the reservoir so we catch all the gunk
before the reservoir reservoir back to the pump and the Vicious Cycle continues
cool are we expecting a lot of Gunk no I feel like we should have also filtered
the water coming in we'll put a valve down here so that
when the water comes in well at the start we can close this and the water
will go into the reservoir which will make it easier to bleed then once the
system is running we just open this valve
and most of the water will just go directly into the pump and kind of ignore the reservoir when it's coming
back up here that way we don't have like the full force of the pump just like
gushing into the reservoir constantly making a bunch of noise I wish we had
like straps this Mak this much easier unless we use the LT Cable Management
things to this metal pipe oh hell yeah
the reason that we're putting valves here and actually at every piece is a so
we can service a piece individually you shut off the valves and then we can take
it out and swap it or change it or replace it whatever we're doing and B
because it's like like I said before this is all going to get redone properly and I want it to be easy for him to just
take apart whoa it's like it's designed
for dude it's mint male a te and a valve
and a valve yeah this isn't going to leak no it's going to be great they're
shark bites sick before we had this Grand idea
of water cooling the server room we tried to mitigate this problem by adding
a duct and a fan which was pretty easy to do because right where we are right
here is about 10 ft from the server room
and there's just a hallway so we added like this insulated duct between and a
big fan and it did help but ultimately once it gets up to equilibrium it wasn't
enough hey it's through oh that wasn't
even that hard huh huh mint we forgot
about the filter I realized Alex is putting this all together I'm like oh yeah where are we putting the filter
damn it right where Alex is
working oh no brother brother did they
sell me the mother oh my God why
didn't I check this no filter it
is since a long-term plan is to rip the RADS out of these giant units and just
stick it right on the back of the rack so it captures the heat directly uh I
don't care that much about where this goes since it's probably only going to be here for a couple months later so
we're thinking we just leave it up against the back here that way the hot air comes out get get sucked in on the
back and shoots out cool air in the front and because of that we're thinking
we just leave the back door off and then I can just pull this tubing kind
of right in there beautiful look at that
that's mint good job Alex I'm excited to
see if it leaks or not the great thing is we won't know until it's far too late
on the bottom side yeah oh shut it off
here's the problem we're running into this tank is thick down the middle
Middle where the seam is and when this is sitting in here it looks good like
that except when you tighten it down that extra material pushes it out on an
angle and now the seals can't seal I walk back into this project and I'm
immediately told well it leaked um oh who whoa whoa be careful there's
fittings on the bottom but we have a fix for the leak and it's glue no not that
not that it's the the ooo this stuff is like $40 a tube it
does not leak it's a nice non hardening
so it stays kind of plasticky and goopy solution silicone yeah I wasn't able to
do it exactly how I was hoping because it's still kind of cockeyed but I don't
think it's going to leak I put I put a lot in there but I have a job for you
cool I uh spent some of your money and got the wrong thing so we have to do it
again if I had a dollar for every time he said that wasn't I would be able to
afford all these mistakes what are we building uh a filter housing they sold
me the wrong size of PVC even though I was like I just need to connect these
two things and they were like okay we're going to give you one inch PVC because it's one inch well it's supposed to be
inch and a quarter cool uh so I have 20t of 1in PVC that I have no use for and
then we went to the store they were like okay we could we can tell you this it'll
probably be long enough so you have to cut the threads off and then cut it in
half and hopefully that is like enough oh man
that's going to be uh you have one shot at this don't screw it up and this is my
saw yeah look this is what the professionals told us to buy they might
have told us the wrong thing last time but here we are again surely that
wouldn't happen twice no got the
clamps not really what this is for but it's working I cut off part of the
threads I sanded down the little nubbins now they fit in here which is why I was
doing that by the way really uh oh wait
oh balls we have to cut this down the middle too okay well um I have to go do
a conference call so what I mean you only got one thing done and so you
didn't even get it done I don't know if you're getting your minutely bonus
minutely
bonus look at that worky oh nice yeah I
didn't know that this glue set up in 10 minutes there's your filter perfect
um damn it I looked in the manual for
the cooling tower things and I did randomly stumble upon uh them mentioning
a filter and they said a 20 mesh and I think this is 100 so it's a lot finer
should be fine as long as our pump has enough power for it it doesn't really
matter it I mean it's not that fine yeah also it's still it's still like a mint
for 1in tubing so it should be fine now we just have to find out where we can
put it now we just have to pose with it like not everyone's a poser don't
worry can you walk me through what's going on here a little bit because I
missed the part where you guys drilled holes in the building we've got the cooling tower over there we've got the
tubes coming off of that and we already have the tubes in the server room
already hooked up we had previously cut a hole for our ducting solution so we
just stuffed the tubes down the duct that goes through the like hallway there
into the Ser oh wow and these awesome
Cable Management things the magnet is so strong that it just it's kind of
perfect in terms of pump this is not our end game this is like a cast iron pump
and we're using oxyard tubing which it
limits the amount of oxygen it's probably f look your house is probably
the same cast iron pump in it for like 20 years and it's fine but it isn't the
end game I picked up a stainless version
of the one step up from this pump this one I think does around 20 gallons per
minute and the big boy that we're getting does around 40 wow which is a
lot of gallons a lot of gallons per minute okay are we ready to hook this up
in theory yeah I mean let's do it we tried before so let's try again hold on
before you do that that tube is plugged into the wrong thing on the top of that
oh did you guys anything right on the first attempt or that is a excellent
question and we decided to not Mount the filter up
there he always chirps me for doing things like kind of the easy way and
then fixing it later and he's literally just like purposely doing that right now
well we're just not going to fix it later it's fine I'm going to fix it later what no don't waste time on that
it's fine well you I'm not the one that wasted time don't move it it doesn't
matter you told me it doesn't matter I'm going to send you a video of me moving
it later no that is going to be a huge waste of time cuz you'll have to
disconnect the tubes in order to you could have just fixed it right now let's
put the tubes in come on I'm coming back here with a rotary tool I'm actually
going to shave off the tops of these screws let's give ourselves a little bit
of breathing a lot of breathing room that's fine I like to breathe a lot we
we know I plan to keep doing
it easy peasy by the way I found an easier way to do this just take the
white thing out first oh okay okay this is bothering me do you hear that now is
everyone hearing that I don't know what you're talking about every just said it's bothering me editor's note cut out
this entire story line okay it's on the wall filtered what I dropped one of the
little shark bite white thingies and it fell down the hole hey
lonus no you
suck oh my God Jake did you turn the water on no why is it wet I don't
know where's the water shut off is this
valve broken I I was dumping water down the hole oh
my God you suck do you want me to turn the water on okay that actually does
pour down into an electrical box so you really shouldn't do that oh what really
uh yeah I I honestly have no idea where
the shark bite thing went
taada what oh she's on no it's in it's
in somebody needs to go in the server room and watch okay in case that leaks
because if that starts leaking it's going to be a while before it all stops
leaking ooh
dick ah a dick I get it no stop you a
his first is my loging in high school was a. dick okay and then you want it
one click apparently right 7/8 inch ballcock let's turn this thing on yeah
go for it see you later okay it's on I I
got water coming that's good do it full send let's go I can't believe this tank
is not leak are you ready Mr Dick I ready it's
coming whoa it sell fills how's it looking why didn't we do this at my
house this is awesome no idea it's actually great you know V2
right is it fine yeah like everything's fine down there Jake we have a leak oh
God uh where uh on on the uh the return
fitting no we got it we got it I got it I tell you guys standing right at the
back of the air output of these servers it's
exhausting Alex Alex Alex he may be a
dick but he's our dick this is the
world's slowest leak at this point ready one more and nice note that every
fitting I did is not leaking this is the one you did I didn't do that one yeah
you did you stuck the tote I don't know what you're talking about we managed to fix our leaks I replaced the fittings
down here on both sides I think we just used Too Much Teflon tape uh the first
time we hooked it up and it kind of just like bunged up the threads and then
upstairs I just cut the PEX sanded it down stuck it back in and everything's
good no more leaks now I've got the million dollar PC hooked up and running
again and I want to test something before Linus gets here uh one of the big
things you want to do in a setup like this especially if you were doing this
in a data center is contain the hot air so we're going to make ourselves a
little bit of a hot aisle with some cardboard right now it says 31 is De on
the intake maybe it's 31.1 31.2 let's see if it goes up any Higher by
just adding the cardboard I want to just kind of proof of concept before I spend
the time doing a better job of this obviously still a giant Gap here but
let's see if it makes a difference oh 31.3 31.4 it's going
up5 6 okay maybe it's only like half a degree but half a degree is important
because the whole game here is temperature differential the larger we
can make the difference in temperature on this side to the temperature out in
the warehouse the better this system will work hey that's not
bad we have our hot aisle which is going through the cooler and the rest of the
server room is our cold aisle it's not a perfect science but it's better than
when it was open oh oh god look I wanted optimal
circumstances for our very scientific test it's definitely hot how do you like
it I have so many questions that's great it's hot in here hey it's really hot hot
in here but would you believe me if I told you the air conditioning's off
really it's 32° in here don't get me wrong it is hot but the AC is off off
and this has been running for like hours
that's actually kind of cool so look at the UPS we're drawing 4.95 Kow except no
we're not half of these servers are hooked up to the Main's power so we're
probably drawing more like probably like 8 to 10 Kow however the can we turn the
AC on yes aren't we going to be taking
cold air that's been heated up a little
and running it through our heat exchanger potentially cooling the
warehouse look I don't know if it's going to get that cold uh I don't know
if you've stood Behind these things at all they're hot it's quite hot the exit
temperature of these servers is probably like 40° even with air condition air
coming through it but you are right it will be colder that being said before
when these actually running with just the air conditioning it was 40° in here
I think we should turn that off and see how hot it gets in here okay cuz it's
been 32 for over an hour this is a cool software called check MK it's a
monitoring tool that's open source very awesome we've hooked up the systems with
SNMP which is a monitoring protocol and you can see the coolant coming in is 28°
from coming from the warehouse yes after it's been cooled with air that's around
21° and then and leaving here it's 32°
that's a 4° difference that's a lot of
heat it doesn't sound like a crazy number but at the fluid flow rate that
we've got that's a freak ton of heat yeah clearly we have a problem of it not
quite getting cooled over there as much but I had a brilliant idea we have six
of the bloody things let's just put another up there we could why not or
hear me out we bury a coil in the parking lot oh I'm totally down for that
give me the approval I will find a contractor today that is not that's
above my pay grade these days oh I got to ask paren yes hi Money free cooling
lus said it was a good idea anyway yeah we could throw another one up there for
sure like pretty easily we just actually need two T's that's it interestingly too
the return versus Supply Air is a 3°
difference right now so the air coming into the back is almost 40° 37 and then
the air coming out of the front is 33.6 they are still going up a bit but like I
said it's been at 30 2 for an hour uh I
imagine if we left this overnight we probably come back and it might be like
34 or 35 in here but considering we're
not air conditioning this at all let's see what happens if we turn it off oh
God I don't think the air conditioning is going to make that much of a
difference in terms of the air coming out of here the metal on the back of
this rack is up to 46° the air coming out of the back of
this power supply is as high as
60° oh 76 it's uh 33 .2 now this may not
be safe for this hardware for much longer it's 33.3 now it's gone up a full
degree in like 3 minutes it's getting warmer
34.1 I'm getting quite uncomfortable I think what I want to see now is if the
air conditioner can actually work cooperatively with the heat exchanger to
get it down to like normal temperature in here yeah okay all right let's do it
so our Baseline is when we started this project it was hitting about 40° in here
with just the air conditioner running if we can beat that it means that somehow
through the magic of thermodynamics they are working together 34 and2 plug the
goddamn thing back in a sweet relief so our Peak was
42 with the AC off and now it's come down
to 37 37.8 and up here is equally as mint
you like my sound ding yeah I mean it'll do something not much in this config
that's what I was going for is something something long term we get sheet of
plywood and attach the rock wall on the inside of it to kind of make a baffle if
I stand here it's pretty loud but if I go behind this
thing it's a pretty substantial difference we just have to create kind
of a a sound blocking maze that air can
move through but um block sound from reaching people below us and it's also
at like 80% fan right now if we put a second one up here and run them like
half the speed should be tolerable all right wait we should look at the filter
oh is there shoo in there not too bad and we're back so not as good as it was
before no it says 1.8 th000 Watts on the
readout which oh no which is a lot given
the amount of power that we're putting into it what one pump in some fans huh
22° in here right now Supply versus return air is still like a Delta of more
than 2° that's solid and interestingly the cooling
temperature is lower but what I realized
look at this the fan speed is lower oh
what I also realized after looking at this is that there's a valve I remember
when you guys did the thread R cooling there's a manual valve right that's like
the bypass valve and you guys were like oh you're supposed to close it so it doesn't bypass actually you're supposed
to leave it open because there's a mixing valve in the unit for your
application actually probably just shut it doesn't matter but the thing can
dynamically adjust how much water goes through the radiator and how much gets
bypassed I think to maintain to maintain a Delta ideal temperature yeah it's
totally fine in here now it's working I'm surprised it works this well well
the ducting was key but the other big thing is I think that we were just on
the edge of this air conditioning unit's capacity so just taking the edge off
taking another thousand couple thousand Watts out of here and what's cool is
because we've got it hooked up direct directly to the exhaust from this rack
adding more to this rack is going to increase that Inlet air temperature
which increases the efficiency of the thermal transfer of our heat exchanger
mission accomplished yeah I'm actually kind of shocked I mean 2° cut off the
exhaust temp doesn't sound like a lot but that's like already outside of the
server and we're talking 2° and a lot of air oh I can't wait for part two it's
going be so much better when this is directly on the rack I'm thinking we put
two on the mezzanine for like winter time but we could also put one like
outside somewhere we could have valves and we could decide where we want to
dump heat from this room we could automate them pretty easily too we got
have electric Valves and a couple temp sensors and a Ras pie so we can send the
heat outside when it's hot in the warehouse and send it inside when we
want to heat the warehouse yeah and interestingly we could put it in that
mechanical room and just like cut a hole in the one put a big thing yep it's our
mechanical one yeah what's the landlord going to do about it screw that guy and screw the segue to our sponsor
Squarespace creating your own website doesn't have to be difficult and luckily
with Squarespace it isn't they're all in one platform makes it easy to get your
website up and running quickly and designing with their fluid engine site
Builder is easy it's so easy start with
a template and customize the details to your heart's desire with intuitive drag
and drop elements on both desktop and mobile you can also use their asset
library to manage all your files from one Central Hub and use them across the
entire Squarespace platform plus with squarespace's analytic insights you can
see what's working well and What needs a little TLC just a little
bit you know and if you need help Squarespace has got your back with
helpful guides and a 247 support team they do not sleep head to
squarespace.com and get 10% off today if you guys
enjoyed this video why not check out the time we did this at home with radiators
from PT Cruisers oh yeah yeah this would have worked a lot better yeah but those
work pretty good you could just take one of these home no they work great I'm not
changing anything oh yeah okay ducting I could duct to the PT Cruiser radiators
but that's all I would change oh my God we're getting that elu like 1 M by 1 m x
1 M 3D printer and we have a 3D scanner they could 3D scan the back of your rack
and make like a a duct shroud look he's actually like considering it right now
you were thinking cardboard but we could do it in here oh yeah totally cuz we
could take the AC straight to the front of this rack we want some over here too
I guess if the ambient is low enough it's probably fine it'll be fine yeah
okay