between the radiation the micro gravity
and the extreme constraints on Power and cooling space is just about the worst
possible place to put a computer can you
guys let me in that's better now if you follow space
research and exploration you probably know that space is just full of
computers like this one so how do they
do that well the short answer is by
replacing them a lot the ISS takes
regular shipments of dozens of laptops at a time which get and this is great
this is a direct quote absolutely
destroyed but not every computer can be
disposable and in 2017 the madlads at
Nasa HP Enterprise and kokia who
sponsored this video collaborated to create spaceborn one the first Edge
Computing server that was intended to run for an extended period of time on
the International Space Station of course it being their first attempt
some uh let's say learning took place and it turned out that the super
capacitors in the ssds were prone to radiation related failure who knew but
since all it was ever meant to do was run benchmarks anyway the mission was
considered a huge success and in 2021
they launched spaceborn 2 whose purpose was to move Beyond proof of concept and
explore practical applications for onst compute especially data analysis using
AI but the story doesn't end there behind me is new space born 2 for
administrative reasons it has the same name and Core specs as last time but it
just took off earlier this year and it features more storage than ever over 130
tabt which is an incredible feat when you consider the design challenges I
mean where do they even install these things look up oh right I guess they
don't really need a ladder do they no but you might
to illustrate why Edge Computing is needed on the ISS let's look at a use
case that's focused on astronaut safety these are the Eva gloves that the crew
members wear during space walks and according to this article from 2016 they
were responsible for half of all space
suit injuries so to ensure their integrity between you es NASA requires
the crew to take hundreds of photographs of them from every angle and then beam
them back to Earth where machine learning is used to analyze them for
scratches or other hazards except for one small problem that data transfer
takes five days but with spaceborne oh I
don't know how about 45 seconds not only
is this a huge time sa but with only a handful of pictures needing to go to
Earth for further analysis spaceborn can free up a significant amount of the
crew's limited network bandwidth for other more interesting things with such
obvious benefits then you got to be wondering why did no one ever try to put
a server on the ISS before the short answer is after seeing how the crew
laptops fared many people thought that they just plane wouldn't work and even
if they did there were a host of other hurdles to clear like the launch okay
this is really cool rocket companies like SpaceX and Northrop Grumman have
Shak test machines that are programmed with profiles that will simulate the
launch conditions of their respective rockets and if you've seen that viral
video of the machine that disassembles hard drives by vigorously shaking them
you're going to know that surviving that kind of treatment is no mean feed well
these machines Managed IT both in the simulation and in the real world they
actually lift it off at the end of this January every piece of equipment must
also pass an acoustic chamber test and a US user friendliness evaluation to
ensure the station crew can install and manage it and uh oh here's a good one
apparently all equipment sent up to the ISS goes through what's called a white
glove test which uh thankfully is not
what it sounds like basically you put on a pair of white gloves and then you just
manhandle the crap out of it if the gloves snag or tear on
anything yeah that's a potential source of injury I'm going to need you to file
that down which f fun fact they actually do onsite and then repeat the test I
just hope they weren't filing any RAM sticks speaking of let's take a closer
look at these machines machine Z cuz it's not just one in here I know it was
kind of the point of this whole experiment but it still weirds me out
that these are just bog standard HP Enterprise systems that you could order
on their site today they don't even have lead armor or anything in this case
we're looking at an edgeline 4000 which is a multi-blade system and a dl360 dual
socket server we asked why these specific machines and the answer we got
was shockingly relatable we sorted the HP server catalog by depth power draw
and GPU support and these were the ones we were left with all right fair enough
as for why two different machines well here's the thing in a perfect world
multiples of the same machine would have been better but due to power constraints
they chose to have one with more CPU cores for more additional scientific
applications and one with pure CPU cores
but with a GPU for deep learning and AI one thing they needed for both however
is Ample Storage kokia generously sponsored this and brought us out here
so let's take a look at the let's call them unique choices that they made for
their storage configurations first up
obviously gone is any trace of super capacitors so kokia can proudly say that
their ssds are space ready I guess but
but what's less obvious is why they chose a SAS interface Drive rather than
NVMe for their high-speed bulk storage I mean you would think this is Space Age
Technology they'd want the fastest thing possible but these drives were selected
for their balance of performance reliability and especially power
efficiency when you're looking at a shared power budget across two servers
that is less than a typical gaming rig every single lot counts oh right and
that's even under ideal conditions at any given time to conserve power for
other priorities on the station the team can be asked to operate in half power
mode or even to shut down entirely for large operations like docking so the new
DL 360 server this guy right here gets
four 3.72 TB pm6 Enterprise drives
totaling 120 tabt of raw bulk storage for
scientific data and for backups then for
application drives we've got a really wild config again they went with four
drives but this time it's their RM sixs
again they're using SAS for lower power but this time two of the drives are
operating in a data redundant mirror and the other two are basically just
chilling there ready to be put into action in the event of a failure two
warm spares out of a four Drive array
would sound like crazy Paran oia on Earth but I assure you that in space
where bit flips from random radiation are much more common it's perfectly
reasonable I mean other than the overkill Drive config and the 28v power
conversion that they need to run it on the ISS there's not much to say about
this thing it's pretty much a bog standard server there is one cool demo
that they said we could run though oh yeah they offered to let us pull one of
the drives out of this dummy machine and live swap it in into this running
machine to show that no data loss will occur you want to do the honors sure all
they asked is that you put it in Bay 8 see you can see the drive is actually operating use Bay s that works look at
that I mean that's good rm6 that would be bad if it wasn't
kokia and then let's check the size beautiful 3.8 tab exactly what we want
they're not the exact drives that are on the space station but we wanted a
different capacity to show you that it's working and those are expensive
look at that status rebuilding I mean this seems like a lot of extra steps we
could have just looked up the light yeah the light yeah it's going
success and whether you're looking for a SAS drive an NVMe drive high capacity or
high performance we're going to have a bunch of Kyo has great Enterprise grade
drives Linked In the description down below I think I'll let you take this one
apart it's uh appropriately line this size okay let's take a look at the
second server that's packed into each Locker the el4 ,000 is a blade chassis
so the servers are basically these
little slide in cards that yeah I know AR these cute or what wait pull it out
and they go in on the side look at that
instead of from the front wow and they
managed to sneak four kokia xg6 NVMe
drives into each of these blades well
when I say each of these blades I should say they had the power budget for four
drives but they didn't have the power budget for four blades in the flight
configuration of this system they ship with just one of the four blades
installed though it should be noted they do fly up a spare blade per system in
the event of a failure there's just no way that that poor Locker can support
both these blades and the other server running concurrently let's put you away
and shift our Focus to the locker now
obviously there's no real up or down on
the as it whizzes around the earth at around 28,000 km an hour but to improve comfort
for the astronauts they tend to mount directional items like plants in a fixed
orientation which will put our lockers the drawers that hold our servers in the
ceiling there are two of these lockers each containing an identical system load
out for workload sharing and redundancy and these lockers present some serious
design challenges starting with the fact that they use a standard that quite lit
Lally doesn't exist on Earth Express
rack to pack the servers in then HP Enterprise had to get kind of creative
they found the shortest servers they could and then they stuffed them in
sideways and they're using a combination of air Cooling and water cooling the air
cooling uses a system on the ISS called AAA so at the back of these lockers
there's two cold air supplies and then two hot air returns that handles about
20% of the cooling for the servers op viously 20% not 100% they're going to
need some more and that's where this water cooling comes in this isn't a one:
one for how it would be deployed on the ISS for one thing these fittings 3D
printed mockups real fitting $800 a pop
if you were even allowed to buy them this tubing cheap vinyl from Home Depot
real tubing must be made of stainless steel in fact any wetted surface so
anything that comes in contact with water is supposed to be made out of
stainless steel but we can still illustrate how it's supposed to work so
on this side these go into a heat exchanger much like this one this is
actually from the first generation spaceborn but functionally it's the same
it pumps cold water into here chills the
air inside the system and then takes the warm water out to be dissipated to the
photovoltaic heat exchangers that are plumbed up with liquid ammonia coolant
and mounted to the exterior of the station to sink that heat into space
you need these kinds of special heat exchangers because while we think of
space as cold and we see people you know
oh blasted out of airlocks and they
freeze over or whatever in movies the truth is that for traditional methods of
heat dissipation you need air and in the near vacuum of space well it ain't
there air get
it cheesy jokes aside the two cooling systems together are good for removing
about 400 watts of heat from each locker but that's a combined budget so if this
GPU server kicks into high gear well
these CPUs better just chillax for a little bit now let's talk about one of
my favorite subjects networking there's four standard RJ45 ports on the front of
the okay I'm going to show you on the real one ah as I was saying four ports
on each Locker two of them connect both
of the iss's internal gigabit networks to a separate redundant switch inside
the locker and then the other two links are going to go between the two lockers
at 10 GB why 10 gig well because for
either backups or for multinode workloads that is a heck of a lot better
than gigabit and the power budget didn't allow for anything faster cool I guess
I'm starting to notice a pattern here anyway that's all pretty standard but
things become less so when you look at the station side of these Networks
cables this is a 37 pin military spec
locking connector these are designed for power and data but in this application
just eight of the pins would be used and
it is $220 for just this part now on the space
station NASA provides these cables for you but for testing sake really here on
Earth HP Enterprise had to make their own
fantastic what's really going to blow your mind though is for all of their
expensive networking these machines do not have a normal internet connection
just a private link back to Earth that NASA not only limits to a mighty 1
megabit per second but that they also encourage folks not to make full use of
also even now in 2024 it doesn't have
247 connectivity pretty much every hour
or two there's a period of downtime that can be anywhere as short as a few
minutes or as long long as 45 minutes
and that's because they have to prioritize generating enough power for
the station and when the giant solar arrays Point toward the sun they can
block line of sight with the satellites that provide connectivity which oh
that's a fun fact even though the ISS orbits less than 500 km from the surface
of the Earth our ping times to the ISS
and yes we got to Ping the ISS which was
pretty cool but our ping times were a atrocious reaching nearly a second as we
uploaded some of the dankest memes that Earth had to offer LT store.com now we
asked why that is and the answer was twofold one it's really old okay fair
enough but also too the station's Internet relay is in geosynchronous
orbit over 35,000 kilm from the Earth's surface and
uh well what I said was H well there's your problem right there it's just
really far and you might be wondering well why don't they just use
starlink that's a good question um someday they might but for now they
don't and HP and the team on the ISS have to work around the constraints of
the current setup I mean for crying out loud it took them four years to validate
that you know we can even just run a normal computer up here and actually
expect this thing to be reliable they can't just switch to something and go I
don't know I hope it works um oh by the way here's another fun one there's no
API to determine if if their connection is up or down so instead what they do is
buffer all their Communications in basic terms that means that they ping every
second and if the ping succeeds they send data and then hope that the
connection stayed up during that time it's a pretty good system okay not a
perfect one but certainly enough for us to learn a lot from the spaceborn
project even though spaceborn 2 has been in action for 3 years there's still so
much to learn two of the servers have hard Hardware raid cards for their
drives for example costing both mass and power consumption while two of them use
software raid which obviously doesn't consume any Mass but could impact power
consumption even more depending on the loads and one of them could be more or
less reliable than the other we won't know until we try which is kind of a
recurring theme here so if you want to learn more about the spaceborn computer
project we're going to have some resources linked for you down below and
we're also going to have a L to some great Enterprise storage options from
our sponsor kokia we're truly grateful for this unique opportunity to get
realistically as close as I ever will to the real ISS a prop in a sound stage in
LA but hey thanks kogia for the opportunity and for your long-term
partnership