If I were a shovel manufacturer during the gold rush, what sort of shovel would
I build for myself? Well, that's exactly the question that I'm answering today as
we tour the Nyx supercomput at NVIDIA
headquarters here in Santa Clara. Behind this door are 1,192
B200 GPUs. Absolute monsters rated for
up to,200 watts of power each. And they've been used for everything from AI
research to DLSS upscaling for gamers to
good old-fashioned NLPF drag racing. You know, to put the other AI up starts in
their place. So, what are we waiting for? Let's gear up and get inside. Uh
oh, shoot. What was the password again? Oh, that's right. S E G [music] U E for
our sponsor, MSI. Their Crosshair A16 HX
AI is an 18-in high-end gaming laptop that comes with a 50 series card and a
240 Hz display. Experience smooth gaming
on the go with the link in the video description. Are you tired of boring
screwdrivers? Boom. Four new colors of
the brand new LT transparent screwdriver. Plasma purple, cryo teal,
molten orange, and carbon black. They're see-through. They're awesome. and
they're probably going to sell out because for a limited time only. If you
buy through YouTube shopping at the link in the video description, you will get
20% off your driver, but only until December 30th. Collect them all or maybe
pair them with our brand new bid case and all over print hoodie. This
particular data center space is used for a combination of R&D and NVIDIA internal
production, meaning that it gets reconfigured on a pretty regular basis
[music] as NVIDIA works through validation of new chip or server
designs. That's why the raised floor has these plumbing access hatches for
highdensity water cooled deployments, even though at the moment it's set up
for air cooling to accommodate the DGX B200 racks that currently line the
floor. We're in the first of two rooms that are linked by this fiber optic
cabling, each of which contains two cats or cold air containment units. We're
going to go in one of them in a second, but before we do, I want to talk a
little bit about some of the details that would be easy to miss in here, like
this decibel meter here that probably illustrates why it's better for us to
talk out here. The deployment is absolutely peppered [music] with
sensors. obvious ones like temperature on both sides of the rack like this one
right here and also less obvious ones like humidity and air pressure. Humidity
management is very important. Too low and static electricity becomes a
problem. Too high and your servers start
to sweat and then you're going to start to sweat. Air pressure or maybe more
accurately air flow comes down to cooling. The DGX V200 systems here are
of course equipped [music] with their own cooling fans and the same goes for
the accompanying advanced tablet networking equipment. But for we are
talking about up to 14,000
watts of power consumption in each of these units.
Let's just say that these fans can use all the help that they can get. So,
fresh air gets actively blown up through the floor, creating positive pressure
inside this room where it gets pushed through our DGXs and out into the hot
aisle where uh things definitely get a
lot hotter. I mean, I was getting kind of chilly in there, but here I would be
sweating in a matter of minutes. But in spite of the challenges, the systems are
running perfectly. And NVIDIA pointed out that some of these nodes are running
full tilt on undisclosed workloads as we speak. Now, because NVIDIA uses spaces
like these for validation, it's important that they get all the little
details right so customers can just take this blueprint, copy it, and paste it
into their own facility and trust that it's going to run at scale. The
networking racks, for instance, are strategically positioned to minimize the
length of fiber optic cabling between the four cats that make up this cluster.
Two in this room and two in the next one over. This is both to maintain signal
integrity. They found any higher than 100 to 150 m can be problematic at these
speeds, but it's also to maintain the integrity of the ceiling. See, it turns
out that when you start bundling up hundreds of fiber optic cables, they can
get pretty heavy. And the longer the runs, the more strain they put on the
ceiling. It also helps save on cost. This I don't have much to say about
other than Oh my god, isn't it
beautiful? Okay, I lied. I do have some stuff to
say. This cable management is not just for looks, but rather to maintain air
flow. When you have this many cables, you actually do need to bundle them. And
it also helps facilitate maintenance in the event of a broken or damaged cable.
They run extra fiber in each bundle that can be terminated as needed, and it is a
lot easier to find that when things are organized. [music] Careful thought goes
into rack serviceability, too. While the networking racks use traditional
vertical PDUs for their power distribution, the DGX racks use three
top-mounted PDUs that help balance the three [music] phases of power coming in.
On the subject of power, we skipped over these at the SFU data center, but uh
have you ever wondered what a >> 415 volt 100 amp
>> power plug looks like? Wonder no longer. Another key advantage of putting all the
PDUs up top is it makes it a little easier to get the DGX units in and out.
They uh turn this one off and agreed to pull it out of the deployment just to
show us how it's done. But when I asked if we could tear it down, they were
like, "Huh?" Not because they have anything to hide,
but just cuz they thought this soon to be decommissioned and recycled
engineering unit that we can really get our greasy meat mitts into would be a
lot more [music] fun. Woo!
If you ever wondered what the heat sink would look like on a 1200 W GPU, wonder
no more. How many heat pipes is this?
It's like a forest of heat pipes in there. Oh, they're flat. That makes so
much sense cuz you'll get more air flow through the fins. And I mean, that's
some pretty hard working air dealing with not one but two rows of these GPUs.
Not to mention this middle heat sink here that seems to be for the Envy Link
switching equipment. I mean, I knew you need some beefy to allow these GPUs to
pull the 192 gigs of HBM 3 memory that
they have, but still, I'd never seen the cooling for it. Another thing I've never
gotten to see, or at least never gotten to film, is NVIDIA's proprietary SXM
interface. I know because I was looking for a B-roll shot of one of these a
little while ago, and I realized I've never actually held one up on camera.
So, now I've done it. And they also gave us this to show what Okay, not the same
generation, but a similar GPU might look like without the heat sink. Oh, we've
also got to look at the SXM interface that goes back to the rest of the system
and also carries power for these [music] GPUs. These are 1,200 W each, but these
cards go as high as 1,400 W. I mean, how
high would the cooler be at that point? JK, it would be water cooled. We were
hoping to see some water cooled machines today, but I guess NVIDIA has to save
[music] something for next time. I got to say though, if the new stuff is even
just equally cool to the last gen H100 water cooled machines we saw at the SFU
Fur Supercomput, I'm sure it's absolutely mind-blowing. All of which is
really cool, but what do they need all of this for? Well, the folks who are
responsible for getting us access to everything today are actually from the
GeForce gaming team. So, one of their big uses for these internal compute
resources is, of course, deep learning superers sampling or DLSS. In a
nutshell, DLSS allows your GPU to render your game at a lower resolution, then
use deep learning or AI to upscale each output frame to your monitor's
resolution. The benefit of this is that you can run at a higher frame rate for
improved animation smoothness. But the drawback is that you can't create
something from nothing and upscaled images struggle to achieve the same
fidelity as a native rendered image. With that said, DLSS has improved a lot
over the years and I got a chance to sit down and chat with Edward Leu from that
team who talked us through some of the processes that they use to develop new
features and new fixes. The first thing he pointed out is that while this hot
new AI model took a thousand GPUs two weeks to train or whatever, makes for a
good headline, it overlooks most of the actual cost and time, which is in the
test training runs that take place before the hero run. His team is
constantly iterating on the data they're feeding into DLSS and [music] the
waiting, and they're evaluating new innovations in the AI space. Sometimes
it's more surgical, like, "Oh, hey, uh, we noticed Cyberpunk has an issue with
cars having like three or four bumpers as you're driving around. How can we
address this with the current model?" That kind of thing can be turned around sometimes in a matter of weeks or months
using the resources that we just saw. Though, uh, he was quick to point out
that Jensen, if you're watching, there's no limit to how many GPUs his team could
use to speed up the process. I told him I'd say that because the faster they can
test a new data set, the faster they can tune it and ship it. Other times, the
changes are more transformative, pun intended, like the move from a
convolutional neural network to the more accurate transformer model that runs
best on NVIDIA's newest cards. This can require basically a complete tear down
and doover of the entire pipeline and can take a year or more. But I mean,
hey, NVIDIA has bet pretty much their entire future on being able to stay at
the forefront of their 21st century shovel technology. So, I guess that's
the price you pay. But what does all this mean for traditional rendering?
Well, the folks here believe very strongly that in time DLSS will not only
be as good, it will be better than [music] traditional. In fact, Edward
pointed to these slides from a talk that he gave a number of years ago, showing
that what we think of as native rendering is already a pretty imperfect
approximation. Here it is compared to a ground truth image, which is rendered at
a much higher resolution than downsampled to 1080p. Native, it's
clearly worse. And as you can see in this comparison, even back then with
DLSS 2.0, So there were situations where
a model trained on these higher resolution or more accurately higher
sampled ground truth images could result in the GPU reconstructing an output with
DLSS that's closer to ground truth [music] than the native image was. I
asked, by the way, how are these ground truth images created? [music] and he
said that typically it's actually just on run-of-the-mill gaming hardware, but
instead of running at 60 or 120 frames per second, they'll sometimes be running
tens or thousands of pixel samples to the point where we're talking more like
60 to 120 seconds per frame or even
more. Man though, if DLSS could consistently reconstruct that ground
truth image from a lower resolution input every time, I'm sure no one would
ever turn it off. Unfortunately for Edward and his team though, no one
notices when DLSS is working perfectly. It's when it
trips over itself that we tend to notice, and it still does do so
fairly regularly. However, with that said, from my own experience, it has
continued to improve at a pretty solid clip since its debut, and the rest of
the industry has pretty much accepted that AI accelerated image enhancement is
the future, whether every gamer wants it or not. So sales of shovels will likely
continue until gamer morale improves. Good luck everyone. I hope it's not a
bubble. This is a nice office. I'd hate for something to happen to it. Just like
this is a nice segue to our sponsor,
Squarespace. If you're building a brand or a business, it's important to have a
website. Designing your own site can feel like a bit of a daunting task, but
it really doesn't have to. Squarespace makes it easy to get your message across
to potential customers and subscribers in a clean and digestible way. Their
design intelligence tool utilizes AI and works with your own creativity to create
a theme and style for your site that matches your personality. You can also
use Squarespace to directly invoice your customers with payment options like a
direct debit, Apple Pay, CLA, and more.
And use their analytic tools to track sales, strategize, and continue to build
your brand. There's millions of URLs available, and Squarespace's domain tool
will help you search for the right address just for you. We've even used
Squarespace for some of our own websites here. Start building your website today,
and you'll receive 10% off your first purchase by visiting squarespace.com/LTT.
If you guys enjoyed this video, maybe go check out the time we I don't know,
something invidated. Let's do a throwback video. How about
when I checked out the launch of G-Sync? The production values were lower, but
hey, it was fun.