it's pretty clear where NVIDIA's priorities lie these days we're here at
the computex booth of one of their Partners Gigabyte and this is the entire
gaming showcase that's because they like the
rest of the industry understand that the future of computing lies in the data
center that is where the grace super
chip comes in under each of these gigantic heat spreaders are 72 of
NVIDIA's graay CPU course connected
together using what NVIDIA calls the Envy link chipto chip interconnect for a
total of 44 cores except that's just one
of the nodes This Server from Gigabyte accepts not one not two but four of
these modules in its four separate nodes
that is an absolutely mindbending
576 cores in a 2u server rack but these
are not the types of CPUs that you have in your gaming PC at home those
processors from the likes of AMD and Intel are based on the x86 architecture
so similar to what Apple did with their M series M1 and M2 processors NVIDIA is
making use of a different processor architecture called ARM and uh we
actually did get permission to do this we're going to be taking a closer look
here oh it doesn't look much like it but this
is the same style of processor that you might find in your phone ARM processors
have a lot of advantages first and foremost being that they're typically
more power efficient thanks to their relatively lightweight and structure set
so much so that NVIDIA claims these gray
CPUs have twice the performance per watt of the latest x86 chips but the
disadvantage is that also require software like your operating system and
all the programs you need to run to be coded and compiled specifically for ARM
now for the PC market because 86 has been the standard for so long it's
difficult to justify switching over to ARM it would cost you so much in terms
of backwards compatibility but in the data center the types of customers who
are going to buy a processor like this are usually developing their own
software anyway like let's say Google to
run the algorithms that power Google search or YouTube recommendations for
them switching over to ARM isn't as big a deal and in fact companies like Amazon
who are developing their own ARM-based CPUs are already doing it and very
effectively I mean hey if my next gaming
CPU could be half the power draw and the same performance of my current one I'd
be stoked but this is even better imagine if instead of one computer
you're talking in thousands or tens of thousands the savings start to become so
large that it's less a question of can we afford this migration and more a
question of can we afford not to make it
now I didn't ask permission for this part but nobody seems to be stopping me
or even really paying attention to me so let's take apart Grace super
chip on each gray super chip is up to
480 GB of lpddr 5x ECC memory per CPU
and what's really cool is that that can actually be accessed by either CPU over
the Envy link interconnect that's how
fast this new Envy link is the only downside to this approach since we're
making comparisons to Apple is that just like with your M2 MacBook you better
decide how much memory you want in your server right at the time you buy it
unless you want to replace the entire ire compute engine while you perform a
memory upgrade given that the rumored price of their h100 gpus is
$11,000 I don't even want to know what this thing costs but hopefully you get a
bit of a discount when you buy it together with the gray super chip CPU
let me show you this can't believe they're letting me take this off the
wall okay success we have dropped nothing
important important so far today this is Grace Hopper on the one
side we've got the same 72 core Grace ARM CPU that we just saw but on the
other side the oo shiny latest NVIDIA
h100 Hopper GPU you can probably see
where this is going just like with the Dual CPU Grace module these two are also
EnV link chipto chip interconnected meaning that the CPU and GPU have a
whopping 900 gab per second of
theoretical bandwidth to talk to each other so first some perspective a GPU
using a full 16 Lane Gen 5 PCIe slot
would only have about 64 GB a second of
peak throughput that is 114th as much as this and that's far
from the only mindbending number that this thing is capable of while the CPU
side uses the same up to 480 GB of lpddr
5x for the GPU side they need much
faster hbm3 memory that runs at a whopping 4
terabytes per second it's about four
times faster that's why the memory needs to be right on the package right next to
the GPU now all that is great and cool and
all but hbm is very expensive and as you
can see there's only so much space here so the H1 100 only gets 96 GB of memory
okay yeah for gaming that certainly sounds like a lot but AI data sets can
involve terabytes of data so it can get used up very quickly that's where the
interconnect comes in it allows the GPU
to access the CPU's memory in a very direct and transparent way giving the
h100 hopper GPU a functional memory capacity of nearly
600 GB in Practical terms according to
NVIDIA that puts Grace Hopper anywhere from about 2 and 1/2 times to nearly
four times as fast as an x86 CPU paired
with their last generation a100 GPU and
where things get really wild is in the data center with an Envy link switch
system you could connect up to 256 gpus together giving them access to
up to 150 terab of high bandwidth memory
I mean you guys remember that crazy Mars Lander demo that we showed off on the
paby of flash array you could load that entire 1 billion Point data set into
memory in that configuration and still have 50 tabt to spare now this module
little bit more power hungry than the Dual CPU version 1,000 versus 500 watts
per module but I mean that's for CPU GPU
and RAM for both of them and with this kind of performance
of course not everybody wants to move to an ARM hybrid CPU GPU architecture so
NVIDIA is still going to be supporting their uh oldfashioned configurations be
they h100 gpus in a PCIe form factor or
their hgx h100 with up to eight SMX 5
gpus each of these draws a massive 700
Watts making an RTX 490 look like a child's play thing and supports n link
between these gpus and envy switch to
additional servers this is the G 593 sd0
and Gigabyte was very proud of the fact that they are the first NVIDIA certified
hgx h100 8gpu server in a 5u chassis man
that is a lot of compute in a tiny space Jake's in my ear here telling me I
should pull one of the power supplies but if you've noticed it getting darker
it's because they're actually shutting down the pre-show and uh they're trying
to get us out of here but there is one more thing that we wanted to talk about
where'd it go dang it Jake no oh my God oh my God okay well this is uh no wait
this isn't the one I wanted okay it's a connect X7 this is an even faster
network card so this is probably the first NVIDIA developed melanox network
card given that uh the acquisition was what about two years ago yeah conx was
already out yeah but NVIDIA didn't buy melanox just to make faster connectx
cards no it was to make these
this is a Bluefield 3 so it has networking on it this is a 100 GB one
but it's available it speeds up to 400 gbit but what's really special about it
is that it has up to 16 processing cores
on it why you might ask well just like
in the old days when we started offloading tcpip processing to our
network cards rather than having our CPU handle them this is going to offload all
kinds of interesting things like encryption of your network traffic or
say for example handling managing your file system because when you're someone
like an AWS and you want to squeeze as much revenue as possible out of every
CPU in your data center you don't want it handling stupid BS that you could
just offload to your network card so the idea here is to free up CPU resources
that can be leased to customers by putting them onto the network card
itself and this is especially true for software where the developer sells you a
license per core that's why even though these are going
to be wildly expensive a lot more than
the 4060 TI NVIDIA is going to sell shed
loads of them just like I sold this segue to our sponsor pulseway are you
sick of feeling like a prisoner changeed to a desk managing it systems Unleash
Your Inner it hero with pulseway remote monitoring and management software
pulseway platform gives you the power to manage your it infrastructure from
anywhere even from the comfort of your own couch and with realtime alerts and
notifications you can be the first to know about potential issues before
anyone else on your team it's accessible through whatever device is close to you
thanks to their convenient apps allowing you to control your it systems like a
boss even if you're lounging in your pjs so say goodbye to the boring routine of
it management and hello to the fun of being an IT hero with pulseway advanced
technology don't wait this is your chance to become a legend in the IT
world just try pulseway for free today and experience the power of simplified
it infrastructure management click the link below to get started if you guys
enjoyed this video why don't you check out oh the paby of flash that was a good
one well we're at the Gigabyte Boo come on uh the g- one yeah g oh actually no
new wanic new new new wanic 3 wanic 4 I
mean damn it