WEBVTT

00:00:00.120 --> 00:00:06.839
AI chips have suddenly become a big selling point for phones but that might

00:00:04.839 --> 00:00:11.200
seem a little surprising that your little smartphone which already has

00:00:09.040 --> 00:00:16.720
serious limitations on power consumption and heat generation can run something as

00:00:13.480 --> 00:00:19.279
seemingly complicated as AI so how

00:00:16.720 --> 00:00:24.080
exactly do they pull this off well these neural processing units or npus are

00:00:22.439 --> 00:00:29.199
quite a bit different than your phone's main CPU cores features like Apple's

00:00:27.160 --> 00:00:34.520
neural engine or the machine learning engine on a Google tensor chip are

00:00:31.480 --> 00:00:36.520
highly optimized for AI tasks but

00:00:34.520 --> 00:00:41.239
probably suck at pretty much anything else it's kind of like how a GPU Works

00:00:39.680 --> 00:00:45.320
although they are much better for rendering Graphics than a more general

00:00:43.160 --> 00:00:50.440
purpose CPU you're not going to run your operating system off of your graphics

00:00:47.000 --> 00:00:52.840
card they are embarrassingly parallel a

00:00:50.440 --> 00:00:58.039
relatively small amount of die area then that is dedicated to AI can effectively

00:00:55.239 --> 00:01:01.920
run machine learning based tasks without sucking down too much power but that

00:01:00.320 --> 00:01:05.920
doesn't answer the question of why there's such a push to put these chips

00:01:03.519 --> 00:01:10.880
in our phones in the first place I mean we hear so much about Cloud AI where

00:01:08.360 --> 00:01:15.400
neural networks run on powerful servers so can't we just offload tasks like

00:01:13.159 --> 00:01:20.920
image optimization and voice recognition to the cloud well the answer lies in how

00:01:18.040 --> 00:01:25.600
large and complex the AI models are that your device needs to use models for

00:01:23.079 --> 00:01:30.720
common smartphone AI features such as voice recognition facial recognition and

00:01:28.320 --> 00:01:35.040
some kinds of image correction are often relatively small meaning that they can

00:01:32.600 --> 00:01:39.960
be run on device on a limited amount of silicon and if these functions can be

00:01:37.520 --> 00:01:45.280
run locally instead of in the cloud it's generally better to do so for example if

00:01:43.159 --> 00:01:50.040
you use an Android phone's speech recognition button you will wait around

00:01:47.920 --> 00:01:53.479
for your phone to send your speech over to a server over the Internet wait for

00:01:52.320 --> 00:01:57.600
that server to figure out what you're trying to say and then wait to get the

00:01:55.520 --> 00:02:02.479
results back to your phone if you could get results right now that would be a

00:02:00.000 --> 00:02:06.719
big selling point for a modern phone so even though Cloud Hardware might be more

00:02:04.320 --> 00:02:10.720
powerful the latency advantage of having a chip on your device makes this

00:02:08.560 --> 00:02:15.200
trade-off worth it not to mention that it helps protect your privacy by keeping

00:02:12.760 --> 00:02:19.959
as much of your data on your phone as possible but when may it not make sense

00:02:17.959 --> 00:02:24.640
to rely on a phone's npu we're going to tell you right after we thank MSI for

00:02:22.560 --> 00:02:32.680
sponsoring this video introducing the MSI mag 1250g pci5 power supply yeah you

00:02:30.599 --> 00:02:37.120
can keep your build Simple and Clean because this puppy is fully modular and

00:02:35.280 --> 00:02:42.159
why not clean up some zeros on that energy bill it also has an 80 plus gold

00:02:39.640 --> 00:02:48.519
certification so you know it's power efficient upgrade your PC's power with

00:02:44.519 --> 00:02:51.280
the MSI mag 1250g PCIe 5 check it out at

00:02:48.519 --> 00:02:55.159
the link below more advanced forms of generative AI aren't quite at the point

00:02:53.879 --> 00:03:00.000
where you can run them on a phone efficiently and by generative AI I mean

00:02:58.440 --> 00:03:06.000
artificial intelligence that can can create new media think about the stories

00:03:03.200 --> 00:03:10.360
that get generated by chat GPT or the AI art from services like mid Journey now

00:03:08.560 --> 00:03:15.239
you probably don't expect to run an entire Advanced image generation model

00:03:12.560 --> 00:03:19.920
on a phone at least with npus the size they are now but what about commonly

00:03:17.760 --> 00:03:24.200
touted features like Google's Magic editor on its pixel lineup well magic

00:03:22.239 --> 00:03:28.239
editor appears to need an internet connection since the feature uses enough

00:03:26.360 --> 00:03:31.840
generative AI to the point where the phone has to rely on cloud servers in

00:03:30.319 --> 00:03:36.840
order to give you the image you want in a reasonable amount of time however less

00:03:35.000 --> 00:03:42.920
demanding features such as live translate can run on device since the

00:03:40.480 --> 00:03:47.480
idea of AI specific Hardware on consumer devices is still relatively new tech

00:03:45.319 --> 00:03:52.920
companies are still trying to figure out exactly where The Sweet Spot is in terms

00:03:50.040 --> 00:03:57.640
of which tasks can and should be done on device versus which ones should be

00:03:54.640 --> 00:04:00.360
offloaded to the cloud in fact lots of

00:03:57.640 --> 00:04:05.000
AI as a service type products don't yet have a clear pathway to monetization

00:04:03.000 --> 00:04:09.799
instead it's more common for Tech firms to roll the features out now figure out

00:04:07.360 --> 00:04:13.400
how they work and then Jam them into their business model at some point Down

00:04:11.640 --> 00:04:18.440
the Line This is actually part of the reason that the die areas of npus and

00:04:15.760 --> 00:04:23.000
phones are still relatively small Hardware manufacturers would rather have

00:04:20.639 --> 00:04:28.320
enough inside the phone to enable AI features but then figure out exactly

00:04:25.720 --> 00:04:32.960
what the use cases are before they dedicate more hard Hardware to AI you're

00:04:31.120 --> 00:04:37.080
also seeing this on the desktop and laptop side of things with both AMD and

00:04:35.479 --> 00:04:41.479
Intel coming out with consumer processors that include npus and the

00:04:39.639 --> 00:04:45.759
idea is that features like Windows Studio Effects will run on device so

00:04:43.759 --> 00:04:50.400
your video calls look a little bit nicer But as time goes on both PC and phone

00:04:48.160 --> 00:04:54.280
manufacturers are aiming to get more and more AI functions running locally you're

00:04:52.639 --> 00:04:58.120
already seeing the push for this with how both team red and team blue have

00:04:56.479 --> 00:05:02.240
partnered with a number of outside software developers to make applic that

00:05:00.199 --> 00:05:06.199
can take advantage of their npus while it remains to be seen what AI features

00:05:04.280 --> 00:05:09.600
will become Mainstays it's clear that your gadgets are going to have

00:05:07.479 --> 00:05:14.320
significantly more brain power going forward for better or for

00:05:14.919 --> 00:05:21.720
worse if you guys enjoyed this video leave a like or a dislike depending on

00:05:19.560 --> 00:05:24.720
how you feel check out our video on the hardware that runs chat GPT if you're

00:05:23.520 --> 00:05:31.639
looking for something else to watch and leave a comment if you have a suggestion for a future video and of course don't

00:05:28.479 --> 00:05:31.639
forget to subscribe