1
00:00:00,120 --> 00:00:06,839
AI chips have suddenly become a big selling point for phones but that might

2
00:00:04,839 --> 00:00:11,200
seem a little surprising that your little smartphone which already has

3
00:00:09,040 --> 00:00:16,720
serious limitations on power consumption and heat generation can run something as

4
00:00:13,480 --> 00:00:19,279
seemingly complicated as AI so how

5
00:00:16,720 --> 00:00:24,080
exactly do they pull this off well these neural processing units or npus are

6
00:00:22,439 --> 00:00:29,199
quite a bit different than your phone's main CPU cores features like Apple's

7
00:00:27,160 --> 00:00:34,520
neural engine or the machine learning engine on a Google tensor chip are

8
00:00:31,480 --> 00:00:36,520
highly optimized for AI tasks but

9
00:00:34,520 --> 00:00:41,239
probably suck at pretty much anything else it's kind of like how a GPU Works

10
00:00:39,680 --> 00:00:45,320
although they are much better for rendering Graphics than a more general

11
00:00:43,160 --> 00:00:50,440
purpose CPU you're not going to run your operating system off of your graphics

12
00:00:47,000 --> 00:00:52,840
card they are embarrassingly parallel a

13
00:00:50,440 --> 00:00:58,039
relatively small amount of die area then that is dedicated to AI can effectively

14
00:00:55,239 --> 00:01:01,920
run machine learning based tasks without sucking down too much power but that

15
00:01:00,320 --> 00:01:05,920
doesn't answer the question of why there's such a push to put these chips

16
00:01:03,519 --> 00:01:10,880
in our phones in the first place I mean we hear so much about Cloud AI where

17
00:01:08,360 --> 00:01:15,400
neural networks run on powerful servers so can't we just offload tasks like

18
00:01:13,159 --> 00:01:20,920
image optimization and voice recognition to the cloud well the answer lies in how

19
00:01:18,040 --> 00:01:25,600
large and complex the AI models are that your device needs to use models for

20
00:01:23,079 --> 00:01:30,720
common smartphone AI features such as voice recognition facial recognition and

21
00:01:28,320 --> 00:01:35,040
some kinds of image correction are often relatively small meaning that they can

22
00:01:32,600 --> 00:01:39,960
be run on device on a limited amount of silicon and if these functions can be

23
00:01:37,520 --> 00:01:45,280
run locally instead of in the cloud it's generally better to do so for example if

24
00:01:43,159 --> 00:01:50,040
you use an Android phone's speech recognition button you will wait around

25
00:01:47,920 --> 00:01:53,479
for your phone to send your speech over to a server over the Internet wait for

26
00:01:52,320 --> 00:01:57,600
that server to figure out what you're trying to say and then wait to get the

27
00:01:55,520 --> 00:02:02,479
results back to your phone if you could get results right now that would be a

28
00:02:00,000 --> 00:02:06,719
big selling point for a modern phone so even though Cloud Hardware might be more

29
00:02:04,320 --> 00:02:10,720
powerful the latency advantage of having a chip on your device makes this

30
00:02:08,560 --> 00:02:15,200
trade-off worth it not to mention that it helps protect your privacy by keeping

31
00:02:12,760 --> 00:02:19,959
as much of your data on your phone as possible but when may it not make sense

32
00:02:17,959 --> 00:02:24,640
to rely on a phone's npu we're going to tell you right after we thank MSI for

33
00:02:22,560 --> 00:02:32,680
sponsoring this video introducing the MSI mag 1250g pci5 power supply yeah you

34
00:02:30,599 --> 00:02:37,120
can keep your build Simple and Clean because this puppy is fully modular and

35
00:02:35,280 --> 00:02:42,159
why not clean up some zeros on that energy bill it also has an 80 plus gold

36
00:02:39,640 --> 00:02:48,519
certification so you know it's power efficient upgrade your PC's power with

37
00:02:44,519 --> 00:02:51,280
the MSI mag 1250g PCIe 5 check it out at

38
00:02:48,519 --> 00:02:55,159
the link below more advanced forms of generative AI aren't quite at the point

39
00:02:53,879 --> 00:03:00,000
where you can run them on a phone efficiently and by generative AI I mean

40
00:02:58,440 --> 00:03:06,000
artificial intelligence that can can create new media think about the stories

41
00:03:03,200 --> 00:03:10,360
that get generated by chat GPT or the AI art from services like mid Journey now

42
00:03:08,560 --> 00:03:15,239
you probably don't expect to run an entire Advanced image generation model

43
00:03:12,560 --> 00:03:19,920
on a phone at least with npus the size they are now but what about commonly

44
00:03:17,760 --> 00:03:24,200
touted features like Google's Magic editor on its pixel lineup well magic

45
00:03:22,239 --> 00:03:28,239
editor appears to need an internet connection since the feature uses enough

46
00:03:26,360 --> 00:03:31,840
generative AI to the point where the phone has to rely on cloud servers in

47
00:03:30,319 --> 00:03:36,840
order to give you the image you want in a reasonable amount of time however less

48
00:03:35,000 --> 00:03:42,920
demanding features such as live translate can run on device since the

49
00:03:40,480 --> 00:03:47,480
idea of AI specific Hardware on consumer devices is still relatively new tech

50
00:03:45,319 --> 00:03:52,920
companies are still trying to figure out exactly where The Sweet Spot is in terms

51
00:03:50,040 --> 00:03:57,640
of which tasks can and should be done on device versus which ones should be

52
00:03:54,640 --> 00:04:00,360
offloaded to the cloud in fact lots of

53
00:03:57,640 --> 00:04:05,000
AI as a service type products don't yet have a clear pathway to monetization

54
00:04:03,000 --> 00:04:09,799
instead it's more common for Tech firms to roll the features out now figure out

55
00:04:07,360 --> 00:04:13,400
how they work and then Jam them into their business model at some point Down

56
00:04:11,640 --> 00:04:18,440
the Line This is actually part of the reason that the die areas of npus and

57
00:04:15,760 --> 00:04:23,000
phones are still relatively small Hardware manufacturers would rather have

58
00:04:20,639 --> 00:04:28,320
enough inside the phone to enable AI features but then figure out exactly

59
00:04:25,720 --> 00:04:32,960
what the use cases are before they dedicate more hard Hardware to AI you're

60
00:04:31,120 --> 00:04:37,080
also seeing this on the desktop and laptop side of things with both AMD and

61
00:04:35,479 --> 00:04:41,479
Intel coming out with consumer processors that include npus and the

62
00:04:39,639 --> 00:04:45,759
idea is that features like Windows Studio Effects will run on device so

63
00:04:43,759 --> 00:04:50,400
your video calls look a little bit nicer But as time goes on both PC and phone

64
00:04:48,160 --> 00:04:54,280
manufacturers are aiming to get more and more AI functions running locally you're

65
00:04:52,639 --> 00:04:58,120
already seeing the push for this with how both team red and team blue have

66
00:04:56,479 --> 00:05:02,240
partnered with a number of outside software developers to make applic that

67
00:05:00,199 --> 00:05:06,199
can take advantage of their npus while it remains to be seen what AI features

68
00:05:04,280 --> 00:05:09,600
will become Mainstays it's clear that your gadgets are going to have

69
00:05:07,479 --> 00:05:14,320
significantly more brain power going forward for better or for

70
00:05:14,919 --> 00:05:21,720
worse if you guys enjoyed this video leave a like or a dislike depending on

71
00:05:19,560 --> 00:05:24,720
how you feel check out our video on the hardware that runs chat GPT if you're

72
00:05:23,520 --> 00:05:31,639
looking for something else to watch and leave a comment if you have a suggestion for a future video and of course don't

73
00:05:28,479 --> 00:05:31,639
forget to subscribe