1
00:00:00,000 --> 00:00:04,880
There's one big compromise that gamers have had to make for a long time.

2
00:00:04,880 --> 00:00:08,680
You want your games to look better, or do you want them to run faster?

3
00:00:08,680 --> 00:00:13,160
Typically, this has meant turning down your graphics setting to get more frames per second,

4
00:00:13,160 --> 00:00:20,800
especially if you don't have a high-end graphics card. But today, we're instead going to talk about the amount of input lag that gets introduced

5
00:00:20,800 --> 00:00:26,480
when you're trying to upscale a game. We consulted with our friend Amin Shabane over at Merci to put this video together, so we

6
00:00:26,480 --> 00:00:33,880
like to thank him for his help. Now to be clear, I am not talking about your GPU rendering frames from scratch.

7
00:00:33,880 --> 00:00:38,480
What I'm referring to instead is what happens after your GPU finishes rendering a frame,

8
00:00:38,480 --> 00:00:45,760
and either the GPU or your display resizes the image to make it fit a certain resolution.

9
00:00:45,760 --> 00:00:49,600
You can see this if you're running a PC game at below your monitor's native resolution

10
00:00:49,600 --> 00:00:55,080
to improve performance, or if you hooked up an older console to modern flat-panel TV.

11
00:00:55,360 --> 00:00:59,600
There are different forms of upscaling, some of which look nicer than others, but they

12
00:00:59,600 --> 00:01:05,520
all require a certain amount of post-processing time which can introduce noticeable input lag,

13
00:01:05,520 --> 00:01:08,840
meaning that there's a delay between when you press a button or move a thumbstick or

14
00:01:08,840 --> 00:01:12,680
move the mouse, and the corresponding action appearing on the screen.

15
00:01:12,680 --> 00:01:18,400
This can seriously hinder gameplay for obvious reasons, especially in older titles like classic

16
00:01:18,400 --> 00:01:24,240
platforms where responsiveness is a huge part of making the game feel like you remember.

17
00:01:24,240 --> 00:01:31,360
But why does it introduce so much lag? Well to get the image looking as nice as possible, some algorithms look at the frames that are

18
00:01:31,360 --> 00:01:37,280
rendered before and after the frame to be upscaled to better understand what a higher res version

19
00:01:37,280 --> 00:01:40,920
of the same image is supposed to look like.

20
00:01:40,920 --> 00:01:45,760
Then they apply what they think are correct changes to the frame.

21
00:01:45,760 --> 00:01:49,760
This method of analyzing multiple frames that are held in what's called the frame buffer

22
00:01:50,200 --> 00:01:54,080
before they're shown to the user can definitely yield visual improvements.

23
00:01:54,080 --> 00:02:00,040
But not only is this a computationally time-consuming process that adds lag, it can also result

24
00:02:00,040 --> 00:02:04,240
in worse image quality if the frames it's examining were highly compressed.

25
00:02:04,240 --> 00:02:09,720
For example, if you're watching a movie. An alternative approach to reduce lag is to, instead of relying on multiple frames at

26
00:02:09,720 --> 00:02:15,440
one time, have the algorithm look at certain elements of a single frame that human brains

27
00:02:15,440 --> 00:02:24,040
are typically sensitive to. For example, Mercedes-Mersai, Mercedes-Mersay, Mercedes-M classic smart HDMI cable has a built-in

28
00:02:24,040 --> 00:02:28,800
library of objects like edges and textures that we naturally key in on.

29
00:02:28,800 --> 00:02:34,720
Think about how jaggies caused by bad anti-aliasing of edges are often really noticeable to us.

30
00:02:34,720 --> 00:02:40,080
Interestingly, characters' eyes are also a focus as humans are psychologically programmed

31
00:02:40,080 --> 00:02:43,720
to be very sensitive to what someone else's eyes are doing.

32
00:02:43,720 --> 00:02:47,520
This kind of strategy of focusing mostly on key visual elements can greatly reduce lag

33
00:02:47,520 --> 00:02:52,680
time while improving visual quality due to its reliance on predetermined visual cues

34
00:02:52,680 --> 00:02:57,720
for the algorithm to focus on, as well as the fact it only examines one frame.

35
00:02:57,720 --> 00:03:02,920
But like other upsampling methods, it's not perfect, so can we do better?

36
00:03:02,920 --> 00:03:06,880
It turns out the answer is yes, though we might still be some years away from seeing

37
00:03:06,880 --> 00:03:13,640
it becoming widely available. Rather than programming a scaler to spot a few specific elements, computer scientists

38
00:03:13,640 --> 00:03:18,360
have been training artificial intelligences to recognize what more complex objects are

39
00:03:18,360 --> 00:03:26,560
supposed to look like. Accurately scaling an HD image to 4K or even 8K is a very computationally intensive problem,

40
00:03:26,560 --> 00:03:31,560
so large amounts of AI training will reduce the reliance of predefined features and allow

41
00:03:31,560 --> 00:03:36,600
a scaler to recognize anything from whether or not an object is a dog to how it handles

42
00:03:36,600 --> 00:03:42,520
scenes with complicated lighting. We're already seeing this to some extent with NVIDIA's Deep Learning Super Sampling

43
00:03:42,520 --> 00:03:47,920
or DLSS, where a supercomputer is fed with lots of frames from different games and figures

44
00:03:47,920 --> 00:03:52,960
out an algorithm to produce something close to an ideally anti-aliased image.

45
00:03:52,960 --> 00:03:56,920
These algorithms are then pushed out to individual users through software updates.

46
00:03:56,920 --> 00:04:00,560
Not only does this allow gamers to improve how their games look without lowering frame

47
00:04:00,560 --> 00:04:05,200
rates, the more efficient post-processing algorithms optimized through AI should hopefully

48
00:04:05,200 --> 00:04:12,040
make games feel more responsive as well. Don't remember that if you just suck at games like CSGO because you have straight-up terrible

49
00:04:12,040 --> 00:04:17,320
reflexes, AI probably won't help you, so you might want to just give turn-based games

50
00:04:17,320 --> 00:04:22,560
a shot. So thanks for watching, guys. If you liked this video, give it a thumbs up, subscribe, and be sure to hit us up in

51
00:04:22,560 --> 00:04:28,760
the comment section for your ideas about future videos that we should make, about tech topics

52
00:04:28,760 --> 00:04:31,640
that you want explained. We'll do it.