{"video_id":"mk_2-BBGZtA","title":"How CPUs Use Multiple Cores","channel":"Techquickie","show":"Techquickie","published_at":"2024-05-04T14:58:16Z","duration_s":315,"segments":[{"start_s":0.0,"end_s":6.5600000000000005,"text":"If I tell you I'm multitasking, don't believe me, the best that I can do as a weak pitiful human","speaker":null,"is_sponsor":0},{"start_s":6.5600000000000005,"end_s":12.64,"text":"is to quickly switch my attention from one single thing to another. But is that how your CPU works?","speaker":null,"is_sponsor":0},{"start_s":13.44,"end_s":19.04,"text":"Actually, no. Your CPU relies on a strategy called scheduling, where the operating system","speaker":null,"is_sponsor":0},{"start_s":19.04,"end_s":24.48,"text":"will decide what sets of instructions or threads will get put through the CPU first,","speaker":null,"is_sponsor":0},{"start_s":24.56,"end_s":30.0,"text":"second, third, and so on. Understanding scheduling is crucial to understanding","speaker":null,"is_sponsor":0},{"start_s":30.0,"end_s":35.36,"text":"exactly how modern multi-core CPUs work. So we got a little bit of help for this video,","speaker":null,"is_sponsor":0},{"start_s":35.36,"end_s":39.2,"text":"and we'd like to thank our friends at Intel for their assistance as we run through this thought","speaker":null,"is_sponsor":0},{"start_s":39.2,"end_s":44.4,"text":"experiment. Each thread that your computer is working on gets assigned a value called priority,","speaker":null,"is_sponsor":0},{"start_s":44.4,"end_s":49.12,"text":"so a thread with a higher priority value will be further ahead in line to get processed by your","speaker":null,"is_sponsor":0},{"start_s":49.12,"end_s":55.12,"text":"CPU. Developers can assign priority values when they write a program, and Windows can change a","speaker":null,"is_sponsor":0},{"start_s":55.12,"end_s":60.08,"text":"thread's priority based on how many system resources it needs and what the exact nature","speaker":null,"is_sponsor":0},{"start_s":60.08,"end_s":65.6,"text":"of the instructions are. For example, if a thread is tied to what you're inputting with a keyboard","speaker":null,"is_sponsor":0},{"start_s":65.6,"end_s":70.72,"text":"and mouse, such as clicking buttons in your web browser or aiming your weapon in a game,","speaker":null,"is_sponsor":0},{"start_s":70.72,"end_s":76.08,"text":"it's likely going to get a higher priority. By contrast, background activities that aren't","speaker":null,"is_sponsor":0},{"start_s":76.08,"end_s":81.92,"text":"time sensitive, those will get a lower priority. Similarly, developers can assign another value","speaker":null,"is_sponsor":0},{"start_s":81.92,"end_s":89.2,"text":"called quality of service, or QOS, to each thread. QOS is a way for devs to signal to Windows","speaker":null,"is_sponsor":0},{"start_s":89.2,"end_s":94.08,"text":"how much processor power a thread might need, and while Windows doesn't treat it as important","speaker":null,"is_sponsor":0},{"start_s":94.08,"end_s":99.44,"text":"as priority, QOS is still a factor that the operating system will take into account when it","speaker":null,"is_sponsor":0},{"start_s":99.44,"end_s":105.44,"text":"comes to scheduling. QOS is especially relevant when it comes to power management, as the OS can","speaker":null,"is_sponsor":0},{"start_s":105.44,"end_s":111.44,"text":"use QOS to request a certain frequency from the CPU, and the CPU's hardware can decide","speaker":null,"is_sponsor":0},{"start_s":111.44,"end_s":116.64,"text":"if it can ramp up to that frequency while staying within power limits. That's especially useful","speaker":null,"is_sponsor":0},{"start_s":116.64,"end_s":124.0,"text":"on laptops. Which is all very cool, but in terms of our guy doing two things at once analogy,","speaker":null,"is_sponsor":0},{"start_s":124.0,"end_s":130.64,"text":"priority and QOS pertain more to what I'm choosing to do, like texting while watching a movie,","speaker":null,"is_sponsor":0},{"start_s":130.64,"end_s":137.44,"text":"rather than how I'm juggling those two tasks. So let's drill down into that. Traditionally,","speaker":null,"is_sponsor":0},{"start_s":137.44,"end_s":144.0,"text":"CPU threads would go through one at a time, like my brain, but this isn't how modern multicore","speaker":null,"is_sponsor":0},{"start_s":144.0,"end_s":149.76,"text":"systems operate, especially considering some newer chips even have different kinds of cores","speaker":null,"is_sponsor":0},{"start_s":149.76,"end_s":156.08,"text":"all on one package. One goal in scheduling on multicore is not to context switch more than is","speaker":null,"is_sponsor":0},{"start_s":156.08,"end_s":162.56,"text":"necessary. And context switching is just another fancy term for moving a thread from one core","speaker":null,"is_sponsor":0},{"start_s":162.56,"end_s":168.8,"text":"onto a different one. In general, you want each thread to start and finish on one single core,","speaker":null,"is_sponsor":0},{"start_s":168.8,"end_s":173.44,"text":"and you want to load balance across cores so that none of them are getting overwhelmed.","speaker":null,"is_sponsor":0},{"start_s":173.44,"end_s":177.68,"text":"And there's been a lot of tweaking over the years to figure out how to do this easily","speaker":null,"is_sponsor":0},{"start_s":177.68,"end_s":183.2,"text":"and efficiently, such as with libraries, which are software resources that sit between the","speaker":null,"is_sponsor":0},{"start_s":183.2,"end_s":188.32,"text":"program and the operating system and allow multi-threading without making a developer do","speaker":null,"is_sponsor":0},{"start_s":188.32,"end_s":194.72,"text":"a ton of extra coding. On newer chips, how exactly threading works out across cores becomes even more","speaker":null,"is_sponsor":0},{"start_s":194.72,"end_s":200.96,"text":"important because of hybrid chips like Intel's Alder Lake lineup. These chips have higher performance,","speaker":null,"is_sponsor":0},{"start_s":200.96,"end_s":208.8,"text":"or P cores, and lower performance, but more power saving, E cores. It might be confusing as to why","speaker":null,"is_sponsor":0},{"start_s":208.8,"end_s":213.44,"text":"they put E cores on a desktop CPU where you aren't worried about your battery running out,","speaker":null,"is_sponsor":0},{"start_s":213.44,"end_s":218.88,"text":"but because E cores are physically smaller, you can actually pack more of them onto a chip,","speaker":null,"is_sponsor":0},{"start_s":218.88,"end_s":224.88,"text":"meaning you can in some cases get better performance if the scheduler can spread a program out","speaker":null,"is_sponsor":0},{"start_s":224.88,"end_s":230.72,"text":"across many E cores compared to a smaller number of P cores. Now, even though the Windows scheduler","speaker":null,"is_sponsor":0},{"start_s":230.72,"end_s":237.28,"text":"will still use P cores first by default if they're not busy, it can then use an E core for a second","speaker":null,"is_sponsor":0},{"start_s":237.28,"end_s":243.52,"text":"thread before using the slower hyperthreading feature of a P core. We're also getting to the","speaker":null,"is_sponsor":0},{"start_s":243.52,"end_s":248.08,"text":"point where, although Windows is still making final decisions on scheduling, there are now","speaker":null,"is_sponsor":0},{"start_s":248.08,"end_s":255.04,"text":"hardware assist features on newer CPUs that will actually give advice to Windows, so a CPU can look","speaker":null,"is_sponsor":0},{"start_s":255.04,"end_s":260.96,"text":"at the incoming workload and then advise Windows on the best current core for a certain thread,","speaker":null,"is_sponsor":0},{"start_s":260.96,"end_s":265.04,"text":"depending on whether Windows wants more performance or better power efficiency.","speaker":null,"is_sponsor":0},{"start_s":265.6,"end_s":271.04,"text":"So there's a lot that goes into deciding who gets to go first when it comes to your CPU,","speaker":null,"is_sponsor":0},{"start_s":271.04,"end_s":277.68,"text":"and a multi-core model can legitimately multitask. Also, my wife can do it somehow.","speaker":null,"is_sponsor":0},{"start_s":277.68,"end_s":284.16,"text":"If you guys enjoyed this video, check out our other videos, leave a comment if you have a suggestion for a future fastest Techquickie,","speaker":null,"is_sponsor":0},{"start_s":284.16,"end_s":285.52,"text":"and don't forget to subscribe.","speaker":null,"is_sponsor":0}],"full_text":"If I tell you I'm multitasking, don't believe me, the best that I can do as a weak pitiful human is to quickly switch my attention from one single thing to another. But is that how your CPU works? Actually, no. Your CPU relies on a strategy called scheduling, where the operating system will decide what sets of instructions or threads will get put through the CPU first, second, third, and so on. Understanding scheduling is crucial to understanding exactly how modern multi-core CPUs work. So we got a little bit of help for this video, and we'd like to thank our friends at Intel for their assistance as we run through this thought experiment. Each thread that your computer is working on gets assigned a value called priority, so a thread with a higher priority value will be further ahead in line to get processed by your CPU. Developers can assign priority values when they write a program, and Windows can change a thread's priority based on how many system resources it needs and what the exact nature of the instructions are. For example, if a thread is tied to what you're inputting with a keyboard and mouse, such as clicking buttons in your web browser or aiming your weapon in a game, it's likely going to get a higher priority. By contrast, background activities that aren't time sensitive, those will get a lower priority. Similarly, developers can assign another value called quality of service, or QOS, to each thread. QOS is a way for devs to signal to Windows how much processor power a thread might need, and while Windows doesn't treat it as important as priority, QOS is still a factor that the operating system will take into account when it comes to scheduling. QOS is especially relevant when it comes to power management, as the OS can use QOS to request a certain frequency from the CPU, and the CPU's hardware can decide if it can ramp up to that frequency while staying within power limits. That's especially useful on laptops. Which is all very cool, but in terms of our guy doing two things at once analogy, priority and QOS pertain more to what I'm choosing to do, like texting while watching a movie, rather than how I'm juggling those two tasks. So let's drill down into that. Traditionally, CPU threads would go through one at a time, like my brain, but this isn't how modern multicore systems operate, especially considering some newer chips even have different kinds of cores all on one package. One goal in scheduling on multicore is not to context switch more than is necessary. And context switching is just another fancy term for moving a thread from one core onto a different one. In general, you want each thread to start and finish on one single core, and you want to load balance across cores so that none of them are getting overwhelmed. And there's been a lot of tweaking over the years to figure out how to do this easily and efficiently, such as with libraries, which are software resources that sit between the program and the operating system and allow multi-threading without making a developer do a ton of extra coding. On newer chips, how exactly threading works out across cores becomes even more important because of hybrid chips like Intel's Alder Lake lineup. These chips have higher performance, or P cores, and lower performance, but more power saving, E cores. It might be confusing as to why they put E cores on a desktop CPU where you aren't worried about your battery running out, but because E cores are physically smaller, you can actually pack more of them onto a chip, meaning you can in some cases get better performance if the scheduler can spread a program out across many E cores compared to a smaller number of P cores. Now, even though the Windows scheduler will still use P cores first by default if they're not busy, it can then use an E core for a second thread before using the slower hyperthreading feature of a P core. We're also getting to the point where, although Windows is still making final decisions on scheduling, there are now hardware assist features on newer CPUs that will actually give advice to Windows, so a CPU can look at the incoming workload and then advise Windows on the best current core for a certain thread, depending on whether Windows wants more performance or better power efficiency. So there's a lot that goes into deciding who gets to go first when it comes to your CPU, and a multi-core model can legitimately multitask. Also, my wife can do it somehow. If you guys enjoyed this video, check out our other videos, leave a comment if you have a suggestion for a future fastest Techquickie, and don't forget to subscribe."}