WEBVTT 00:00:00.880 --> 00:00:09.519 165,000 CPU cores, 20 million of GPU, 00:00:06.480 --> 00:00:12.400 and a cool pabyte of RAM. I wouldn't 00:00:09.519 --> 00:00:18.160 normally describe myself as a fur bree, but the new fur supercomput has 00:00:15.280 --> 00:00:22.720 definitely awakened some feelings that I can't say I've ever felt before. 00:00:19.920 --> 00:00:29.359 Feelings like wanting to go deep inside it, to gently remove its panels, and 00:00:25.920 --> 00:00:31.039 maybe some light screwing. And thanks to 00:00:29.359 --> 00:00:34.480 our friends here at Simon Fraser University in beautiful British 00:00:32.640 --> 00:00:40.399 Columbia, we are going to be doing just that. Going deep under the hood of the 00:00:37.120 --> 00:00:42.640 CPU and GPU compute deployment that is 00:00:40.399 --> 00:00:47.440 going to be serving tens of thousands of scientists and researchers in fields all 00:00:44.879 --> 00:00:52.559 the way from AI to zoology all over the country for years to come. This will be 00:00:49.760 --> 00:00:57.039 our first upclose look at a realworld deployment that uses direct die liquid 00:00:55.039 --> 00:01:03.199 cooling to increase cooling efficiency from about 30% to over 90%. Or at least 00:01:01.120 --> 00:01:09.200 it'll be the first data center grade deployment. Mine doesn't count and it 00:01:06.240 --> 00:01:16.880 doesn't look nearly as sexy. But what is sexy is this segue to our sponsor MSI. 00:01:14.000 --> 00:01:20.799 MSI's MAG B850 Tomahawk Max Wi-Fi motherboard has bells, whistles, and a 00:01:19.280 --> 00:01:25.200 bunch of other things that make noise. With support for the latest AMD CPUs, 00:01:22.960 --> 00:01:31.799 DDR5, and a slew of unique features, it's a performance monster. Check it out 00:01:27.280 --> 00:01:31.799 today using our link down below. 00:01:38.960 --> 00:01:47.280 In the row behind me is 640 NVIDIA H180 00:01:43.759 --> 00:01:49.340 GB GPUs, each with an estimated cost of 00:01:47.280 --> 00:01:50.640 around $31,000. 00:01:50.640 --> 00:01:58.719 Even at less than half of the maximum density, just 20 nodes per rack, the 00:01:56.399 --> 00:02:02.399 team here had to reroute power from elsewhere in the building and 00:02:00.640 --> 00:02:07.600 significantly upgrade the building's cooling system just to accommodate the 00:02:05.040 --> 00:02:11.920 incredible power requirements of these NVIDIA hoppers. This is actually a 00:02:10.319 --> 00:02:16.400 common theme that I hear from basically anyone in the data center space. I mean, 00:02:14.239 --> 00:02:20.879 we tried to build for the future, but we couldn't have possibly seen this coming. 00:02:19.040 --> 00:02:25.920 And there's no sign of things slowed down. We'll get to that later, though. 00:02:23.680 --> 00:02:30.319 First, the most exciting part of the tour. They pulled one of their spares 00:02:27.920 --> 00:02:38.160 out of the rack for us to crack open and get up close and personal with. And oh 00:02:33.760 --> 00:02:41.040 my god, look at this thing. 00:02:38.160 --> 00:02:47.360 It's heavy. I guess when you got this much copper in you, like, wow. 00:02:44.879 --> 00:02:52.000 It's kind of scary handling it. I mean, this one you node alone is worth more 00:02:49.680 --> 00:02:56.239 than my car. And a rack of these is worth more than my house. 00:02:54.560 --> 00:03:02.000 It's a little sketchy, but I want you guys to be able to see it. The CPUs are 00:02:58.879 --> 00:03:05.360 epic genoa. So, last generation Zen 00:03:02.000 --> 00:03:09.120 4-based, but Genova still supports up to 00:03:05.360 --> 00:03:12.080 12 channel DDR5 memory and 128 lanes of 00:03:09.120 --> 00:03:18.319 PCIe Gen 5, which is plenty to keep these GPU cores fed. if more CPU compute 00:03:15.680 --> 00:03:25.120 is needed. Clearly, there is support for dual CPU sockets, but the team at SFU 00:03:21.519 --> 00:03:27.200 found that eight CPU cores per GPU was 00:03:25.120 --> 00:03:35.599 plenty for their purposes, and they opted for a single 48 core CPU and 1.152 00:03:31.920 --> 00:03:37.519 TB of RAM in each of their nodes. Now, 00:03:35.599 --> 00:03:42.159 for a closer look at the GPUs. Unfortunately, I'm not allowed to take 00:03:39.519 --> 00:03:49.680 the coolers off them. But under each of these four cold plates is an NVIDIA H100 00:03:45.680 --> 00:03:54.560 SXM5 80 gig GPU, giving us a total of 00:03:49.680 --> 00:03:57.519 320 GB of VRAM per node. And guys, 00:03:54.560 --> 00:04:05.760 that's not just any VRAM. That is HBM3 running on a 5,120 bit bus for a total 00:04:01.280 --> 00:04:08.640 bandwidth per GPU of 3.36 terabytes per 00:04:05.760 --> 00:04:15.280 second. For context, a top-of-the-line consumer card, the RTX 5090, achieves 00:04:11.760 --> 00:04:17.519 just over half of that bandwidth. This 00:04:15.280 --> 00:04:24.880 kind of power does come with drawbacks, however, like for example, heat. Each of 00:04:20.959 --> 00:04:27.040 these is rated for 700 W of power 00:04:24.880 --> 00:04:30.720 consumption through the SXM socket that's underneath them. And that is 00:04:28.560 --> 00:04:35.759 where the incredible cooling solution in this Lenovo Node comes in. As a liquid 00:04:33.280 --> 00:04:41.120 cooling nerd, I got to say, guys, this is the coolest part for me. I mean, did 00:04:38.479 --> 00:04:49.199 you notice that there isn't a single fan in sight anywhere in this machine? That 00:04:44.240 --> 00:04:52.720 is because everything, CPUs, GPUs, VRM, 00:04:49.199 --> 00:04:56.560 network interface, SSD caddy, even the 00:04:52.720 --> 00:04:58.400 system memory is directly liquid cooled. 00:04:56.560 --> 00:05:03.120 All of it. This feels a little bit like doing the maze and highlights magazine. 00:05:00.800 --> 00:05:07.520 So, here's our inlet over here, which splits into two main loops that go 00:05:05.360 --> 00:05:11.039 through the system. The primary loop, which we can tell because it has a 00:05:09.039 --> 00:05:16.080 thicker pipe coming off of it, goes straight to the middle of our four GPUs, 00:05:13.520 --> 00:05:21.919 where this manifold splits fresh incoming water out to our four GPUs. Two 00:05:19.680 --> 00:05:26.720 of them just spit right back into the outlet here, while the other two run up 00:05:24.479 --> 00:05:32.240 to this networking board and then consolidate back to the outlet. That's 00:05:29.520 --> 00:05:38.240 our primary loop. Our secondary loop comes through here handling some of the 00:05:34.320 --> 00:05:42.320 power delivery and then carries over to 00:05:38.240 --> 00:05:44.080 interesting. It splits out doing the RAM 00:05:42.320 --> 00:05:49.039 next. I am not 100% sure what to make of that 00:05:46.880 --> 00:05:53.759 because I would think RAM would be a tertiary priority in terms of cooling, 00:05:51.600 --> 00:05:58.080 but that's what they've done. We go through the RAM splitting into three 00:05:56.160 --> 00:06:03.919 different tubes that sit between our DIMs down both rows. Then one side 00:06:01.520 --> 00:06:09.280 handles this network caddy here and the other side handles our SSD caddy. Then 00:06:07.039 --> 00:06:15.520 each of those come back to one of the CPUs which come out into the middle here 00:06:12.560 --> 00:06:19.600 and then run back to the outlet here. Not maybe the way I would have laid it 00:06:17.280 --> 00:06:23.840 out. There's a lot of 90° turns in here, meaning a lot of restriction. But I'm 00:06:22.160 --> 00:06:27.600 sure the engineers at Lenovo know what they're doing. There's a ton of other 00:06:25.680 --> 00:06:31.840 cool stuff to unpack here, too. You probably noticed there's no power 00:06:28.880 --> 00:06:35.840 supply. That's because it uses these chunky connectors here at the back to 00:06:34.080 --> 00:06:40.240 plug into a back plane in the back of the rack. As for the coolant 00:06:37.919 --> 00:06:45.440 connections, well, according to the manufacturer, these do have a little bit 00:06:42.720 --> 00:06:52.000 of natural leakage, but uh it's on the order of molecules, which is pretty damn 00:06:48.880 --> 00:06:54.080 impressive. There are sensors all over 00:06:52.000 --> 00:06:58.479 the motherboard to detect any kind of leakage. Now, the team here wasn't sure 00:06:56.400 --> 00:07:02.240 about the exact chemistry of the coolant they're using, but they did tell me that 00:07:00.240 --> 00:07:06.400 it has antimicrobial properties to prevent anything from growing in the 00:07:03.759 --> 00:07:10.639 loop. There's some other fun stuff. There's a little stylus in here. 00:07:08.479 --> 00:07:14.639 Apparently, this is meant to uh assist in removing memory, which is great. I'd 00:07:13.440 --> 00:07:20.960 actually love to see more gaming motherboards come with that. 00:07:17.120 --> 00:07:23.120 I thought this lone 7.68 TB NVMe drive 00:07:20.960 --> 00:07:30.960 was interesting, too. I mean, the networking is 400 GB per second* 2 to 00:07:27.759 --> 00:07:33.440 the 2 pabytes of NVMe storage, not to 00:07:30.960 --> 00:07:38.000 mention 49 pabytes of spinning Rust that's right over there. But according 00:07:35.759 --> 00:07:42.639 to the team here, occasionally they need node local storage to improve GPU 00:07:40.560 --> 00:07:46.880 performance a little bit. So, you'd never boot off of this or anything, but 00:07:44.639 --> 00:07:52.000 it's nice to have there as a scratch. Also, the button cell in here is mounted 00:07:49.520 --> 00:07:56.800 in a vertical caddy because the density is so high in this one you node that 00:07:54.879 --> 00:08:01.360 they just couldn't give up the space that it would have taken to mount it 00:07:58.240 --> 00:08:03.919 parallel to the board. I also spotted a 00:08:01.360 --> 00:08:08.000 micro SD header. If anyone out there works in the data 00:08:05.680 --> 00:08:12.400 center and knows what that's for, I haven't seen it before. And Jim and I 00:08:09.919 --> 00:08:17.520 just assumed that I typed. Oh, there was one other thing we wanted 00:08:13.919 --> 00:08:19.199 to look at. These big power bad boys. We 00:08:17.520 --> 00:08:24.560 couldn't see them until we got that shroud off. So these they're just bus 00:08:22.400 --> 00:08:29.120 bars. They're going from power supply here, which is a DC toDC power supply, 00:08:27.039 --> 00:08:34.800 >> 48 volt it looks like, and they're going over to our GPUs. Damn. What's 00:08:32.560 --> 00:08:39.839 interesting to me that I just noticed is that there's a clear delineation between 00:08:37.120 --> 00:08:44.320 the NVIDIA engineered parts of this with the black PCB and they're completely 00:08:42.399 --> 00:08:48.240 separate from everything else and the Lenovo engineered parts of this. So 00:08:46.399 --> 00:08:53.279 Lenovo is acting like more of a system integrator around this compute block 00:08:51.440 --> 00:08:58.880 here. Like you can even see the silk screening on the PCB is distinctly 00:08:56.160 --> 00:09:05.200 NVIDIA and Lenovo's just doing their DC toDC power. So, it's just power in here 00:09:02.000 --> 00:09:08.000 and then PCIe in here in the form of 00:09:05.200 --> 00:09:13.920 these four MCIO connectors right here. This is essentially like plugging a GPU 00:09:10.720 --> 00:09:15.200 into your Legion gaming PC. 00:09:13.920 --> 00:09:20.720 >> The the GPU house >> with extra steps. >> Yeah. 00:09:17.519 --> 00:09:23.200 >> Before we poke around in one of the 192 00:09:20.720 --> 00:09:28.640 core CPU nodes that they've got, let's take a look at one of the racks that 00:09:24.880 --> 00:09:31.200 these boys slide into. 00:09:28.640 --> 00:09:35.760 They're still using a very similar rear door chilled liquid rack like we saw 00:09:33.839 --> 00:09:44.399 with their air cooled modes when we did a tour oh lordy 8 years ago. Um 00:09:40.800 --> 00:09:46.720 anywh who the point is that chilled 16 00:09:44.399 --> 00:09:52.800 1/2° cooling comes from the evaporative cooling towers outside. Then hot air 00:09:50.160 --> 00:09:57.600 from the power supplies and any of the networking equipment that's in the rack 00:09:54.880 --> 00:10:05.040 runs through here and wow is that ever hot. Then it spits out nice comfortable 00:10:02.240 --> 00:10:10.560 room temperature air on the other side. Each of these racks is fed by dual 00:10:07.760 --> 00:10:18.399 three-phase 60 amp feeds for a total of about 70,000 watts per rack. Now, if SFU 00:10:15.200 --> 00:10:21.360 had the power and cooling in this 1960s 00:10:18.399 --> 00:10:26.240 bunker, they could juice these up to 180,000 watts per rack. But they don't. 00:10:23.920 --> 00:10:32.640 Hence the empty rack space. Since we have this open, oh wow, that is a big 00:10:30.720 --> 00:10:37.680 difference between the cold side and the hot side going into the the back of 00:10:35.200 --> 00:10:42.240 these back planes for the servers. I don't have to ask which one's the 00:10:39.120 --> 00:10:43.760 supply. That's the cold side, which 00:10:42.240 --> 00:10:48.000 since we're on the subject, this is a perfect time to look at the cooling 00:10:45.519 --> 00:10:54.160 distribution system. This is the Liber XPU from Vertip. It can do 600,000 00:10:51.839 --> 00:10:59.360 watts of cooling capacity per one of these cooling distribution units or 00:10:56.320 --> 00:11:03.120 CDUs. Water comes in the supply side 00:10:59.360 --> 00:11:05.040 here. This thick boy, she's chilly. 00:11:03.120 --> 00:11:10.959 That's coming from the cooling towers outside. Then that runs all the way down 00:11:08.800 --> 00:11:15.200 to the bottom here to the heat exchanger in the front. This liquid toliquid heat 00:11:13.200 --> 00:11:20.079 exchanger does exactly what it says on the tin. Taking that cold water from the 00:11:17.440 --> 00:11:24.800 primary loop that goes outside and using it to chill the warm water that is 00:11:22.399 --> 00:11:30.959 coming directly off of the blocks that are going to our nose. This unit uses 00:11:27.600 --> 00:11:33.600 dual redundant pumps. And if we go back 00:11:30.959 --> 00:11:38.560 around to the back, uses these manifolds and valves to control flow to up to six 00:11:36.640 --> 00:11:43.680 different racks. And it's very easy to tell which is the cold side that's being 00:11:40.880 --> 00:11:48.720 chilled in, which is the hot side here. Wow. 00:11:45.920 --> 00:11:51.760 I want one. Von, can I have one? Probably the coolest part is this little 00:11:50.240 --> 00:11:55.600 touchcreen display on the front that much more succinctly illustrates what I 00:11:53.680 --> 00:11:59.040 just said. Here's your primary loop. Here's your secondary loop. Here's all 00:11:57.360 --> 00:12:04.640 your flow rates, all your temperatures, and here's an alarm that they assure me 00:12:01.040 --> 00:12:06.480 is totally fine. 00:12:04.640 --> 00:12:11.120 This data is super important because if they accidentally had water that was too 00:12:08.800 --> 00:12:14.240 cool going into the servers behind me, then they could end up with 00:12:12.000 --> 00:12:18.880 condensation, which hopefully I don't have to explain why that's super super 00:12:16.320 --> 00:12:22.639 bad. Everything's hooked up using AquaM tubing from Germany. The admin here 00:12:21.040 --> 00:12:27.120 spoke with some other facilities that used stainless steel and one of them got 00:12:24.880 --> 00:12:30.720 rust in their cooling system. is a big big mess. They've been really really 00:12:28.959 --> 00:12:34.560 happy with their offerings. Now, let's go check out a CPU mode. Contrary to 00:12:33.120 --> 00:12:39.200 what NVIDIA would like everyone to believe, not everything runs best on a 00:12:36.720 --> 00:12:45.920 GPU, even today. And that's where these come in. Each of these one U racks 00:12:42.720 --> 00:12:51.680 contains two nodes. And each node 00:12:45.920 --> 00:12:55.440 contains 192 Zen 5 epic Turing cores 00:12:51.680 --> 00:12:59.519 with 768 gigs of memory. So that's a 00:12:55.440 --> 00:13:04.160 total of nearly 400 cores in each of 00:12:59.519 --> 00:13:06.399 these one use. Holy freaking 00:13:04.160 --> 00:13:12.160 for networking. They actually don't go as heavy on these using 200 gig 00:13:08.880 --> 00:13:15.120 connections and NDR to dynamically share 00:13:12.160 --> 00:13:19.680 that 200 Gbit link between the two nodes depending on their needs. This approach 00:13:17.760 --> 00:13:23.839 does have the drawback of meaning that if the primary node goes down, we lose 00:13:22.160 --> 00:13:28.720 network connection to the secondary one. But I have to assume that the cost 00:13:25.760 --> 00:13:34.240 savings outweigh the disadvantages in this case. In terms of loop layout, this 00:13:31.760 --> 00:13:40.160 one is much simpler coming in to both sides and then out of both sides. But 00:13:37.600 --> 00:13:44.399 just like the GPU nodes, the goal here is to get a water tube up against pretty 00:13:42.959 --> 00:13:48.639 much anything in the server that generates heat because there are no fans 00:13:47.040 --> 00:13:52.639 whatsoever. One cool thing we missed on the GPU node 00:13:50.720 --> 00:13:56.399 was we never got to look under the little cooling plates that the SSDs and 00:13:54.639 --> 00:14:01.519 network cards sit on. So, here's what it looks like. It pretty much looks like a 00:13:59.120 --> 00:14:05.519 heat pipe except instead of being full of what is usually a vapor and sometimes 00:14:03.920 --> 00:14:10.160 a liquid that circulates just within itself, it's just full of water or other 00:14:08.320 --> 00:14:15.279 coolant that is circulating to an external system. Now, let's take a look 00:14:12.639 --> 00:14:20.959 at the racks that these live in. Each of these racks contains 72 of the nodes 00:14:18.959 --> 00:14:26.120 that I just showed you guys, top to freaking bottom, with roughly 13,824 00:14:26.160 --> 00:14:33.120 cores. Each three racks is an island 00:14:30.480 --> 00:14:39.600 with a non-blocking 800 gig connection between islands. So, 41,000 cores can 00:14:36.639 --> 00:14:43.760 represent a single job with no blocking. They have some other specialized nodes 00:14:41.920 --> 00:14:47.600 like the storage ones, including the ones on the other side of the aisle that 00:14:45.519 --> 00:14:51.839 hold data for our local particle collider. Try them. We've got a whole 00:14:49.519 --> 00:14:55.839 video about that. Along with some 8 TB RAM nodes, which are, I think, pretty 00:14:53.760 --> 00:14:59.839 self-explanatory. They're for jobs that would overflow on a regular node as long 00:14:58.320 --> 00:15:05.519 as you don't mind them having a few bugs. And finally, a single AMD MI300X 00:15:03.600 --> 00:15:09.199 node to I don't know what, keeping video on the 00:15:07.199 --> 00:15:13.680 toes or We could do it. We could buy more than 00:15:11.279 --> 00:15:18.160 one of these. You better not charge too much, especially when you factor in 00:15:15.519 --> 00:15:22.959 modern security needs. There are six zones of security to get to some of the 00:15:20.320 --> 00:15:26.560 cages that actually have biometric locks on them where not only do you need to 00:15:24.480 --> 00:15:30.800 know the pin code, but you have to put your hand under it and it will check if 00:15:28.880 --> 00:15:35.279 that hand is attached to a living person. There's cameras everywhere in 00:15:33.440 --> 00:15:39.120 the data center with full visibility in all directions. And our tour guide today 00:15:37.519 --> 00:15:42.560 actually said that there's someone monitoring them so often that it's 00:15:40.880 --> 00:15:47.680 become a bit of a game where they'll send non-flattering pictures of him 00:15:44.720 --> 00:15:51.920 moving around in the data center just to make sure he knows they're watching. 00:15:49.680 --> 00:15:56.160 Cooling everything are these evaporative cooling towers behind me. The three that 00:15:54.480 --> 00:15:59.360 are closest to the building, they were there the last time we were here, but I 00:15:58.000 --> 00:16:04.639 couldn't show them to you for reasons that involve red tape and approvals. So, 00:16:02.240 --> 00:16:08.480 here they are. I still can't get any closer to them for reasons that involve 00:16:06.160 --> 00:16:12.399 red tape and approvals, but hey, we can check out the acoustic damping that's on 00:16:10.560 --> 00:16:17.199 these ones on the other side. That's impressive. Here I am right at the 00:16:14.639 --> 00:16:22.959 intake next to these sound baffles. And for context, here's the untreated ones. 00:16:20.000 --> 00:16:28.320 The total cooling capacity is about 4.7 megawatt, which that is way more than 00:16:26.079 --> 00:16:32.399 what's needed for the machines inside. But just like power and storage, you 00:16:30.480 --> 00:16:36.240 want to have some extra for resiliency in the event of an equipment failure. 00:16:34.480 --> 00:16:39.920 Hey, what's one of these worth? Maybe I'll pick one up for the office. 00:16:37.759 --> 00:16:43.680 >> $1.2 million each. >> Oh, 00:16:41.680 --> 00:16:50.320 never mind. Frankly, I'd rather have one of these anyway. To augment the original 00:16:46.720 --> 00:16:52.720 pumps for Cedar, which could do 800 gall 00:16:50.320 --> 00:16:59.440 per minute of coolant, they added these two new ones that do 1,500 00:16:56.320 --> 00:17:01.759 gallons per minute. They also now have 00:16:59.440 --> 00:17:06.880 two mechanical chillers which can be useful during the times of year that we 00:17:03.519 --> 00:17:09.039 get outside temperatures above 33 C. So 00:17:06.880 --> 00:17:12.720 for maybe 6 hours a day, they'll switch over to mechanical chilling to help out 00:17:11.199 --> 00:17:18.480 their evaporative cooling tower. Probably the coolest thing about this gear though is how smart it is. They've 00:17:16.559 --> 00:17:22.880 got telemetry capture for things like temperature and flow rates, and it all 00:17:20.559 --> 00:17:25.919 feeds into a third party called Kaizen that helps with logging and determining 00:17:24.640 --> 00:17:31.039 if something's gone wrong with the system. Fun fact, by the way, the twoft 00:17:29.120 --> 00:17:36.000 thick concrete floor that I'm standing on is so burdened by all of this heavy 00:17:33.919 --> 00:17:42.640 equipment and coolant that it actually deflects half an inch in the center. Is 00:17:39.440 --> 00:17:44.960 that Is that okay? Am I going to break 00:17:42.640 --> 00:17:48.799 it? I mean, the place used to be the power distribution center for the 00:17:46.559 --> 00:17:53.840 southern half of our province, and it's built like a bunker, but that's not 00:17:51.039 --> 00:18:01.039 enough. Who paid for it all? Fur had a total budget of about $82 million which 00:17:57.120 --> 00:18:03.760 oh I assume is Canadian rubbledos. Uh so 00:18:01.039 --> 00:18:08.480 a little under $60 million US and that came from a combination of the digital 00:18:05.440 --> 00:18:10.880 research alliance of Canada, BCKDF and 00:18:08.480 --> 00:18:14.480 vendor inind contributions which I just learned are a vendor giving significant 00:18:12.960 --> 00:18:18.080 discounts. How how do I get signed up for that program? Is that only for 00:18:16.080 --> 00:18:21.919 educational institutions? Anyway, they did ask us to shout out a 00:18:20.320 --> 00:18:27.280 couple of companies who helped them out. Lenovo, DDN, and Vertive on the cooling 00:18:25.440 --> 00:18:30.640 side. Um, and they didn't ask us to shout these guys out, but we're going to 00:18:28.480 --> 00:18:34.000 do it anyway. A shout out for our sponsor, Vessie. We're partnering up 00:18:32.480 --> 00:18:39.120 with them to bring you an exclusive giveaway extravaganza all month long. By 00:18:37.440 --> 00:18:43.440 entering, you'll get a chance to win one of seven prize packs, each loaded with 00:18:41.120 --> 00:18:46.400 Vessie gear, LT Storm merch, and more. We're talking footwear like their 00:18:44.720 --> 00:18:51.120 Stormbburst hightops or bags like our very own commuter backpack, Sony XM6 00:18:48.799 --> 00:18:55.120 headphones, and some prizes so special we can't even tell you about them yet. 00:18:52.880 --> 00:18:57.679 It's 100% free to enter. But if you're looking for some stylish shoes to 00:18:56.400 --> 00:19:01.360 protect yourself from the rain right now, well, we've got a special discount 00:18:59.679 --> 00:19:05.280 for you that's valid until the end of the month, 20% off. So, head over to 00:19:03.600 --> 00:19:10.000 vessie.com/LMG to enter the giveaway today and get an 00:19:07.360 --> 00:19:14.720 instant 20% off code sent directly to your inbox. Act fast, though. Everything 00:19:12.000 --> 00:19:19.360 closes on October 31st. Once again, that's vessie.com/LMG. 00:19:17.520 --> 00:19:22.559 Take part in an exclusive giveaway and secure your 20% off code. 00:19:20.960 --> 00:19:26.000 >> If you guys enjoyed this video, why not check out the tour we did of Triumph, 00:19:24.160 --> 00:19:31.520 the particle accelerator that is just down the road. Well, down the hill, down 00:19:28.960 --> 00:19:35.120 the really long road. Canada's only road. It's pretty long.