{"video_id":"o1qg272T_gU","title":"The State of Labs 2025","channel":"FP Exclusives","show":"FP Exclusives","published_at":"2025-09-06T14:53:29Z","duration_s":2390,"segments":[{"start_s":0.0,"end_s":2.335,"text":"Luke: Sorry my brain is effed right now. 2 00:00:02.669 --> 00:00:05.405 Luke: My sleep was really not great last night. 3 00:00:05.538 --> 00:00:09.476 Luke: Hey Floatplane, it's been almost a year since I joined the Labs; 4 00:00:09.576 --> 00:00:12.612 and we're gonna talk about what's happened. 5 00:00:12.712 --> 00:00:16.616 Clearly we've reorganized a bit, so we might as well start talking off with- 6 00:00:16.616 --> 00:00:17.684 Luke: Talking off? 7 00:00:19.786 --> 00:00:21.955 Luke: We're gonna introduce ourselves; my name is Luke, 8 00:00:22.022 --> 00:00:22.455 Nick: I- 9 00:00:22.522 --> 00:00:23.256 Luke: I do stuff. 10 00:00:26.559 --> 00:00:27.761 Nick: Why do you sound like you're... 11 00:00:27.761 --> 00:00:28.728 Luke: I interrupt Nick. 12 00:00:29.963 --> 00:00:34.067 Nick: Hi, I'm Nick; I'm a software development and testing supervisor. 13 00:00:35.101 --> 00:00:39.706 Lucas: I'm Lucas; I test power supplies and do some other supervision of you know, 14 00:00:39.839 --> 00:00:42.709 developing verticals and the website a little bit, publication. 15 00:00:42.709 --> 00:00:44.811 Luke: And some writing, and... 16 00:00:44.811 --> 00:00:45.779 Lucas: Some occasional writing, 17 00:00:45.779 --> 00:00:46.413 Luke: Stuff. 18 00:00:46.413 --> 00:00:47.213 Lucas: and a bunch of random stuff. 19 00:00:47.213 --> 00:00:49.482 Lucas: Calibrations, you know, look after a bunch of stuff. 20 00:00:51.451 --> 00:00:54.788 Luke: In regards to team and operations, we're hiring; 21 00:00:54.788 --> 00:00:57.590 we're trying to gain ability to test more things, 22 00:00:57.590 --> 00:00:58.925 that's one of the ideas around hiring. 23 00:00:58.925 --> 00:01:01.761 Luke: We've increased the testing team pretty significantly, 24 00:01:01.761 --> 00:01:04.464 and we're looking to increase the testing team even more. 25 00:01:04.564 --> 00:01:08.868 What do we even call them? I know Subject Matter Experts was like a title for a while. 26 00:01:09.002 --> 00:01:10.236 Lucas: Steward of the verticals, 27 00:01:10.370 --> 00:01:10.970 Luke: Yeah. 28 00:01:10.970 --> 00:01:13.306 Lucas: you know, CPUs or GPUs or whatever else. 29 00:01:13.339 --> 00:01:19.412 Luke: Basically our testing verticals, or like a type of products that we do testing on 30 00:01:19.446 --> 00:01:21.614 like GPUs or CPUs or whatever else. 31 00:01:21.648 --> 00:01:24.984 We need people to make sure that those tests are staying up to date 32 00:01:24.984 --> 00:01:26.519 and progressing with the industry. 33 00:01:26.586 --> 00:01:30.890 It doesn't necessarily mean that you would have to come in already having 34 00:01:30.890 --> 00:01:34.761 like absolute perfect domain expertise, but it means that 35 00:01:34.761 --> 00:01:38.598 you should be good at researching and keeping up with the curve, 36 00:01:38.598 --> 00:01:43.236 and keeping us on the leading edge or at least caught up with the leading edge 37 00:01:43.403 --> 00:01:45.772 for these various verticals. 38 00:01:45.772 --> 00:01:48.975 Sometimes that'll result in needing to work with someone like a Nick, 39 00:01:48.975 --> 00:01:53.580 because if you're going to try to help with say CPU or GPU, 40 00:01:53.613 --> 00:01:56.216 well that's going to be directly tied into Markbench, 41 00:01:56.216 --> 00:01:58.218 or some type of system that we already have; 42 00:01:58.251 --> 00:02:01.020 so it's going to end up being decently collaborative, but- 43 00:02:01.121 --> 00:02:04.791 Sammy: You can tell you're hiring because this is the whole new lab to set up right? 44 00:02:04.891 --> 00:02:05.725 Sammy: In the background. 45 00:02:06.159 --> 00:02:07.327 Luke: That means we're hiring? 46 00:02:07.460 --> 00:02:09.729 Sammy: Well, like you're looking to expand more you know? 47 00:02:09.729 --> 00:02:12.265 Nick: Every time I move a desk I hire someone. 48 00:02:13.199 --> 00:02:15.635 Sammy: No, well like you got like a lot more space, 49 00:02:15.635 --> 00:02:17.303 you had the little things on top now. 50 00:02:17.303 --> 00:02:17.637 Sammy: Like- 51 00:02:17.637 --> 00:02:21.608 Luke: Yeah, like technically we don't actually have any more space at all. 52 00:02:21.774 --> 00:02:25.845 It just feels like we have a ton more space because how the lab used to be set up; 53 00:02:25.945 --> 00:02:30.183 it was, it was sort of divided into like one third and then two thirds, 54 00:02:30.183 --> 00:02:32.385 I think is maybe somewhat accurate. 55 00:02:32.485 --> 00:02:37.390 Luke: And we've kind of moved things around and made it more of like a pit design. 56 00:02:37.390 --> 00:02:42.896 Luke: So we have the perimeter with the dimensions and storage shelves and stuff like that. 57 00:02:42.929 --> 00:02:48.868 And then there's the two general seating areas, one more designed for testers 58 00:02:48.868 --> 00:02:51.704 and the other one more designed for like everything that isn't testers. 59 00:02:52.539 --> 00:02:54.340 I can probably let you guys speak to that more 60 00:02:54.340 --> 00:02:57.177 because you had significantly more to do with designing this. 61 00:02:57.177 --> 00:02:59.412 I love it, I think this is way better. 62 00:02:59.546 --> 00:03:00.513 Nick: Yeah, Lucas? 63 00:03:02.282 --> 00:03:06.119 Lucas: It's a lot more flexible; so like a lot of our tables are on wheels now, 64 00:03:06.119 --> 00:03:08.121 and we have our project tables are on wheels, 65 00:03:08.254 --> 00:03:11.558 and all the GPU testing ventions and stations are on wheels; 66 00:03:11.958 --> 00:03:15.495 with all the drops like Sammy was saying for network and power. 67 00:03:15.495 --> 00:03:17.363 So we can move stuff around if needed, 68 00:03:17.497 --> 00:03:19.999 and if we expand with a bunch of people then we can do that. 69 00:03:20.099 --> 00:03:23.736 Or if we have some sudden project like Secret Shopper or all that stuff, 70 00:03:23.736 --> 00:03:24.537 takes a bunch of room. 71 00:03:24.604 --> 00:03:28.308 Luke: When you guys already had these vention desks, which are on wheels; 72 00:03:28.308 --> 00:03:31.311 but then all of your power was like hardwired into the ground, 73 00:03:31.344 --> 00:03:33.913 so you could move the desk but then you couldn't plug it in. 74 00:03:33.947 --> 00:03:38.651 Luke: So now it it seems like you're kind of realizing that vision even more. 75 00:03:38.685 --> 00:03:40.787 And I really like the project desk idea, 76 00:03:40.820 --> 00:03:45.091 because we'll run into things like Secret Shopper or whatever else; this is your example, 77 00:03:45.091 --> 00:03:47.427 where we might have a ton of different systems to test, 78 00:03:47.427 --> 00:03:51.431 Luke: or there's some huge keyboard thing showcase, whatever, words. 79 00:03:52.031 --> 00:03:53.533 Nick: Giant keyboard, massive. 80 00:03:53.533 --> 00:03:55.935 Luke: Basically, you need a bunch of space to work on things, 81 00:03:55.935 --> 00:03:59.038 and the Ventions don't really have any extra space, 82 00:03:59.138 --> 00:04:02.275 and often the stations don't really have any extra space. 83 00:04:02.275 --> 00:04:05.445 So by having these large work surfaces on wheels 84 00:04:05.445 --> 00:04:07.947 that are can just be moved around however we need; 85 00:04:07.947 --> 00:04:12.719 helps us be malleable to whatever projects might come in from the LTT team or whatever else. 86 00:04:12.885 --> 00:04:16.556 Because the routine testing that Labs does doesn't really need that so much, 87 00:04:16.689 --> 00:04:20.660 but some off the wall thing coming from somebody from the writing team. 88 00:04:20.960 --> 00:04:24.797 On here it says that we want to highlight that while it is really quick and easy 89 00:04:24.797 --> 00:04:26.666 to get simple proof of concept things together, 90 00:04:26.666 --> 00:04:29.902 it takes a long time to properly develop something and stand it up. 91 00:04:29.936 --> 00:04:32.238 Nick: Lucas has a lot to say on this actually. 92 00:04:32.372 --> 00:04:36.809 Lucas: Well it's just you know, it's easy to think like why don't they just do this? 93 00:04:36.809 --> 00:04:37.944 Why don't they just test this? 94 00:04:38.077 --> 00:04:41.214 And while it is there, it's easy to stand up a proof of concept 95 00:04:41.247 --> 00:04:44.284 or just test something and get to the point where you can show your buddies or whatever, 96 00:04:44.284 --> 00:04:45.118 maybe for a video. 97 00:04:45.218 --> 00:04:47.620 It takes a long time for the graphs to get generated, 98 00:04:47.754 --> 00:04:49.989 for the data pipeline to get worked through; 99 00:04:50.023 --> 00:04:53.559 make sure you're not doing it wrong or that you're collecting valuable information, 100 00:04:53.760 --> 00:04:54.994 so it takes a long time. 101 00:04:55.128 --> 00:04:57.096 Luke: Yeah, like do you roughly know how many tests? 102 00:04:57.230 --> 00:05:00.166 I know you've documented the number, but do you know off the top of your head 103 00:05:00.166 --> 00:05:02.135 how many tests we ran for the 5000 series launch? 104 00:05:02.235 --> 00:05:05.538 Because there's the like, oh yeah, just run a game on it dummy. 105 00:05:05.538 --> 00:05:06.272 Nick: It's a couple thousand. 106 00:05:06.272 --> 00:05:08.308 Luke: But then we run that many tests. 107 00:05:08.308 --> 00:05:08.741 Nick: Yeah. 108 00:05:08.741 --> 00:05:12.478 Luke: And then just like storing and sorting all of that data and that information, 109 00:05:12.478 --> 00:05:15.581 and actually getting something from it; knowing what games to test, 110 00:05:15.581 --> 00:05:18.918 knowing what settings to run them on, and then doing that same thing but for 111 00:05:18.918 --> 00:05:24.157 a bunch of verticals that might not be as straightforward as running a game on a computer. 112 00:05:24.490 --> 00:05:29.595 Nick: It's like on the surface, testing a graphics card or running a game is pretty- 113 00:05:29.595 --> 00:05:30.730 It's pretty simple. 114 00:05:30.763 --> 00:05:34.133 Like if we didn't, if we just wanted to get numbers into a video really quick, 115 00:05:34.133 --> 00:05:37.837 you just, we can run- They all have built in benchmarks, most of them right? 116 00:05:37.837 --> 00:05:39.339 You get your FPS out of it. 117 00:05:39.439 --> 00:05:43.443 You can record frame times with presentmon, jump that into a CSV and go. 118 00:05:43.576 --> 00:05:48.414 So it slows us down trying to do kind of create more of a framework for everything; 119 00:05:48.414 --> 00:05:53.119 so like we're doing a whole data- Like you know, an ecosystem so that when... 120 00:05:53.119 --> 00:05:59.959 so we can run a game, but we can run that game 50 times unattended. 121 00:06:00.360 --> 00:06:03.429 But it doesn't mean with automation where like press button, walk away. 122 00:06:03.429 --> 00:06:05.231 Like, I guess the FPS is okay then. 123 00:06:05.431 --> 00:06:09.235 Nick: It's like, no we're using all this automation to supplement so that we spend... 124 00:06:09.335 --> 00:06:13.506 We're sub design- Sub design- We're making it easier; 125 00:06:15.007 --> 00:06:20.780 so that technicians, instead of us spending time staring at a benchmark over and over again, 126 00:06:20.780 --> 00:06:26.519 or staring at the CSVs, we can spend more time that like exploratory testing. 127 00:06:26.519 --> 00:06:31.090 And we have systems in place that we'll talk about later about like how, 128 00:06:31.090 --> 00:06:33.826 how to ensure that we do catch anomalous errors. 129 00:06:33.960 --> 00:06:37.597 And by scaling up the testing we can actually work collaboratively 130 00:06:37.597 --> 00:06:39.899 five technicians on a single project, 131 00:06:40.333 --> 00:06:44.203 and run like thousands of tests for a single GPU project. 132 00:06:44.203 --> 00:06:49.776 Luke: Which is cool; and where alongside that we're building systems to try to detect, 133 00:06:49.776 --> 00:06:52.678 like early warning detection systems for potential problems. 134 00:06:53.212 --> 00:06:53.946 Nick: Not missiles. 135 00:06:54.313 --> 00:06:54.781 Luke: No. 136 00:06:54.781 --> 00:06:55.882 Yeah, yeah, not- Well... 137 00:06:56.382 --> 00:06:57.784 Luke: Yeah, not so much. 138 00:06:57.784 --> 00:06:58.251 Nick: Not yet. 139 00:06:58.985 --> 00:06:59.519 Luke: No, no, no. 140 00:06:59.852 --> 00:07:02.688 Luke: Yeah, hopefully flys at one point; we'll get into that later. 141 00:07:03.823 --> 00:07:07.193 Luke: Like, oh, a bunch of these tests were ran without G-Sync, 142 00:07:07.193 --> 00:07:08.861 and then one of them randomly had G-Sync on, 143 00:07:08.995 --> 00:07:13.833 flagging that because it would cause major issues is helpful 144 00:07:13.833 --> 00:07:18.671 to help reduce error output and also just help relax things. 145 00:07:18.938 --> 00:07:22.275 Because if one of the numbers comes out, out of whack 146 00:07:22.275 --> 00:07:25.645 and you just don't really have a lot of information to go on as to why, 147 00:07:25.778 --> 00:07:27.180 it can be really nerve wracking. 148 00:07:27.313 --> 00:07:31.717 If one of the numbers comes out of whack and then you can look into the records and be like, 149 00:07:31.717 --> 00:07:34.620 okay this setting or that setting or whatever was out of place. 150 00:07:34.787 --> 00:07:37.623 And then you can go back and retest it with those things corrected; 151 00:07:37.623 --> 00:07:40.560 and then it, the- The result falls back in line. 152 00:07:40.660 --> 00:07:43.129 Luke: Oh, okay; everything's fine. 153 00:07:43.296 --> 00:07:46.365 Nick: We're going for like extreme traceability. 154 00:07:46.499 --> 00:07:50.069 Nick: Versions, test results, Windows version; 155 00:07:50.069 --> 00:07:53.673 down to like, we're getting to the point where we're going to have like temperature clouds. 156 00:07:53.806 --> 00:07:55.641 Luke: Nice, I like it. 157 00:07:55.808 --> 00:08:00.346 Luke: There's a line here that says we also spend a bunch of time doing random other things. 158 00:08:00.346 --> 00:08:02.281 Nick: What are those random things? 159 00:08:02.415 --> 00:08:03.449 Lucas: Everything, I mean... 160 00:08:03.583 --> 00:08:07.553 Lucas: All the, all the random tests; you know, just takes a while. 161 00:08:07.553 --> 00:08:09.489 This is my main highlight for this is like, 162 00:08:09.722 --> 00:08:13.192 sometimes people are sick for a week you know, and then things get delayed. 163 00:08:13.192 --> 00:08:14.160 Luke: No, I mean that's fair. 164 00:08:14.160 --> 00:08:17.430 Lucas: Or we have a company event, or we're moving everything in the lab; 165 00:08:17.530 --> 00:08:19.398 so that takes up a few days or something. 166 00:08:20.066 --> 00:08:23.503 Luke: I mean, even in specifically the testing realm I think it's fair; 167 00:08:23.503 --> 00:08:26.138 cause sometimes there's a bunch of tests that we do that don't end up in videos, 168 00:08:26.138 --> 00:08:26.973 or on the website. 169 00:08:27.106 --> 00:08:30.576 I would say this probably most often happens with ShortCircuit, 170 00:08:30.676 --> 00:08:33.846 because we'll do a bunch of tests but they're not particularly interesting; 171 00:08:33.980 --> 00:08:35.281 so they don't end up in the video. 172 00:08:35.281 --> 00:08:38.818 Once we have a website level vertical then those tests can go up there anyways; 173 00:08:38.818 --> 00:08:43.256 but while it's just a video, if it doesn't make sense narratively for the video 174 00:08:43.256 --> 00:08:45.925 because there wasn't a story out of the testing that we did, 175 00:08:46.826 --> 00:08:48.294 it's, it's not going to go anywhere. 176 00:08:48.327 --> 00:08:51.931 So what I mean by website level vertical is as we kind of mentioned, 177 00:08:51.931 --> 00:08:56.369 it's not that hard to just slap a product on a bench and run a test on it. 178 00:08:56.369 --> 00:09:02.508 So we can, we can do that without a ton of planning for a lot of different potential things. 179 00:09:02.675 --> 00:09:05.912 I can't get into why, but recently we were testing thermal paste. 180 00:09:06.045 --> 00:09:09.081 We don't have a developed test suite for thermal paste, 181 00:09:09.081 --> 00:09:11.217 but we can still do some tests on thermal paste. 182 00:09:11.317 --> 00:09:15.087 The issue starts to be is how you scale that and how you build standards for that 183 00:09:15.221 --> 00:09:16.522 as we were discussing earlier. 184 00:09:16.656 --> 00:09:20.393 So when you kind of scale from ad hoc, which is just like, hey test a thing. 185 00:09:20.493 --> 00:09:22.361 Can you figure out how to test a thing? Yeah, probably, sure. 186 00:09:22.361 --> 00:09:27.199 Up to do we have some standards of some amount of tests, 187 00:09:27.233 --> 00:09:30.770 some amount of idea where we can communicate things with graphs and whatnot; 188 00:09:30.870 --> 00:09:33.072 and then that can be our ShortCircuit level. 189 00:09:33.205 --> 00:09:37.643 Then there's our LTT level where it's a little bit more confident, a little bit more scalable. 190 00:09:37.743 --> 00:09:42.148 And then there's the website level where you can really start pumping these things out 191 00:09:42.148 --> 00:09:45.651 and there's established standards; and we've, we've very likely published 192 00:09:45.651 --> 00:09:49.789 a Labs blog post talking about how we're going to do that testing. 193 00:09:49.822 --> 00:09:54.727 And you can go through a variety of different reviews and compare them, 194 00:09:54.727 --> 00:09:59.765 and actually understand a range of products on the web all at once. 195 00:09:59.765 --> 00:10:06.706 And getting a vertical from ad hoc all the way to web level can take a lot of work. 196 00:10:06.706 --> 00:10:08.774 Sammy: Sorry, do you mind repeating those last three sentences? 197 00:10:08.774 --> 00:10:10.843 Sammy: Sorry, John's coughing so it caught on the mic. 198 00:10:10.843 --> 00:10:11.210 Sammy: Sorry. 199 00:10:11.210 --> 00:10:11.944 Lucas: Get out of here John. 200 00:10:12.278 --> 00:10:13.946 Luke: Stop dying, John. 201 00:10:13.946 --> 00:10:15.081 Jon: They’re working me too much 202 00:10:16.549 --> 00:10:17.149 Lucas: The mines. 203 00:10:17.249 --> 00:10:18.084 Nick: Cut, cut, cut. 204 00:10:18.084 --> 00:10:20.219 Luke: We actually just had one walk by and it made me think of it, 205 00:10:20.219 --> 00:10:25.658 but the team has also changed significantly; so not only are we hiring new people, 206 00:10:25.658 --> 00:10:28.094 but we have hired new people already. 207 00:10:28.094 --> 00:10:30.896 A pretty significant portion of the team is under probation or would need- 208 00:10:30.930 --> 00:10:33.332 Probably seen if you've looked on the lab website, 209 00:10:33.332 --> 00:10:37.436 some of the articles that have been written recently are written by, is it Fuzzy Door? 210 00:10:37.436 --> 00:10:40.339 That's similar to how in the credits of an LTT video 211 00:10:40.339 --> 00:10:41.907 you might see code names every once in a while. 212 00:10:41.907 --> 00:10:44.477 That's a code name within the Lab for somebody who's on probation. 213 00:10:45.511 --> 00:10:48.481 I think they're all generally going pretty well, which is great; 214 00:10:48.481 --> 00:10:52.118 but like the team looks quite different already and we're continuing to hire, 215 00:10:52.118 --> 00:10:54.487 so we'll look different in the future and things are going well. 216 00:10:54.587 --> 00:11:00.059 But over the last quarter we've largely been like training and moving the lab around, 217 00:11:00.059 --> 00:11:02.328 and adapting to what the new team is going to be like. 218 00:11:02.428 --> 00:11:05.598 Sammy: Speaking of the website, you guys basically did a relaunch, 219 00:11:05.598 --> 00:11:07.566 like a whole UX update about it. 220 00:11:07.566 --> 00:11:11.003 Sammy: Do you want to talk about what prompted that and how that's been going so far? 221 00:11:11.003 --> 00:11:13.372 Because I personally used it a little bit just to try it out, 222 00:11:13.372 --> 00:11:15.474 and I really like it compared to the old one. 223 00:11:15.574 --> 00:11:17.076 Nick: Hey, he's one of our ten users. 224 00:11:19.278 --> 00:11:19.812 Luke: Rough. 225 00:11:20.212 --> 00:11:23.182 Luke: The website also recently went through a pretty major redesign, 226 00:11:23.182 --> 00:11:27.920 in my opinion and apparently Sammy's it's much more usable now, which is cool. 227 00:11:27.953 --> 00:11:31.757 The comparison tool has had a lot of effort put into it 228 00:11:31.757 --> 00:11:33.826 and the front page has had a lot of effort put into it. 229 00:11:33.826 --> 00:11:37.029 Trying to help people get to where they want to go faster. 230 00:11:37.029 --> 00:11:41.367 So we've surfaced like the categories in easier to find ways 231 00:11:41.367 --> 00:11:46.372 and in multiple locations, and we've made the front page much more dense. 232 00:11:46.505 --> 00:11:49.842 There's a lot more information in a smaller area. 233 00:11:50.843 --> 00:11:53.045 We've also been pushing pretty heavily. 234 00:11:53.045 --> 00:11:56.115 Luke: This isn't like web design, but we've been pushing pretty heavily 235 00:11:56.115 --> 00:12:00.152 to publish a lot of articles about what we're doing, what's going on. 236 00:12:00.152 --> 00:12:04.590 Some of my favorite ones are from Lucas actually talking about power supply testing 237 00:12:04.590 --> 00:12:09.562 and like, hey we might change how we communicate a thing; this is why, 238 00:12:09.562 --> 00:12:13.099 this is what we're planning on doing, and can we get your feedback on it? 239 00:12:13.099 --> 00:12:15.701 We're trying to build in public and talk about what we're doing 240 00:12:15.701 --> 00:12:18.204 and get feedback early and often; 241 00:12:18.337 --> 00:12:21.707 so that people kind of know like what we're doing and why, 242 00:12:21.807 --> 00:12:24.210 kind of learn with us and give input as we go, 243 00:12:24.210 --> 00:12:26.779 so that people can kind of see it before it comes out. 244 00:12:26.879 --> 00:12:33.686 I don't want to have someone who has been tuned in see a series of tests 245 00:12:33.686 --> 00:12:35.788 and be like, how did they come to this conclusion? 246 00:12:35.888 --> 00:12:38.758 Why are they doing these things this way? I don't understand. 247 00:12:38.858 --> 00:12:41.827 You can see that we've been doing quite a bit of work here 248 00:12:41.827 --> 00:12:46.699 on the power supply testing and the audio testing realm; 249 00:12:46.832 --> 00:12:49.335 so if you're interested in those things, check out the website. 250 00:12:49.468 --> 00:12:51.971 There's also been a ton of work done to the comparison tool, 251 00:12:51.971 --> 00:12:55.207 including Bring Your Own Price, which is live now. 252 00:12:55.241 --> 00:12:58.344 So if you go to the comparison tool and you have two things in there 253 00:12:58.344 --> 00:13:02.314 that have comparative data, you can edit the prices. 254 00:13:02.381 --> 00:13:06.452 If you find something on the used market or if your local store is not at MSRP, 255 00:13:06.452 --> 00:13:09.455 like almost all graphics cards, you can put in your own price 256 00:13:09.455 --> 00:13:10.923 to see price to performance graphs. 257 00:13:10.923 --> 00:13:14.126 Yeah, we have some disclaimers there on like how to understand that data; 258 00:13:14.160 --> 00:13:17.730 because just looking at price to performance of the raw price of two GPUs 259 00:13:17.763 --> 00:13:20.533 it's like not super representative of the impact on your system 260 00:13:20.533 --> 00:13:21.267 and blah, blah, blah, blah, 261 00:13:21.267 --> 00:13:22.868 but we talked about it on the site, so just check that out. 262 00:13:22.868 --> 00:13:28.174 We are also rolling out some pricing data from different partners, 263 00:13:28.307 --> 00:13:30.609 including the first one will probably be Newegg. 264 00:13:30.609 --> 00:13:33.312 Along with Bring Your Own Price, there's going to be another feature 265 00:13:33.312 --> 00:13:37.449 that makes it so that you can just input the price from that store; 266 00:13:37.449 --> 00:13:41.854 so there will be like a suggested price from Newegg or whatever, 267 00:13:41.854 --> 00:13:43.823 you can click that and it will just fill it in for you. 268 00:13:44.690 --> 00:13:46.325 So you don't have to go hunt for the prices. 269 00:13:46.525 --> 00:13:50.362 Also, graphing tool; the web team has been working on the graphing tool 270 00:13:50.362 --> 00:13:55.501 so that we can spend less time in Illustrator manually painting every pixel. 271 00:13:55.501 --> 00:13:56.836 Nick: What do you mean WE spent time? 272 00:13:57.503 --> 00:14:03.509 Luke: Nick, specifically, can spend less time in Illustrator. 273 00:14:03.509 --> 00:14:07.780 Luke: I like to imagine that you were like pixel painting every single graph. 274 00:14:07.813 --> 00:14:10.816 Nick: No, but maybe; felt like it. 275 00:14:11.250 --> 00:14:14.286 Luke: Yeah, it was rough, and now it is less rough. 276 00:14:14.286 --> 00:14:17.890 It doesn't mean we never do that, just usually not. 277 00:14:17.923 --> 00:14:21.393 Sammy: Actually, it was on the list here, but I actually want to quickly touch on it. 278 00:14:21.527 --> 00:14:23.095 PSU Circuit, how's that been going? 279 00:14:23.195 --> 00:14:24.029 Lucas: Thriving. 280 00:14:24.730 --> 00:14:25.764 Lucas: Best thing we've done yet. 281 00:14:26.899 --> 00:14:29.001 Nick: Least worst thing we've done yet. 282 00:14:30.069 --> 00:14:31.237 Lucas: Perfect, yeah. 283 00:14:31.270 --> 00:14:33.873 Lucas: I think it's going well; obviously it's very formulaic. 284 00:14:35.307 --> 00:14:36.275 But you know, I enjoyed it. 285 00:14:36.308 --> 00:14:38.077 There's some interesting results with some of them; 286 00:14:38.210 --> 00:14:42.014 and obviously, you know, it's a bit of a video version of the website; 287 00:14:42.047 --> 00:14:45.150 so I prefer people go to the website because there's much more information there, 288 00:14:45.284 --> 00:14:48.153 I think, and better explored because it doesn't follow as much of a script. 289 00:14:48.287 --> 00:14:50.522 But I think the video can be helpful you know, 290 00:14:50.623 --> 00:14:53.592 show people some graphs and make it approachable. 291 00:14:53.692 --> 00:14:57.129 Luke: We recently realized... by we I mean Linus and myself, 292 00:14:57.263 --> 00:15:00.266 Luke: that it's not... the editing is not automated. 293 00:15:00.266 --> 00:15:01.033 Lucas: Yeah, yeah. 294 00:15:01.834 --> 00:15:03.402 Lucas: He's been saying that a couple of times, but- 295 00:15:03.502 --> 00:15:05.037 Luke: Yeah, we're going to keep doing it; 296 00:15:05.170 --> 00:15:11.644 but if that's the case and the AI voice is actually like fairly tedious to get working, 297 00:15:11.644 --> 00:15:14.313 we might change how they're made, 298 00:15:14.413 --> 00:15:18.617 in a way that honestly is likely going to just improve them anyways. 299 00:15:18.617 --> 00:15:22.388 Luke: Because like if it's going to take us longer to AI generate a voice 300 00:15:22.388 --> 00:15:25.557 than it would be for someone to just read it, then like what are we doing? 301 00:15:25.691 --> 00:15:26.926 Nick: We're being the future. 302 00:15:27.793 --> 00:15:31.230 Nick: And how come there's no power supplies under 750 watts? 303 00:15:31.864 --> 00:15:33.699 Lucas: We started at 750 and went up; 304 00:15:33.799 --> 00:15:37.870 but we've got some on the shelf right now that are less, so those are coming. 305 00:15:38.604 --> 00:15:40.940 Nick: I'm just trying to ask what I've seen in the community. 306 00:15:41.073 --> 00:15:43.542 Luke: Product verticals? You want to take it from there? 307 00:15:44.576 --> 00:15:46.712 Lucas: Sure, I just had a couple of points here that you know, 308 00:15:46.946 --> 00:15:49.248 there's some interesting stuff; but like Luke was saying, 309 00:15:49.348 --> 00:15:52.551 there's a bit of a pyramid to it, of like steps to go down. 310 00:15:52.551 --> 00:15:55.454 So like ShortCircuit first and then LTT videos. 311 00:15:55.454 --> 00:15:57.222 It takes a lot to get to the website state, 312 00:15:57.222 --> 00:15:59.825 where you need to have all the everything fixed in place 313 00:15:59.825 --> 00:16:03.429 and know how you're going to visualize and make sure that it makes sense to people. 314 00:16:03.562 --> 00:16:06.165 But right now I'm doing a lot of stuff with the USB-C power delivery app, 315 00:16:06.165 --> 00:16:06.899 and that's pretty interesting. 316 00:16:07.032 --> 00:16:10.903 A lot of interesting things there, and we're obviously getting a bit more into audio again; 317 00:16:10.903 --> 00:16:13.205 so that I think that is interesting as well. 318 00:16:14.773 --> 00:16:15.808 Lucas: Yeah, some good results there. 319 00:16:16.308 --> 00:16:18.110 Luke: I was just checking if there's anything else you need to go over; 320 00:16:18.110 --> 00:16:18.944 Luke: but yeah, that's fine. 321 00:16:18.944 --> 00:16:19.812 Nick: What about the- 322 00:16:19.812 --> 00:16:21.914 Lucas: Some random AI made these notes. So I don't- 323 00:16:21.914 --> 00:16:22.514 Luke: Yeah, yeah. 324 00:16:22.514 --> 00:16:23.248 Sammy: Whoa, whoa, whoa, whoa! 325 00:16:23.248 --> 00:16:25.551 Luke: The AI categorized them, these are your notes. 326 00:16:25.684 --> 00:16:29.154 Nick: Why don't you talk about the quark or like you did- 327 00:16:29.154 --> 00:16:31.290 Like your charging or like USB... 328 00:16:31.290 --> 00:16:32.491 Luke: Yeah, talk about the quark. 329 00:16:32.591 --> 00:16:34.560 Nick: You said we're doing cool things about USB-C. 330 00:16:34.560 --> 00:16:38.364 Nick: What are you doing that's cool about USB-C? Educate us. 331 00:16:38.364 --> 00:16:40.199 Me and Luke are ready to learn. 332 00:16:40.299 --> 00:16:41.667 John's on the edge of his seat. 333 00:16:42.134 --> 00:16:43.702 Lucas: I don't know, USB-C is kind of interesting. 334 00:16:43.802 --> 00:16:46.638 How we deliver so much power with like you know, the connector that usually isn't. 335 00:16:46.739 --> 00:16:50.676 But before we use this Quarch PAM power analysis module, 336 00:16:50.709 --> 00:16:54.713 which measures at the wall plug point, basically it simulates the wall plug; 337 00:16:54.713 --> 00:16:58.150 and you can measure the wattage through the charger and the phone. 338 00:16:58.183 --> 00:17:01.353 But now we have this new Infineon CY4500, 339 00:17:01.353 --> 00:17:04.757 which allows you to go on the USB-C line and read all the messages, 340 00:17:04.757 --> 00:17:08.293 and you can see all the negotiations and display those and how much power 341 00:17:08.427 --> 00:17:11.864 it's trying to draw from the power brick, 342 00:17:11.864 --> 00:17:15.234 and graph that along with the voltages negotiating in the current. 343 00:17:15.768 --> 00:17:22.041 Luke: Can you talk about your graphs and how you've done like both percentage and time? 344 00:17:22.074 --> 00:17:25.444 Lucas: We're able to measure the wattage over time that's delivered to the phones; 345 00:17:25.577 --> 00:17:29.782 and by integrating that over the full period we can measure the energy, 346 00:17:29.815 --> 00:17:33.018 the total energy delivered to the phone, and split that up to simulate, 347 00:17:33.018 --> 00:17:39.958 oh, 50% of the energy was delivered before this point or after this point at 25%, 50%, 75%. 348 00:17:40.059 --> 00:17:43.996 And simulate what might be like the charge percent in 20 minutes 349 00:17:43.996 --> 00:17:46.465 without having to go and manually check the phone each time. 350 00:17:46.465 --> 00:17:50.069 For, oh, is it at 50% now? And I think those are pretty interesting. 351 00:17:50.069 --> 00:17:53.138 It's still just an approximation because your phone will, you know, 352 00:17:53.138 --> 00:17:56.041 obviously report different values; but I think they're pretty good, 353 00:17:56.041 --> 00:17:59.378 and it tells a lot better story than just the total charge time. 354 00:17:59.378 --> 00:17:59.778 Luke: Yeah. 355 00:18:00.079 --> 00:18:01.313 Lucas: Maybe we'll insert a graph? 356 00:18:01.346 --> 00:18:04.483 I have some, maybe, do we have the ability for that? 357 00:18:04.483 --> 00:18:05.684 Nick: Look at this graph. 358 00:18:05.684 --> 00:18:07.052 Lucas: Yeah, doing this side. 359 00:18:07.453 --> 00:18:08.720 Sammy: You can point that camera. 360 00:18:09.721 --> 00:18:10.989 Sammy: Yep; show the graph from that camera. 361 00:18:12.458 --> 00:18:13.826 Sammy: Show the graph from that camera. 362 00:18:13.826 --> 00:18:18.564 Luke: I love that you both are just like, ugh! I got it, mission accomplished. 363 00:18:18.564 --> 00:18:22.768 Sammy: Have you used like the LTT cable with any of these testing so far or? 364 00:18:22.801 --> 00:18:25.637 Lucas: We have not, I have four on my desk. 365 00:18:26.004 --> 00:18:26.505 Sammy: Oh, exciting. 366 00:18:26.505 --> 00:18:29.108 Lucas: Because it was, you know to talk about like if there is a voltage drop, 367 00:18:29.108 --> 00:18:30.509 that's something you might want to see. 368 00:18:30.509 --> 00:18:32.311 Like with higher currents there can be a voltage drop 369 00:18:32.311 --> 00:18:35.347 depending on the quality of the cable and the conductors, or the length; 370 00:18:35.481 --> 00:18:36.582 so I wanted to test that. 371 00:18:36.715 --> 00:18:41.653 That's next but you know, everything takes time; I'm doing this video right now. 372 00:18:44.523 --> 00:18:47.659 Luke: And next up, Markbench and AI projects with Nick. 373 00:18:47.659 --> 00:18:48.160 Nick: AI. 374 00:18:49.995 --> 00:18:50.696 Nick: The future? 375 00:18:52.231 --> 00:18:52.664 Nick: No. 376 00:18:52.764 --> 00:18:53.432 Luke: Or the now. 377 00:18:53.565 --> 00:18:54.433 Nick: Or the now. 378 00:18:55.434 --> 00:18:56.034 Nick: I can't. 379 00:18:56.802 --> 00:19:02.040 Nick: Markbench is started as just a collection of test scripts, 380 00:19:02.040 --> 00:19:06.245 or we call them harnesses, primarily in Python. 381 00:19:06.311 --> 00:19:09.882 That's the open source repo you can actually go in and look at 382 00:19:09.882 --> 00:19:11.350 and try and run some of it yourself. 383 00:19:11.483 --> 00:19:17.823 Markbench has grown now into the core desktop UI is the orchestration 384 00:19:17.923 --> 00:19:21.160 that schedules and orchestrates running all the test harnesses. 385 00:19:21.193 --> 00:19:26.165 That's how we can queue up like an overnight run of like 14 games 386 00:19:26.198 --> 00:19:28.400 and run eight of them each time. 387 00:19:28.400 --> 00:19:33.272 But Markbench is now more of that like ecosystem we were talking about earlier; 388 00:19:33.305 --> 00:19:37.876 so there's a constellation of services that that kind of serve this. 389 00:19:38.577 --> 00:19:42.247 Nick: A lot of open source stuff we use like Grafana or Redash; 390 00:19:42.814 --> 00:19:45.150 Postgres is our at rest. 391 00:19:45.484 --> 00:19:49.821 Over the last year, the technicians and the developers have expanded the test suite; 392 00:19:49.821 --> 00:19:54.193 our harness suite or our collection of harnesses is like twofold. 393 00:19:54.226 --> 00:19:58.430 There's a whole bunch of games, some I didn't even know existed. 394 00:19:58.463 --> 00:20:01.767 Ages of the Singularity? Or Singularity Ages of? 395 00:20:01.967 --> 00:20:03.068 Luke: Ashes, there we go. 396 00:20:03.068 --> 00:20:04.269 Nick: Ashes of the Singularity, what is that called? 397 00:20:04.269 --> 00:20:06.171 Luke: I was like what is ages of the- Yeah, anyways. 398 00:20:06.171 --> 00:20:07.973 Luke: I've never heard of anyone actually play the game. 399 00:20:07.973 --> 00:20:08.273 Nick: Yeah. 400 00:20:08.273 --> 00:20:11.109 Luke: No offense to whoever made it. 401 00:20:11.109 --> 00:20:12.010 Nick: No offense to anyone that may play. 402 00:20:12.010 --> 00:20:16.648 Nick: So like our test suite right now is almost about 99% automated; 403 00:20:17.849 --> 00:20:20.819 so we can set up runs for like six to seven hours. 404 00:20:20.819 --> 00:20:24.089 So we can make use of time overnight and it allows the technician to do other stuff. 405 00:20:24.089 --> 00:20:28.260 So kind of talking about the constellation of services though 406 00:20:28.260 --> 00:20:30.362 and kind of going back to the automation you're saying; 407 00:20:30.929 --> 00:20:33.131 being able to flag problems earlier. 408 00:20:33.298 --> 00:20:38.804 We're using computer vision amongst other things. 409 00:20:38.804 --> 00:20:44.610 For one explicit example, we have a service called Game Settings Error Detection Service. 410 00:20:44.610 --> 00:20:49.147 It needs a better name; maybe we can workshop that on camera. 411 00:20:49.514 --> 00:20:50.115 I'm kidding. 412 00:20:51.183 --> 00:20:52.117 Luke: E-D-S? 413 00:20:52.117 --> 00:20:54.052 Nick: It just sounds like I didn't finish high school. 414 00:20:54.052 --> 00:20:54.786 Luke: G-SEDS. 415 00:20:55.954 --> 00:20:58.490 Nick: Or is it G-E-S-D? What's the high school equivalent? 416 00:20:58.490 --> 00:20:59.157 Luke: G-E-Ds. 417 00:20:59.391 --> 00:21:00.058 Nick: G-E-Ds? 418 00:21:00.926 --> 00:21:01.360 Luke: General- 419 00:21:01.360 --> 00:21:02.194 Lucas: I didn't finish high school. 420 00:21:04.229 --> 00:21:05.130 Sammy: You didn't finish high school!? 421 00:21:05.264 --> 00:21:05.897 Lucas: No, I did. 422 00:21:05.998 --> 00:21:07.065 Nick: He's kidding. 423 00:21:08.533 --> 00:21:13.338 Nick: Anyways, the idea with like using like AI in this instance is, 424 00:21:13.372 --> 00:21:16.041 if I upload like a hundred test runs, 425 00:21:16.174 --> 00:21:19.211 the harness will take pictures of the in-game settings; 426 00:21:19.211 --> 00:21:23.148 and then we have a control of what I expect the settings page to look like. 427 00:21:23.148 --> 00:21:32.958 So the algorithm or the workflow is that we use OCR to pick out all of the text groupings; 428 00:21:32.958 --> 00:21:37.162 because all settings menus are essentially kind of the same like property value, 429 00:21:37.162 --> 00:21:39.965 property value, property value, and they're all just menus right? 430 00:21:39.998 --> 00:21:44.369 So then we compare those text groupings, we hash them and compare them against the control. 431 00:21:44.369 --> 00:21:47.773 Then we can get like a confidence rating of like, hey I think this, 432 00:21:47.773 --> 00:21:51.576 you know this game settings is correct; or if it's not we can flag it, 433 00:21:51.710 --> 00:21:54.079 so that after we come in in the morning I can see like 434 00:21:54.079 --> 00:21:56.615 oh my God, I left rebar off for something. 435 00:21:56.715 --> 00:22:00.052 Luke: And that isn't like the only check that we have for that. 436 00:22:00.052 --> 00:22:03.455 We didn't remove human error checking with this system. 437 00:22:03.455 --> 00:22:06.792 Luke: This is just supposed to be able to try to help us catch it early. 438 00:22:06.825 --> 00:22:07.592 Nick: Fail fast. 439 00:22:07.693 --> 00:22:09.528 Luke: Sometimes when people hear that, 440 00:22:09.528 --> 00:22:12.964 they're like, they're not checking things themselves anymore; 441 00:22:12.964 --> 00:22:15.100 and it's like no, we're doing the same amount of that. 442 00:22:15.133 --> 00:22:18.103 We're just trying to like you said fail fast, 443 00:22:18.103 --> 00:22:21.907 so that if something is wrong we can correct it earlier on in the process 444 00:22:21.907 --> 00:22:23.575 instead of wasting a bunch of time. 445 00:22:23.575 --> 00:22:26.611 Nick: But that's the key, it's about not wasting time. 446 00:22:26.645 --> 00:22:30.248 Nick: You only make these mistakes once where you lose like four hours of test runs; 447 00:22:30.315 --> 00:22:32.050 and you're like, well never doing that again. 448 00:22:33.385 --> 00:22:35.320 Nick: But like the game error detection is like, 449 00:22:35.320 --> 00:22:39.891 it's just one of the- It's just an explicit example of sort of 450 00:22:39.891 --> 00:22:44.029 how we're trying to not replace the tester, but augment the tester. 451 00:22:44.062 --> 00:22:49.234 Soon we're just going to implant them with a Markbench RFID chips, 452 00:22:49.234 --> 00:22:51.069 so that when you walk up to the test bench, 453 00:22:51.103 --> 00:22:55.173 you can just kind of wave your ARM and it'll just download the information to you. 454 00:22:55.173 --> 00:22:56.775 Luke: Oh, do it to the person. 455 00:22:56.775 --> 00:22:57.342 Nick: Mhmm. 456 00:22:57.342 --> 00:22:57.943 Luke: Very good. 457 00:22:58.377 --> 00:22:59.177 Luke: That's not happening. 458 00:22:59.177 --> 00:22:59.544 Nick: No. 459 00:22:59.544 --> 00:23:00.112 Luke: That's stupid, yeah. 460 00:23:00.112 --> 00:23:00.579 Nick: No. 461 00:23:01.913 --> 00:23:04.983 Nick: So Linus tried vibe coding and you saw the results of that. 462 00:23:05.016 --> 00:23:07.452 Nick: Yeah; so without sharing too much of that, 463 00:23:07.486 --> 00:23:10.655 but we don't vibe code, but we definitely use like LLMs. 464 00:23:10.655 --> 00:23:12.624 And if you saw the ShortCircuit, we- 465 00:23:12.624 --> 00:23:18.897 By we, we mean kind of I, convinced Linus to buy the crazy 512 Gigabyte Mac studio. 466 00:23:18.897 --> 00:23:22.801 So we now have two Mac studios and we, 467 00:23:22.801 --> 00:23:29.141 we run an OLAMA and open web UI stack that is internally known as Labs Benchington. 468 00:23:29.141 --> 00:23:35.280 And it serves as our local on-premise LLM; so we use it to assist in coding. 469 00:23:35.280 --> 00:23:37.983 There's others around the company that are starting to use it 470 00:23:37.983 --> 00:23:40.752 for like summary tasks and things like that. 471 00:23:40.752 --> 00:23:43.955 We're not just shipping AI code to production. 472 00:23:43.989 --> 00:23:46.391 We're not like AI generating scripts. 473 00:23:46.391 --> 00:23:46.792 Nick: We're not- 474 00:23:46.792 --> 00:23:47.259 Luke: No. 475 00:23:47.359 --> 00:23:47.859 Nick: Yeah. 476 00:23:48.093 --> 00:23:51.663 Luke: But we're not blind to its influence on things either. 477 00:23:51.696 --> 00:23:56.735 I think the, the lab is, and you and you and Jesse in particular are kind of like 478 00:23:56.735 --> 00:24:01.673 the forefront of testing and experimentation with it in the company. 479 00:24:01.673 --> 00:24:05.277 But we've as a company I think been quite wary about it. 480 00:24:05.277 --> 00:24:09.614 Like I've had quite a few conversations with people about like not using its output, 481 00:24:09.748 --> 00:24:13.585 but using it to look at things that you've done and give you feedback. 482 00:24:13.618 --> 00:24:16.555 Using it as like a rubber ducky, like hey what do you think of this thing? 483 00:24:16.588 --> 00:24:18.089 Can you give me a sentiment analysis on this? 484 00:24:18.223 --> 00:24:20.459 Does this thing look correct, et cetera. 485 00:24:20.459 --> 00:24:24.796 And then again, you can't use it as like for sure fact. 486 00:24:25.230 --> 00:24:28.200 One of my favorite quotes around AI that I use all the time 487 00:24:28.200 --> 00:24:32.037 is that old IBM one that I'm going to probably slightly misquote, 488 00:24:32.037 --> 00:24:35.507 but it's from like the eighties; and it's something along the lines of 489 00:24:35.540 --> 00:24:37.175 \"a computer cannot be held responsible, 490 00:24:37.175 --> 00:24:39.511 therefore computers should never make management decisions\". 491 00:24:39.511 --> 00:24:42.314 Luke: And I think it's still true now where like we should be 492 00:24:42.314 --> 00:24:46.685 very careful about the output, but I don't think that means that we should never use it. 493 00:24:46.818 --> 00:24:49.154 It's a tool like many other tools; 494 00:24:49.921 --> 00:24:54.493 and that might actually feed us into the next section of AI experiments. 495 00:24:54.726 --> 00:24:55.260 Luke: Oh, my. 496 00:24:56.094 --> 00:24:56.495 Nick: Oh, my. 497 00:24:56.495 --> 00:24:57.162 Luke: It's you again. 498 00:24:57.162 --> 00:24:58.063 Nick: Oh, f***. 499 00:24:59.698 --> 00:25:02.400 Nick: Yeah, so like exploring AI, 500 00:25:02.400 --> 00:25:04.035 like there's a whole bunch of stuff that we're doing 501 00:25:04.035 --> 00:25:07.739 and a lot of it's really just experimentation, just kind of build our knowledge base. 502 00:25:07.739 --> 00:25:11.209 But actually, before I talked about some of the more fun experiments, 503 00:25:11.209 --> 00:25:16.882 we are looking, right now, like part of AI benchmarking right? Or sorry, GPU benchmarking. 504 00:25:16.882 --> 00:25:21.119 We don't benchmark the AI; we're using the AI to benchmark the GPU for it. 505 00:25:21.152 --> 00:25:24.422 And inference is usually what most people are after; 506 00:25:24.422 --> 00:25:27.325 and that's like how many tokens per second, time to first token. 507 00:25:27.325 --> 00:25:31.563 How many batches of images can I get from a stable diffusion come to UI workflow? 508 00:25:31.696 --> 00:25:39.271 But where I think is coming home a lot, or coming home sooner is training things, 509 00:25:39.271 --> 00:25:42.374 fine tuning; because I mean not even five years ago, 510 00:25:42.374 --> 00:25:45.243 like training something on your own on like- Or fine tuning something. 511 00:25:45.243 --> 00:25:50.115 You needed like H200s or you know, like some serious horsepower; 512 00:25:50.115 --> 00:25:56.555 but it's within reach to start training your own stuff at home, even just like tiny smaller models. 513 00:25:56.555 --> 00:26:02.160 Cause some of these like newer 7 billion to 12 billion small models are still super capable. 514 00:26:02.160 --> 00:26:04.563 Nick: Yeah, so we're adding that to our GPU testing suite. 515 00:26:04.563 --> 00:26:06.364 Nick: Jesse's been working really hard on that. 516 00:26:07.165 --> 00:26:10.969 Nick: Need some more time to \"cook\" as the kids say, 517 00:26:11.069 --> 00:26:16.374 but we also, we're hopefully going to share and get some community feedback just cause- 518 00:26:16.374 --> 00:26:18.310 Sammy: Sorry, we're going to stop there just cause of the train. 519 00:26:18.310 --> 00:26:21.846 Nick: Okay, so we're going to stop there, train choo-choo and get some community feedback. 520 00:26:23.448 --> 00:26:25.317 Nick: Hopefully soon in the next couple of weeks, 521 00:26:25.317 --> 00:26:29.387 we'll put some stuff out on the forum, get some community feedback; cause yeah. 522 00:26:29.387 --> 00:26:30.488 Nick: And I think it'd be really interesting. 523 00:26:30.622 --> 00:26:37.862 But the fun stuff is... I think I've shared some of the cursed Linus thumbnails? 524 00:26:38.296 --> 00:26:40.165 Luke: Yeah, we can probably show them here. 525 00:26:40.165 --> 00:26:40.732 Nick: Yeah. 526 00:26:43.969 --> 00:26:49.107 Nick: Jesse has created a data set of a couple hundred, 527 00:26:49.107 --> 00:26:51.776 if I'd not think a thousand of, of our own thumbnails 528 00:26:51.943 --> 00:26:58.516 and has tried training stable diffusion, various models and flux and SDXL and stuff like that. 529 00:26:58.516 --> 00:27:01.686 Well, SDXL is stable difusi- Anyway, she's tried training a bunch of things, 530 00:27:01.686 --> 00:27:04.189 just to kind of say like what kind of output we can. 531 00:27:04.189 --> 00:27:06.558 Cause like one of the things with AI, like trade- 532 00:27:06.558 --> 00:27:13.231 Like data is really what's really valuable these days; and we have petabytes of data, 533 00:27:13.231 --> 00:27:17.435 hordes of data, more data that can fit in Lucas's shower. 534 00:27:18.236 --> 00:27:19.137 Nick: I don't know what that was. 535 00:27:21.873 --> 00:27:27.012 Luke: Just Lucas's reaction to that just... was amazing. 536 00:27:27.212 --> 00:27:27.712 Nick: I don't- 537 00:27:27.812 --> 00:27:30.749 Lucas: Nick just starts metaphors without any plan of where they go. 538 00:27:31.650 --> 00:27:34.185 Nick: Data is like the water and you know how you can on the, 539 00:27:34.185 --> 00:27:38.189 on the head though you can change like the... like the different kinds of shower. 540 00:27:38.189 --> 00:27:38.790 Luke: Why is it? 541 00:27:38.823 --> 00:27:40.058 Lucas: Maybe it should be your shower. 542 00:27:40.058 --> 00:27:40.792 Luke: Yeah. 543 00:27:40.892 --> 00:27:42.160 Lucas: No, cause his is taller. 544 00:27:42.260 --> 00:27:43.662 Luke: Does that mean there's more? 545 00:27:43.662 --> 00:27:47.365 Nick: There's more volume to fit the data that is the water. 546 00:27:48.533 --> 00:27:51.136 Nick: The plumbing is PyTorch. 547 00:27:52.604 --> 00:27:54.773 Nick: We're, we're... choo choo. 548 00:27:57.375 --> 00:27:58.510 No, anyway, the data. 549 00:27:58.510 --> 00:28:02.013 Nick: Anyway, we have- We have oodles of data, oodles, tons; 550 00:28:02.781 --> 00:28:06.151 regarding not, we're not done- Not about your shower, but just lots. 551 00:28:06.151 --> 00:28:08.053 Nick: Jesse has, so we have a model; 552 00:28:08.053 --> 00:28:11.289 maybe we can open source that model so people can generate Linus. 553 00:28:11.990 --> 00:28:14.192 Luke: People can already generate Linus with open AI, so- 554 00:28:14.192 --> 00:28:16.628 Nick: But now they can generate worse Linus. 555 00:28:16.628 --> 00:28:17.128 Luke: Yeah. 556 00:28:18.029 --> 00:28:19.330 Nick: Official Linus. 557 00:28:20.065 --> 00:28:21.866 Nick: Officially licensed Linus. 558 00:28:21.900 --> 00:28:24.869 But now we're working, we're experimenting with like video generation; 559 00:28:24.869 --> 00:28:28.473 and actually one of my favorite examples of this is like 560 00:28:28.506 --> 00:28:31.643 Recorder Digital did like the rock paper scissors anime? 561 00:28:31.710 --> 00:28:34.145 So, and that was like what, two and a half years ago now? 562 00:28:34.145 --> 00:28:34.779 Nick: Three years ago? 563 00:28:34.846 --> 00:28:40.752 Anyways, we're doing a lot with cool stuff with AI; just f***ing around. 564 00:28:40.885 --> 00:28:43.688 Sammy: We should probably note that it will not replace anything we do 565 00:28:43.722 --> 00:28:44.656 production wise currently. 566 00:28:44.923 --> 00:28:45.457 Nick: Yeah. 567 00:28:45.457 --> 00:28:46.124 Nick: Yeah, yeah, yeah. 568 00:28:47.625 --> 00:28:50.462 Just like we're not pushing vibe coded stuff to production; 569 00:28:50.495 --> 00:28:54.499 like we're not generating thumbnails, we're not generating videos. 570 00:28:54.499 --> 00:28:59.604 Luke: This is largely an experiment to see how we can test and benchmark these things. 571 00:28:59.637 --> 00:29:02.774 Luke: Not, not to replace functions within the company. 572 00:29:02.774 --> 00:29:03.908 Nick: We're trying to build a knowledge base. 573 00:29:03.908 --> 00:29:04.442 Luke: Yeah. 574 00:29:04.542 --> 00:29:08.747 Nick: Because in- There've been comments from previous AI videos that it's like, 575 00:29:08.747 --> 00:29:10.682 these guys have no idea what they're talking about. 576 00:29:10.849 --> 00:29:13.251 Luke: It's like, okay let's figure out what we're talking about. 577 00:29:13.251 --> 00:29:15.053 Nick: We're trying not to be those guys anymore. 578 00:29:15.086 --> 00:29:15.487 Luke: Yeah. 579 00:29:15.620 --> 00:29:18.323 Luke: I think the biggest ongoing battle we have is with flies. 580 00:29:18.356 --> 00:29:21.025 Luke: My, my next step, which I've been discussing for a while 581 00:29:21.059 --> 00:29:24.829 is to ARM every desk with electric fly swatter. 582 00:29:24.829 --> 00:29:28.967 The plan is like... you know like, oh man, this is going to date me a little bit. 583 00:29:28.967 --> 00:29:34.072 But in Warcraft 3, the humans have an ability where you can ARM the militia, 584 00:29:34.072 --> 00:29:36.141 where you take the like civilian minor people. 585 00:29:36.141 --> 00:29:39.444 And for a temporary period of time, they become soldiers; 586 00:29:39.477 --> 00:29:41.546 and we're going to do that with Labs members, 587 00:29:41.579 --> 00:29:44.215 by giving every single Labs member of a fly swatter. 588 00:29:44.249 --> 00:29:46.985 Nick: So you heard it here, we're hiring for Labs militia. 589 00:29:47.018 --> 00:29:51.589 Luke: No, I think, I think basically over the last year the focus has been on, 590 00:29:52.791 --> 00:29:56.027 to be completely honest, slowing down in a lot of ways; 591 00:29:56.060 --> 00:30:01.499 and with the goal of that being improving the verticals that we have 592 00:30:01.533 --> 00:30:05.970 and working on how we develop verticals in the future, 593 00:30:05.970 --> 00:30:08.106 along with reducing error output. 594 00:30:08.173 --> 00:30:12.677 And I, I think it's pretty objective to say that the errors that we've had, 595 00:30:12.710 --> 00:30:16.648 like let's say within 2025, have all been pretty minor. 596 00:30:16.681 --> 00:30:21.419 I mean, one of the ones that stood out the most was technically an NVIDIA graphic 597 00:30:21.519 --> 00:30:22.620 Luke: that had an error in it. 598 00:30:22.620 --> 00:30:23.121 Nick: Yeah. 599 00:30:23.254 --> 00:30:26.925 Luke: And we've had our slew here or there, but they've all been pretty minor. 600 00:30:26.925 --> 00:30:30.195 Luke: Like when a graph will get through without an updated subtitle. 601 00:30:30.195 --> 00:30:30.695 Nick: Our graphs. 602 00:30:30.695 --> 00:30:32.397 Luke: So it's like better, better, better, better, or whatever. 603 00:30:32.397 --> 00:30:34.032 Nick: I was gonna say, they were better, better, better. 604 00:30:34.065 --> 00:30:37.302 Luke: The, the impact of the errors that we've been having lately are pretty low; 605 00:30:37.302 --> 00:30:40.405 it's not that they don't exist, it's that they're, they're within the like, 606 00:30:40.405 --> 00:30:44.976 you know, we're humans and we're going to screw up sometime range of things 607 00:30:44.976 --> 00:30:48.646 instead of like often and heavy. 608 00:30:48.646 --> 00:30:50.815 That should be the goal moving forward; 609 00:30:50.815 --> 00:30:53.585 Luke: and I think we've been doing a great job of that for quite a while now. 610 00:30:53.618 --> 00:30:54.319 Nick: We've been doing well. 611 00:30:54.319 --> 00:30:54.819 Luke: Yeah. 612 00:30:54.819 --> 00:30:56.120 Luke: But yeah; slowing down a little bit, 613 00:30:56.120 --> 00:31:00.225 error reduction was what we were focusing on for a while. 614 00:31:00.258 --> 00:31:01.993 It's not that we don't want to focus on that anymore; 615 00:31:02.026 --> 00:31:05.296 we obviously want to keep focusing on that, but we're trying to increase our scope and grow. 616 00:31:05.296 --> 00:31:09.934 So we're currently in a hire and training phase and then hopefully 617 00:31:09.934 --> 00:31:12.604 coming closer to the end of the year and moving into the next year, 618 00:31:12.604 --> 00:31:15.139 we're going to try to get some more verticals online on the website, 619 00:31:15.139 --> 00:31:21.613 and start increasing our capabilities to match the new team size 620 00:31:21.613 --> 00:31:22.780 that we'll have when we're done hiring. 621 00:31:22.780 --> 00:31:25.416 Luke: That seemed like a decent summary; did I miss anything? 622 00:31:26.484 --> 00:31:26.918 Nick: Choo choo. 623 00:31:27.085 --> 00:31:29.854 Sammy: Oh, I guess, I guess I have, I have some thoughts. 624 00:31:29.988 --> 00:31:32.190 Where do you see Labs in the year from now? 625 00:31:32.190 --> 00:31:35.293 Like what kind of verticals do you want to be, get into? 626 00:31:35.293 --> 00:31:38.563 I'm assuming a year from now you have most of your current verticals, 627 00:31:38.563 --> 00:31:41.866 like as high as they can go? And you also want to continue expanding on that? 628 00:31:41.866 --> 00:31:46.371 Luke: As high as they can go is honestly not even really a fair statement for verticals, 629 00:31:46.371 --> 00:31:50.141 because as the industry changes we'll have to change with the industry. 630 00:31:50.241 --> 00:31:55.246 So if we're at like a ceiling, that ceiling is inherently temporary. 631 00:31:55.246 --> 00:31:57.582 And in regards to upcoming verticals, 632 00:31:57.582 --> 00:32:00.151 I want to see CPU and headphones be on the website. 633 00:32:00.151 --> 00:32:03.454 Outside of that, there's a lot of different directions we could go, 634 00:32:03.454 --> 00:32:05.990 but I'm more interested in what you guys would like. 635 00:32:05.990 --> 00:32:06.925 Lucas: More power supply. 636 00:32:07.125 --> 00:32:09.627 Luke: More power- More power supplies! 637 00:32:09.794 --> 00:32:11.529 Luke: What would- I guess what would that look like? 638 00:32:11.529 --> 00:32:13.865 Luke: What, what more do you want to do with power supply? 639 00:32:13.898 --> 00:32:17.568 Lucas: Well, we do measure a lot of data right now that we don't publish; 640 00:32:17.602 --> 00:32:19.704 but also like we, it takes a while to you know, 641 00:32:19.737 --> 00:32:22.140 develop how we should actually even publish that data. 642 00:32:22.173 --> 00:32:25.009 Or is, is it interesting? Because there's some tests like, 643 00:32:25.009 --> 00:32:28.079 okay everything passes, like every power supply passes this test; 644 00:32:28.112 --> 00:32:30.248 it's not even worth publishing kind of thing. 645 00:32:30.615 --> 00:32:32.116 But, yeah. 646 00:32:32.116 --> 00:32:36.621 Luke: Yeah, that could be a situation where that's like a blog post that we work on. 647 00:32:36.654 --> 00:32:39.924 It's like, look, there's this test; this thing actually really matters. 648 00:32:41.159 --> 00:32:44.762 Every single power supply does it fine; but like this is a way that this, 649 00:32:44.896 --> 00:32:47.098 this thing that these power supplies are all capable of 650 00:32:47.198 --> 00:32:49.901 is keeping you safe or whatever else. 651 00:32:50.001 --> 00:32:54.939 Lucas: Yeah, I think actually less focusing on verticals, getting verticals up, 652 00:32:54.939 --> 00:32:59.377 like as a way to move forward; and more just increase our capabilities to test things, 653 00:32:59.410 --> 00:33:01.279 and for like testing for videos. 654 00:33:01.312 --> 00:33:03.881 Like oh yeah, we have to test this thermal paste for this video; 655 00:33:03.881 --> 00:33:06.751 so now we have some experience with testing thermal paste. 656 00:33:06.818 --> 00:33:09.754 Maybe we can explore that later in the future, if we get another video on that. 657 00:33:09.754 --> 00:33:14.258 Or we can test- We had built some contraption for measuring like 658 00:33:14.292 --> 00:33:15.893 temperature sensors in multiple points. 659 00:33:15.960 --> 00:33:18.663 So we can build up all these things so that when a video comes in, 660 00:33:18.696 --> 00:33:21.666 we already have some tests that we can slightly adapt to it. 661 00:33:21.699 --> 00:33:22.667 Lucas: Or, you know- 662 00:33:22.667 --> 00:33:23.601 Luke: I like that a lot. 663 00:33:23.668 --> 00:33:26.504 Luke: I, I, sorry for cutting you off; I thought you were kind of done. 664 00:33:26.504 --> 00:33:30.742 But something that we've been working on and I want us to keep working on 665 00:33:30.742 --> 00:33:34.812 is, is the kind of like stability and steady state of things. 666 00:33:34.812 --> 00:33:40.151 Because if we are super sporadic and running around like crazy all the time, 667 00:33:40.151 --> 00:33:43.688 that's where you get things like errors ending up in things. 668 00:33:43.721 --> 00:33:47.658 So if we can move at a more measured pace and move with more experience 669 00:33:47.658 --> 00:33:50.194 because we've done more things in the space. 670 00:33:50.194 --> 00:33:52.630 Whether that's testing thermal paste or whatever else. 671 00:33:52.630 --> 00:33:56.000 When we get some crazy curve ball it's not that weird. 672 00:33:56.000 --> 00:33:58.269 Because if we've already dealt with these types of things 673 00:33:58.302 --> 00:34:00.905 at least once in the past and it's been decently well documented, 674 00:34:00.905 --> 00:34:02.473 we can look back at what we did before, 675 00:34:02.473 --> 00:34:04.776 and apply that to what we're doing in the future. 676 00:34:05.777 --> 00:34:08.713 And this is, this is part of the like move slower thing; 677 00:34:08.713 --> 00:34:14.018 it's not do less, it's take on more at a slower rate 678 00:34:14.018 --> 00:34:15.787 while having the same amount of output. 679 00:34:15.787 --> 00:34:16.621 Luke: Does that make sense? 680 00:34:16.621 --> 00:34:17.255 Nick: No. 681 00:34:19.057 --> 00:34:22.126 Luke: It's like instead of trying to get into... 682 00:34:22.260 --> 00:34:26.030 uh the, the cooling vertical, and the memory vertical, 683 00:34:26.030 --> 00:34:31.035 and the storage vertical, and the GPU vertical, and the CPU vertical, 684 00:34:31.035 --> 00:34:32.570 and the keyboard vertical, and the mouse vertical, 685 00:34:32.570 --> 00:34:33.404 and the monitor vertical- 686 00:34:33.404 --> 00:34:38.543 All at the exact same time; we try to do like one of them at a time, 687 00:34:38.576 --> 00:34:41.979 or spend a pretty significant amount of time refining what we already have, 688 00:34:41.979 --> 00:34:43.548 and then going into what- 689 00:34:43.548 --> 00:34:46.984 Nick: What is that? Like slow is smooth and smooth as fast? 690 00:34:46.984 --> 00:34:47.785 Nick: Or something like that. 691 00:34:47.785 --> 00:34:48.219 Luke: Yeah. 692 00:34:48.219 --> 00:34:50.054 Nick: See, building on what you're saying, 693 00:34:50.054 --> 00:34:52.723 and like by building capabilities and like knowledge base 694 00:34:53.891 --> 00:34:58.529 we've been growingly increasingly better at being like topical, like a cream. 695 00:34:59.730 --> 00:35:01.399 Luke: Yeah, the whole metaphor thing you're talking about. 696 00:35:03.568 --> 00:35:06.504 Lucas: It's, it's more like, like bottom up learning instead of top down; 697 00:35:06.504 --> 00:35:07.872 instead of okay we want to test this vertical, 698 00:35:07.872 --> 00:35:09.807 so let's develop these tests for this vertical. 699 00:35:09.807 --> 00:35:10.174 Nick: Yeah. 700 00:35:10.174 --> 00:35:12.076 Lucas: More like, like USB-C testing. 701 00:35:12.076 --> 00:35:13.711 Lucas: Oh yeah, and now we have some experience with that. 702 00:35:13.711 --> 00:35:16.914 If we have a video that comes in like with Nintendo Switch dock 703 00:35:16.914 --> 00:35:18.149 that we want to look at; oh, USB-C. 704 00:35:18.149 --> 00:35:20.084 Oh, we know how to do that, and we've tested a few things. 705 00:35:20.118 --> 00:35:22.487 We have a bit of a procedure that can apply to it; 706 00:35:22.887 --> 00:35:26.491 or you know, we've tested CPU so we can test thermal paste pretty easily 707 00:35:26.524 --> 00:35:28.092 by just adapting that a little bit. 708 00:35:28.359 --> 00:35:31.028 Nick: Even being able to explore some of like, 709 00:35:31.028 --> 00:35:33.731 when I mean by topical, ditching the cream; 710 00:35:33.764 --> 00:35:36.501 is more of like random stuff from like Reddit threads. 711 00:35:36.501 --> 00:35:40.271 Nick: Like hey like does- If I change the seconds to like track every- 712 00:35:40.271 --> 00:35:41.139 Nick: Am I going to lose power? 713 00:35:41.272 --> 00:35:44.008 Nick: So we set it up and we did it in a week and we're like, 714 00:35:44.008 --> 00:35:46.644 Oh sh**, we kind of do see that. 715 00:35:46.677 --> 00:35:48.212 Nick: Like being able to just- What? 716 00:35:48.212 --> 00:35:49.947 Lucas: We actually just got some updated results with that. 717 00:35:49.947 --> 00:35:50.581 Nick: Actually. 718 00:35:50.581 --> 00:35:53.584 Lucas: We're like exactly opposite, so it's like- 719 00:35:53.584 --> 00:35:54.051 Nick: Oh, is it? 720 00:35:54.051 --> 00:35:57.855 Lucas: Just like basically within, basically just within variance of things. 721 00:35:57.855 --> 00:35:59.423 Like the setups, we did the battery setups; 722 00:35:59.423 --> 00:36:03.861 they were specifically set up as like generic, like somebody's laptop. 723 00:36:03.861 --> 00:36:06.164 We didn't try to lock them down, all the variables. 724 00:36:06.297 --> 00:36:09.700 But yeah, basically if you have Wi-Fi on and Bluetooth on 725 00:36:09.867 --> 00:36:12.470 it could like vary your battery life way more than anything. 726 00:36:12.470 --> 00:36:14.572 But we were able to kind of test it and kind of see something, 727 00:36:14.572 --> 00:36:16.541 because we have a similar test to that. 728 00:36:16.541 --> 00:36:19.343 We can you know, test, test, you know; and say we're interested in this, 729 00:36:19.343 --> 00:36:20.945 hopefully other people are interested in this, 730 00:36:20.945 --> 00:36:23.681 Lucas: or it'll help somebody in the future if they have another question about this. 731 00:36:23.681 --> 00:36:26.150 Nick: But the fact that we were able to engage the conversation, 732 00:36:26.517 --> 00:36:29.153 like that's, that's what I find more important. 733 00:36:29.253 --> 00:36:30.288 Just being able to engage. 734 00:36:30.288 --> 00:36:34.625 Like there's, there's users in Reddit that test more fast than we do. 735 00:36:34.692 --> 00:36:40.131 And like regression test, like kernel changes to Linux within like a week. 736 00:36:40.131 --> 00:36:44.769 And they're like, hey here's like the... the shower of data. 737 00:36:44.769 --> 00:36:45.136 Luke: Yeah. 738 00:36:45.436 --> 00:36:46.370 Luke: Yeah, it's wild, it's cool. 739 00:36:46.504 --> 00:36:47.939 Luke: The showe- Oh man. 740 00:36:49.207 --> 00:36:49.540 Luke: Yeah. 741 00:36:49.540 --> 00:36:49.974 Nick: Data. 742 00:36:49.974 --> 00:36:54.212 Luke: There's a descriptor that you gave a while ago that I really liked; 743 00:36:54.212 --> 00:36:58.282 which was that it's, it's supposed to be like a- Like we grew up... 744 00:36:58.349 --> 00:37:01.552 well I don't know, you guys are... you're a little younger than me. 745 00:37:02.286 --> 00:37:06.324 I don't know how old Nick is, but we grew up on Bill Nye the science guy? 746 00:37:06.324 --> 00:37:07.758 Did you guys watch that in school? 747 00:37:07.758 --> 00:37:10.328 Did they roll out an old CRT? That still happened? 748 00:37:10.328 --> 00:37:12.530 Nice, cool; not that old, got it. 749 00:37:12.530 --> 00:37:16.801 Luke: But yeah like those, the like fun kind of \"oh, that's interesting\". 750 00:37:16.801 --> 00:37:19.337 Let's try it, let's experiment, let's play with things. 751 00:37:19.337 --> 00:37:22.106 Luke: That's how we want the lab to feel; 752 00:37:22.106 --> 00:37:26.911 and I think a big win in that department was checking out 753 00:37:26.911 --> 00:37:29.747 that seconds battery usage thing, 754 00:37:29.747 --> 00:37:32.350 because it's an interesting question that the community has. 755 00:37:32.350 --> 00:37:36.354 And I want Lab to be there for those. 756 00:37:36.354 --> 00:37:37.054 Nick: Lab or Labs? 757 00:37:37.188 --> 00:37:37.989 Luke: Labs, whatever. 758 00:37:37.989 --> 00:37:38.456 Nick: Lab. 759 00:37:38.456 --> 00:37:39.056 Luke: I want the- 760 00:37:39.090 --> 00:37:40.258 Nick: We are lab. 761 00:37:40.258 --> 00:37:41.892 Lucas: It's actually LTT Labs. 762 00:37:41.993 --> 00:37:42.560 Luke: Yeah. 763 00:37:42.560 --> 00:37:43.961 Nick: Yeah, do you know your own branding? 764 00:37:43.961 --> 00:37:46.564 Luke: Well, some people call it Lab, The Lab? 765 00:37:46.564 --> 00:37:48.933 Luke: There is only one, why do we call it Labs? 766 00:37:49.267 --> 00:37:51.302 Nick: I don't know, there's more than one lab. 767 00:37:51.302 --> 00:37:54.071 Lucas: I think there's legal reasons, but it's legally LTT Labs, I think. 768 00:37:54.071 --> 00:37:54.672 Luke: Okay. 769 00:37:54.672 --> 00:37:55.606 Lucas: I think, I don't know. 770 00:37:56.374 --> 00:37:57.975 Nick: But we had it in thirds. 771 00:37:59.010 --> 00:38:00.511 Luke: Yeah, but it's just one now. 772 00:38:00.511 --> 00:38:04.081 Luke: Have we evolved backwards? Did it go from Labs to Lab? 773 00:38:04.181 --> 00:38:07.251 Nick: Evolving backwards is called just devolving, but okay. 774 00:38:07.251 --> 00:38:08.386 Lucas: This is devolving. 775 00:38:08.419 --> 00:38:09.687 Luke: Yeah, exactly. 776 00:38:10.121 --> 00:38:11.956 Luke: To try to get back on track, 777 00:38:11.989 --> 00:38:15.593 we want to be able to be ready to test more things, 778 00:38:15.593 --> 00:38:18.162 and to have more fun with testing. 779 00:38:18.195 --> 00:38:21.265 So responding to community inquiries like, 780 00:38:21.265 --> 00:38:23.768 hey, does enabling seconds tank my battery life? 781 00:38:23.768 --> 00:38:28.639 The fact that we did that, and we did that quickly, and we did that in public, 782 00:38:28.639 --> 00:38:31.575 and we did that while communicating in the post, 783 00:38:31.676 --> 00:38:33.544 hey look, this isn't ready; 784 00:38:33.678 --> 00:38:36.947 this is just a few tests that we've done, and it's interesting so far, 785 00:38:37.048 --> 00:38:40.384 and then updating that post over time is awesome. 786 00:38:40.384 --> 00:38:42.019 I like the way that we communicated that. 787 00:38:42.019 --> 00:38:45.523 We weren't like, hey we ran three tests, this is conclusive. 788 00:38:46.357 --> 00:38:49.960 We made it clear to the audience that this is still something that we were working on. 789 00:38:50.861 --> 00:38:54.165 But we had some fun with it and actually did test the off the wall thing 790 00:38:54.165 --> 00:38:56.334 that wasn't necessarily a request from the writers. 791 00:38:56.334 --> 00:39:00.071 It wasn't a demand for a ShortCircuit video, or an LTT video, 792 00:39:00.071 --> 00:39:01.038 or the website or whatever else; 793 00:39:01.038 --> 00:39:03.174 it was just an interesting question, and we responded to it. 794 00:39:03.207 --> 00:39:05.376 And I hope that we keep doing things like that in the future. 795 00:39:05.376 --> 00:39:06.544 Luke: Thanks for watching, everyone. 796 00:39:07.278 --> 00:39:09.647 Luke: Well, I'm sure we'll do another one of these at some point; 797 00:39:09.647 --> 00:39:11.615 maybe next year, maybe before then. 798 00:39:11.615 --> 00:39:14.218 There may even be one that's a little bit more structured 799 00:39:14.218 --> 00:39:17.855 on the public YouTube channel at some point; we'll see. 800 00:39:17.855 --> 00:39:21.292 But it's been nice actually just trying to 801 00:39:21.292 --> 00:39:23.227 communicate to you what's actually been happening. 802 00:39:23.227 --> 00:39:26.430 We've been trying to be at least somewhat quiet for a while now, 803 00:39:26.430 --> 00:39:29.934 so that we can actually get some things done before we really talk about the changes, 804 00:39:29.967 --> 00:39:32.370 and I think it is time. 805 00:39:32.503 --> 00:39:37.908 So if you want to see more cool exclusives of how the company works, 806 00:39:37.908 --> 00:39:41.078 maybe check out the Linus and Terren interview. 807 00:39:42.046 --> 00:39:43.381 Lucas: Or PSU circuit, you know. 808 00:39:43.647 --> 00:39:44.115 Luke: Or that. 809 00:39:44.115 --> 00:39:45.082 Lucas: You're always welcome. 810 00:39:45.149 --> 00:39:48.152 Luke: You can see the inner workings of power supplies. 811 00:39:48.519 --> 00:39:49.854 Luke: But anyways, bye!","speaker":null,"is_sponsor":0}],"full_text":"Luke: Sorry my brain is effed right now. 2 00:00:02.669 --> 00:00:05.405 Luke: My sleep was really not great last night. 3 00:00:05.538 --> 00:00:09.476 Luke: Hey Floatplane, it's been almost a year since I joined the Labs; 4 00:00:09.576 --> 00:00:12.612 and we're gonna talk about what's happened. 5 00:00:12.712 --> 00:00:16.616 Clearly we've reorganized a bit, so we might as well start talking off with- 6 00:00:16.616 --> 00:00:17.684 Luke: Talking off? 7 00:00:19.786 --> 00:00:21.955 Luke: We're gonna introduce ourselves; my name is Luke, 8 00:00:22.022 --> 00:00:22.455 Nick: I- 9 00:00:22.522 --> 00:00:23.256 Luke: I do stuff. 10 00:00:26.559 --> 00:00:27.761 Nick: Why do you sound like you're... 11 00:00:27.761 --> 00:00:28.728 Luke: I interrupt Nick. 12 00:00:29.963 --> 00:00:34.067 Nick: Hi, I'm Nick; I'm a software development and testing supervisor. 13 00:00:35.101 --> 00:00:39.706 Lucas: I'm Lucas; I test power supplies and do some other supervision of you know, 14 00:00:39.839 --> 00:00:42.709 developing verticals and the website a little bit, publication. 15 00:00:42.709 --> 00:00:44.811 Luke: And some writing, and... 16 00:00:44.811 --> 00:00:45.779 Lucas: Some occasional writing, 17 00:00:45.779 --> 00:00:46.413 Luke: Stuff. 18 00:00:46.413 --> 00:00:47.213 Lucas: and a bunch of random stuff. 19 00:00:47.213 --> 00:00:49.482 Lucas: Calibrations, you know, look after a bunch of stuff. 20 00:00:51.451 --> 00:00:54.788 Luke: In regards to team and operations, we're hiring; 21 00:00:54.788 --> 00:00:57.590 we're trying to gain ability to test more things, 22 00:00:57.590 --> 00:00:58.925 that's one of the ideas around hiring. 23 00:00:58.925 --> 00:01:01.761 Luke: We've increased the testing team pretty significantly, 24 00:01:01.761 --> 00:01:04.464 and we're looking to increase the testing team even more. 25 00:01:04.564 --> 00:01:08.868 What do we even call them? I know Subject Matter Experts was like a title for a while. 26 00:01:09.002 --> 00:01:10.236 Lucas: Steward of the verticals, 27 00:01:10.370 --> 00:01:10.970 Luke: Yeah. 28 00:01:10.970 --> 00:01:13.306 Lucas: you know, CPUs or GPUs or whatever else. 29 00:01:13.339 --> 00:01:19.412 Luke: Basically our testing verticals, or like a type of products that we do testing on 30 00:01:19.446 --> 00:01:21.614 like GPUs or CPUs or whatever else. 31 00:01:21.648 --> 00:01:24.984 We need people to make sure that those tests are staying up to date 32 00:01:24.984 --> 00:01:26.519 and progressing with the industry. 33 00:01:26.586 --> 00:01:30.890 It doesn't necessarily mean that you would have to come in already having 34 00:01:30.890 --> 00:01:34.761 like absolute perfect domain expertise, but it means that 35 00:01:34.761 --> 00:01:38.598 you should be good at researching and keeping up with the curve, 36 00:01:38.598 --> 00:01:43.236 and keeping us on the leading edge or at least caught up with the leading edge 37 00:01:43.403 --> 00:01:45.772 for these various verticals. 38 00:01:45.772 --> 00:01:48.975 Sometimes that'll result in needing to work with someone like a Nick, 39 00:01:48.975 --> 00:01:53.580 because if you're going to try to help with say CPU or GPU, 40 00:01:53.613 --> 00:01:56.216 well that's going to be directly tied into Markbench, 41 00:01:56.216 --> 00:01:58.218 or some type of system that we already have; 42 00:01:58.251 --> 00:02:01.020 so it's going to end up being decently collaborative, but- 43 00:02:01.121 --> 00:02:04.791 Sammy: You can tell you're hiring because this is the whole new lab to set up right? 44 00:02:04.891 --> 00:02:05.725 Sammy: In the background. 45 00:02:06.159 --> 00:02:07.327 Luke: That means we're hiring? 46 00:02:07.460 --> 00:02:09.729 Sammy: Well, like you're looking to expand more you know? 47 00:02:09.729 --> 00:02:12.265 Nick: Every time I move a desk I hire someone. 48 00:02:13.199 --> 00:02:15.635 Sammy: No, well like you got like a lot more space, 49 00:02:15.635 --> 00:02:17.303 you had the little things on top now. 50 00:02:17.303 --> 00:02:17.637 Sammy: Like- 51 00:02:17.637 --> 00:02:21.608 Luke: Yeah, like technically we don't actually have any more space at all. 52 00:02:21.774 --> 00:02:25.845 It just feels like we have a ton more space because how the lab used to be set up; 53 00:02:25.945 --> 00:02:30.183 it was, it was sort of divided into like one third and then two thirds, 54 00:02:30.183 --> 00:02:32.385 I think is maybe somewhat accurate. 55 00:02:32.485 --> 00:02:37.390 Luke: And we've kind of moved things around and made it more of like a pit design. 56 00:02:37.390 --> 00:02:42.896 Luke: So we have the perimeter with the dimensions and storage shelves and stuff like that. 57 00:02:42.929 --> 00:02:48.868 And then there's the two general seating areas, one more designed for testers 58 00:02:48.868 --> 00:02:51.704 and the other one more designed for like everything that isn't testers. 59 00:02:52.539 --> 00:02:54.340 I can probably let you guys speak to that more 60 00:02:54.340 --> 00:02:57.177 because you had significantly more to do with designing this. 61 00:02:57.177 --> 00:02:59.412 I love it, I think this is way better. 62 00:02:59.546 --> 00:03:00.513 Nick: Yeah, Lucas? 63 00:03:02.282 --> 00:03:06.119 Lucas: It's a lot more flexible; so like a lot of our tables are on wheels now, 64 00:03:06.119 --> 00:03:08.121 and we have our project tables are on wheels, 65 00:03:08.254 --> 00:03:11.558 and all the GPU testing ventions and stations are on wheels; 66 00:03:11.958 --> 00:03:15.495 with all the drops like Sammy was saying for network and power. 67 00:03:15.495 --> 00:03:17.363 So we can move stuff around if needed, 68 00:03:17.497 --> 00:03:19.999 and if we expand with a bunch of people then we can do that. 69 00:03:20.099 --> 00:03:23.736 Or if we have some sudden project like Secret Shopper or all that stuff, 70 00:03:23.736 --> 00:03:24.537 takes a bunch of room. 71 00:03:24.604 --> 00:03:28.308 Luke: When you guys already had these vention desks, which are on wheels; 72 00:03:28.308 --> 00:03:31.311 but then all of your power was like hardwired into the ground, 73 00:03:31.344 --> 00:03:33.913 so you could move the desk but then you couldn't plug it in. 74 00:03:33.947 --> 00:03:38.651 Luke: So now it it seems like you're kind of realizing that vision even more. 75 00:03:38.685 --> 00:03:40.787 And I really like the project desk idea, 76 00:03:40.820 --> 00:03:45.091 because we'll run into things like Secret Shopper or whatever else; this is your example, 77 00:03:45.091 --> 00:03:47.427 where we might have a ton of different systems to test, 78 00:03:47.427 --> 00:03:51.431 Luke: or there's some huge keyboard thing showcase, whatever, words. 79 00:03:52.031 --> 00:03:53.533 Nick: Giant keyboard, massive. 80 00:03:53.533 --> 00:03:55.935 Luke: Basically, you need a bunch of space to work on things, 81 00:03:55.935 --> 00:03:59.038 and the Ventions don't really have any extra space, 82 00:03:59.138 --> 00:04:02.275 and often the stations don't really have any extra space. 83 00:04:02.275 --> 00:04:05.445 So by having these large work surfaces on wheels 84 00:04:05.445 --> 00:04:07.947 that are can just be moved around however we need; 85 00:04:07.947 --> 00:04:12.719 helps us be malleable to whatever projects might come in from the LTT team or whatever else. 86 00:04:12.885 --> 00:04:16.556 Because the routine testing that Labs does doesn't really need that so much, 87 00:04:16.689 --> 00:04:20.660 but some off the wall thing coming from somebody from the writing team. 88 00:04:20.960 --> 00:04:24.797 On here it says that we want to highlight that while it is really quick and easy 89 00:04:24.797 --> 00:04:26.666 to get simple proof of concept things together, 90 00:04:26.666 --> 00:04:29.902 it takes a long time to properly develop something and stand it up. 91 00:04:29.936 --> 00:04:32.238 Nick: Lucas has a lot to say on this actually. 92 00:04:32.372 --> 00:04:36.809 Lucas: Well it's just you know, it's easy to think like why don't they just do this? 93 00:04:36.809 --> 00:04:37.944 Why don't they just test this? 94 00:04:38.077 --> 00:04:41.214 And while it is there, it's easy to stand up a proof of concept 95 00:04:41.247 --> 00:04:44.284 or just test something and get to the point where you can show your buddies or whatever, 96 00:04:44.284 --> 00:04:45.118 maybe for a video. 97 00:04:45.218 --> 00:04:47.620 It takes a long time for the graphs to get generated, 98 00:04:47.754 --> 00:04:49.989 for the data pipeline to get worked through; 99 00:04:50.023 --> 00:04:53.559 make sure you're not doing it wrong or that you're collecting valuable information, 100 00:04:53.760 --> 00:04:54.994 so it takes a long time. 101 00:04:55.128 --> 00:04:57.096 Luke: Yeah, like do you roughly know how many tests? 102 00:04:57.230 --> 00:05:00.166 I know you've documented the number, but do you know off the top of your head 103 00:05:00.166 --> 00:05:02.135 how many tests we ran for the 5000 series launch? 104 00:05:02.235 --> 00:05:05.538 Because there's the like, oh yeah, just run a game on it dummy. 105 00:05:05.538 --> 00:05:06.272 Nick: It's a couple thousand. 106 00:05:06.272 --> 00:05:08.308 Luke: But then we run that many tests. 107 00:05:08.308 --> 00:05:08.741 Nick: Yeah. 108 00:05:08.741 --> 00:05:12.478 Luke: And then just like storing and sorting all of that data and that information, 109 00:05:12.478 --> 00:05:15.581 and actually getting something from it; knowing what games to test, 110 00:05:15.581 --> 00:05:18.918 knowing what settings to run them on, and then doing that same thing but for 111 00:05:18.918 --> 00:05:24.157 a bunch of verticals that might not be as straightforward as running a game on a computer. 112 00:05:24.490 --> 00:05:29.595 Nick: It's like on the surface, testing a graphics card or running a game is pretty- 113 00:05:29.595 --> 00:05:30.730 It's pretty simple. 114 00:05:30.763 --> 00:05:34.133 Like if we didn't, if we just wanted to get numbers into a video really quick, 115 00:05:34.133 --> 00:05:37.837 you just, we can run- They all have built in benchmarks, most of them right? 116 00:05:37.837 --> 00:05:39.339 You get your FPS out of it. 117 00:05:39.439 --> 00:05:43.443 You can record frame times with presentmon, jump that into a CSV and go. 118 00:05:43.576 --> 00:05:48.414 So it slows us down trying to do kind of create more of a framework for everything; 119 00:05:48.414 --> 00:05:53.119 so like we're doing a whole data- Like you know, an ecosystem so that when... 120 00:05:53.119 --> 00:05:59.959 so we can run a game, but we can run that game 50 times unattended. 121 00:06:00.360 --> 00:06:03.429 But it doesn't mean with automation where like press button, walk away. 122 00:06:03.429 --> 00:06:05.231 Like, I guess the FPS is okay then. 123 00:06:05.431 --> 00:06:09.235 Nick: It's like, no we're using all this automation to supplement so that we spend... 124 00:06:09.335 --> 00:06:13.506 We're sub design- Sub design- We're making it easier; 125 00:06:15.007 --> 00:06:20.780 so that technicians, instead of us spending time staring at a benchmark over and over again, 126 00:06:20.780 --> 00:06:26.519 or staring at the CSVs, we can spend more time that like exploratory testing. 127 00:06:26.519 --> 00:06:31.090 And we have systems in place that we'll talk about later about like how, 128 00:06:31.090 --> 00:06:33.826 how to ensure that we do catch anomalous errors. 129 00:06:33.960 --> 00:06:37.597 And by scaling up the testing we can actually work collaboratively 130 00:06:37.597 --> 00:06:39.899 five technicians on a single project, 131 00:06:40.333 --> 00:06:44.203 and run like thousands of tests for a single GPU project. 132 00:06:44.203 --> 00:06:49.776 Luke: Which is cool; and where alongside that we're building systems to try to detect, 133 00:06:49.776 --> 00:06:52.678 like early warning detection systems for potential problems. 134 00:06:53.212 --> 00:06:53.946 Nick: Not missiles. 135 00:06:54.313 --> 00:06:54.781 Luke: No. 136 00:06:54.781 --> 00:06:55.882 Yeah, yeah, not- Well... 137 00:06:56.382 --> 00:06:57.784 Luke: Yeah, not so much. 138 00:06:57.784 --> 00:06:58.251 Nick: Not yet. 139 00:06:58.985 --> 00:06:59.519 Luke: No, no, no. 140 00:06:59.852 --> 00:07:02.688 Luke: Yeah, hopefully flys at one point; we'll get into that later. 141 00:07:03.823 --> 00:07:07.193 Luke: Like, oh, a bunch of these tests were ran without G-Sync, 142 00:07:07.193 --> 00:07:08.861 and then one of them randomly had G-Sync on, 143 00:07:08.995 --> 00:07:13.833 flagging that because it would cause major issues is helpful 144 00:07:13.833 --> 00:07:18.671 to help reduce error output and also just help relax things. 145 00:07:18.938 --> 00:07:22.275 Because if one of the numbers comes out, out of whack 146 00:07:22.275 --> 00:07:25.645 and you just don't really have a lot of information to go on as to why, 147 00:07:25.778 --> 00:07:27.180 it can be really nerve wracking. 148 00:07:27.313 --> 00:07:31.717 If one of the numbers comes out of whack and then you can look into the records and be like, 149 00:07:31.717 --> 00:07:34.620 okay this setting or that setting or whatever was out of place. 150 00:07:34.787 --> 00:07:37.623 And then you can go back and retest it with those things corrected; 151 00:07:37.623 --> 00:07:40.560 and then it, the- The result falls back in line. 152 00:07:40.660 --> 00:07:43.129 Luke: Oh, okay; everything's fine. 153 00:07:43.296 --> 00:07:46.365 Nick: We're going for like extreme traceability. 154 00:07:46.499 --> 00:07:50.069 Nick: Versions, test results, Windows version; 155 00:07:50.069 --> 00:07:53.673 down to like, we're getting to the point where we're going to have like temperature clouds. 156 00:07:53.806 --> 00:07:55.641 Luke: Nice, I like it. 157 00:07:55.808 --> 00:08:00.346 Luke: There's a line here that says we also spend a bunch of time doing random other things. 158 00:08:00.346 --> 00:08:02.281 Nick: What are those random things? 159 00:08:02.415 --> 00:08:03.449 Lucas: Everything, I mean... 160 00:08:03.583 --> 00:08:07.553 Lucas: All the, all the random tests; you know, just takes a while. 161 00:08:07.553 --> 00:08:09.489 This is my main highlight for this is like, 162 00:08:09.722 --> 00:08:13.192 sometimes people are sick for a week you know, and then things get delayed. 163 00:08:13.192 --> 00:08:14.160 Luke: No, I mean that's fair. 164 00:08:14.160 --> 00:08:17.430 Lucas: Or we have a company event, or we're moving everything in the lab; 165 00:08:17.530 --> 00:08:19.398 so that takes up a few days or something. 166 00:08:20.066 --> 00:08:23.503 Luke: I mean, even in specifically the testing realm I think it's fair; 167 00:08:23.503 --> 00:08:26.138 cause sometimes there's a bunch of tests that we do that don't end up in videos, 168 00:08:26.138 --> 00:08:26.973 or on the website. 169 00:08:27.106 --> 00:08:30.576 I would say this probably most often happens with ShortCircuit, 170 00:08:30.676 --> 00:08:33.846 because we'll do a bunch of tests but they're not particularly interesting; 171 00:08:33.980 --> 00:08:35.281 so they don't end up in the video. 172 00:08:35.281 --> 00:08:38.818 Once we have a website level vertical then those tests can go up there anyways; 173 00:08:38.818 --> 00:08:43.256 but while it's just a video, if it doesn't make sense narratively for the video 174 00:08:43.256 --> 00:08:45.925 because there wasn't a story out of the testing that we did, 175 00:08:46.826 --> 00:08:48.294 it's, it's not going to go anywhere. 176 00:08:48.327 --> 00:08:51.931 So what I mean by website level vertical is as we kind of mentioned, 177 00:08:51.931 --> 00:08:56.369 it's not that hard to just slap a product on a bench and run a test on it. 178 00:08:56.369 --> 00:09:02.508 So we can, we can do that without a ton of planning for a lot of different potential things. 179 00:09:02.675 --> 00:09:05.912 I can't get into why, but recently we were testing thermal paste. 180 00:09:06.045 --> 00:09:09.081 We don't have a developed test suite for thermal paste, 181 00:09:09.081 --> 00:09:11.217 but we can still do some tests on thermal paste. 182 00:09:11.317 --> 00:09:15.087 The issue starts to be is how you scale that and how you build standards for that 183 00:09:15.221 --> 00:09:16.522 as we were discussing earlier. 184 00:09:16.656 --> 00:09:20.393 So when you kind of scale from ad hoc, which is just like, hey test a thing. 185 00:09:20.493 --> 00:09:22.361 Can you figure out how to test a thing? Yeah, probably, sure. 186 00:09:22.361 --> 00:09:27.199 Up to do we have some standards of some amount of tests, 187 00:09:27.233 --> 00:09:30.770 some amount of idea where we can communicate things with graphs and whatnot; 188 00:09:30.870 --> 00:09:33.072 and then that can be our ShortCircuit level. 189 00:09:33.205 --> 00:09:37.643 Then there's our LTT level where it's a little bit more confident, a little bit more scalable. 190 00:09:37.743 --> 00:09:42.148 And then there's the website level where you can really start pumping these things out 191 00:09:42.148 --> 00:09:45.651 and there's established standards; and we've, we've very likely published 192 00:09:45.651 --> 00:09:49.789 a Labs blog post talking about how we're going to do that testing. 193 00:09:49.822 --> 00:09:54.727 And you can go through a variety of different reviews and compare them, 194 00:09:54.727 --> 00:09:59.765 and actually understand a range of products on the web all at once. 195 00:09:59.765 --> 00:10:06.706 And getting a vertical from ad hoc all the way to web level can take a lot of work. 196 00:10:06.706 --> 00:10:08.774 Sammy: Sorry, do you mind repeating those last three sentences? 197 00:10:08.774 --> 00:10:10.843 Sammy: Sorry, John's coughing so it caught on the mic. 198 00:10:10.843 --> 00:10:11.210 Sammy: Sorry. 199 00:10:11.210 --> 00:10:11.944 Lucas: Get out of here John. 200 00:10:12.278 --> 00:10:13.946 Luke: Stop dying, John. 201 00:10:13.946 --> 00:10:15.081 Jon: They’re working me too much 202 00:10:16.549 --> 00:10:17.149 Lucas: The mines. 203 00:10:17.249 --> 00:10:18.084 Nick: Cut, cut, cut. 204 00:10:18.084 --> 00:10:20.219 Luke: We actually just had one walk by and it made me think of it, 205 00:10:20.219 --> 00:10:25.658 but the team has also changed significantly; so not only are we hiring new people, 206 00:10:25.658 --> 00:10:28.094 but we have hired new people already. 207 00:10:28.094 --> 00:10:30.896 A pretty significant portion of the team is under probation or would need- 208 00:10:30.930 --> 00:10:33.332 Probably seen if you've looked on the lab website, 209 00:10:33.332 --> 00:10:37.436 some of the articles that have been written recently are written by, is it Fuzzy Door? 210 00:10:37.436 --> 00:10:40.339 That's similar to how in the credits of an LTT video 211 00:10:40.339 --> 00:10:41.907 you might see code names every once in a while. 212 00:10:41.907 --> 00:10:44.477 That's a code name within the Lab for somebody who's on probation. 213 00:10:45.511 --> 00:10:48.481 I think they're all generally going pretty well, which is great; 214 00:10:48.481 --> 00:10:52.118 but like the team looks quite different already and we're continuing to hire, 215 00:10:52.118 --> 00:10:54.487 so we'll look different in the future and things are going well. 216 00:10:54.587 --> 00:11:00.059 But over the last quarter we've largely been like training and moving the lab around, 217 00:11:00.059 --> 00:11:02.328 and adapting to what the new team is going to be like. 218 00:11:02.428 --> 00:11:05.598 Sammy: Speaking of the website, you guys basically did a relaunch, 219 00:11:05.598 --> 00:11:07.566 like a whole UX update about it. 220 00:11:07.566 --> 00:11:11.003 Sammy: Do you want to talk about what prompted that and how that's been going so far? 221 00:11:11.003 --> 00:11:13.372 Because I personally used it a little bit just to try it out, 222 00:11:13.372 --> 00:11:15.474 and I really like it compared to the old one. 223 00:11:15.574 --> 00:11:17.076 Nick: Hey, he's one of our ten users. 224 00:11:19.278 --> 00:11:19.812 Luke: Rough. 225 00:11:20.212 --> 00:11:23.182 Luke: The website also recently went through a pretty major redesign, 226 00:11:23.182 --> 00:11:27.920 in my opinion and apparently Sammy's it's much more usable now, which is cool. 227 00:11:27.953 --> 00:11:31.757 The comparison tool has had a lot of effort put into it 228 00:11:31.757 --> 00:11:33.826 and the front page has had a lot of effort put into it. 229 00:11:33.826 --> 00:11:37.029 Trying to help people get to where they want to go faster. 230 00:11:37.029 --> 00:11:41.367 So we've surfaced like the categories in easier to find ways 231 00:11:41.367 --> 00:11:46.372 and in multiple locations, and we've made the front page much more dense. 232 00:11:46.505 --> 00:11:49.842 There's a lot more information in a smaller area. 233 00:11:50.843 --> 00:11:53.045 We've also been pushing pretty heavily. 234 00:11:53.045 --> 00:11:56.115 Luke: This isn't like web design, but we've been pushing pretty heavily 235 00:11:56.115 --> 00:12:00.152 to publish a lot of articles about what we're doing, what's going on. 236 00:12:00.152 --> 00:12:04.590 Some of my favorite ones are from Lucas actually talking about power supply testing 237 00:12:04.590 --> 00:12:09.562 and like, hey we might change how we communicate a thing; this is why, 238 00:12:09.562 --> 00:12:13.099 this is what we're planning on doing, and can we get your feedback on it? 239 00:12:13.099 --> 00:12:15.701 We're trying to build in public and talk about what we're doing 240 00:12:15.701 --> 00:12:18.204 and get feedback early and often; 241 00:12:18.337 --> 00:12:21.707 so that people kind of know like what we're doing and why, 242 00:12:21.807 --> 00:12:24.210 kind of learn with us and give input as we go, 243 00:12:24.210 --> 00:12:26.779 so that people can kind of see it before it comes out. 244 00:12:26.879 --> 00:12:33.686 I don't want to have someone who has been tuned in see a series of tests 245 00:12:33.686 --> 00:12:35.788 and be like, how did they come to this conclusion? 246 00:12:35.888 --> 00:12:38.758 Why are they doing these things this way? I don't understand. 247 00:12:38.858 --> 00:12:41.827 You can see that we've been doing quite a bit of work here 248 00:12:41.827 --> 00:12:46.699 on the power supply testing and the audio testing realm; 249 00:12:46.832 --> 00:12:49.335 so if you're interested in those things, check out the website. 250 00:12:49.468 --> 00:12:51.971 There's also been a ton of work done to the comparison tool, 251 00:12:51.971 --> 00:12:55.207 including Bring Your Own Price, which is live now. 252 00:12:55.241 --> 00:12:58.344 So if you go to the comparison tool and you have two things in there 253 00:12:58.344 --> 00:13:02.314 that have comparative data, you can edit the prices. 254 00:13:02.381 --> 00:13:06.452 If you find something on the used market or if your local store is not at MSRP, 255 00:13:06.452 --> 00:13:09.455 like almost all graphics cards, you can put in your own price 256 00:13:09.455 --> 00:13:10.923 to see price to performance graphs. 257 00:13:10.923 --> 00:13:14.126 Yeah, we have some disclaimers there on like how to understand that data; 258 00:13:14.160 --> 00:13:17.730 because just looking at price to performance of the raw price of two GPUs 259 00:13:17.763 --> 00:13:20.533 it's like not super representative of the impact on your system 260 00:13:20.533 --> 00:13:21.267 and blah, blah, blah, blah, 261 00:13:21.267 --> 00:13:22.868 but we talked about it on the site, so just check that out. 262 00:13:22.868 --> 00:13:28.174 We are also rolling out some pricing data from different partners, 263 00:13:28.307 --> 00:13:30.609 including the first one will probably be Newegg. 264 00:13:30.609 --> 00:13:33.312 Along with Bring Your Own Price, there's going to be another feature 265 00:13:33.312 --> 00:13:37.449 that makes it so that you can just input the price from that store; 266 00:13:37.449 --> 00:13:41.854 so there will be like a suggested price from Newegg or whatever, 267 00:13:41.854 --> 00:13:43.823 you can click that and it will just fill it in for you. 268 00:13:44.690 --> 00:13:46.325 So you don't have to go hunt for the prices. 269 00:13:46.525 --> 00:13:50.362 Also, graphing tool; the web team has been working on the graphing tool 270 00:13:50.362 --> 00:13:55.501 so that we can spend less time in Illustrator manually painting every pixel. 271 00:13:55.501 --> 00:13:56.836 Nick: What do you mean WE spent time? 272 00:13:57.503 --> 00:14:03.509 Luke: Nick, specifically, can spend less time in Illustrator. 273 00:14:03.509 --> 00:14:07.780 Luke: I like to imagine that you were like pixel painting every single graph. 274 00:14:07.813 --> 00:14:10.816 Nick: No, but maybe; felt like it. 275 00:14:11.250 --> 00:14:14.286 Luke: Yeah, it was rough, and now it is less rough. 276 00:14:14.286 --> 00:14:17.890 It doesn't mean we never do that, just usually not. 277 00:14:17.923 --> 00:14:21.393 Sammy: Actually, it was on the list here, but I actually want to quickly touch on it. 278 00:14:21.527 --> 00:14:23.095 PSU Circuit, how's that been going? 279 00:14:23.195 --> 00:14:24.029 Lucas: Thriving. 280 00:14:24.730 --> 00:14:25.764 Lucas: Best thing we've done yet. 281 00:14:26.899 --> 00:14:29.001 Nick: Least worst thing we've done yet. 282 00:14:30.069 --> 00:14:31.237 Lucas: Perfect, yeah. 283 00:14:31.270 --> 00:14:33.873 Lucas: I think it's going well; obviously it's very formulaic. 284 00:14:35.307 --> 00:14:36.275 But you know, I enjoyed it. 285 00:14:36.308 --> 00:14:38.077 There's some interesting results with some of them; 286 00:14:38.210 --> 00:14:42.014 and obviously, you know, it's a bit of a video version of the website; 287 00:14:42.047 --> 00:14:45.150 so I prefer people go to the website because there's much more information there, 288 00:14:45.284 --> 00:14:48.153 I think, and better explored because it doesn't follow as much of a script. 289 00:14:48.287 --> 00:14:50.522 But I think the video can be helpful you know, 290 00:14:50.623 --> 00:14:53.592 show people some graphs and make it approachable. 291 00:14:53.692 --> 00:14:57.129 Luke: We recently realized... by we I mean Linus and myself, 292 00:14:57.263 --> 00:15:00.266 Luke: that it's not... the editing is not automated. 293 00:15:00.266 --> 00:15:01.033 Lucas: Yeah, yeah. 294 00:15:01.834 --> 00:15:03.402 Lucas: He's been saying that a couple of times, but- 295 00:15:03.502 --> 00:15:05.037 Luke: Yeah, we're going to keep doing it; 296 00:15:05.170 --> 00:15:11.644 but if that's the case and the AI voice is actually like fairly tedious to get working, 297 00:15:11.644 --> 00:15:14.313 we might change how they're made, 298 00:15:14.413 --> 00:15:18.617 in a way that honestly is likely going to just improve them anyways. 299 00:15:18.617 --> 00:15:22.388 Luke: Because like if it's going to take us longer to AI generate a voice 300 00:15:22.388 --> 00:15:25.557 than it would be for someone to just read it, then like what are we doing? 301 00:15:25.691 --> 00:15:26.926 Nick: We're being the future. 302 00:15:27.793 --> 00:15:31.230 Nick: And how come there's no power supplies under 750 watts? 303 00:15:31.864 --> 00:15:33.699 Lucas: We started at 750 and went up; 304 00:15:33.799 --> 00:15:37.870 but we've got some on the shelf right now that are less, so those are coming. 305 00:15:38.604 --> 00:15:40.940 Nick: I'm just trying to ask what I've seen in the community. 306 00:15:41.073 --> 00:15:43.542 Luke: Product verticals? You want to take it from there? 307 00:15:44.576 --> 00:15:46.712 Lucas: Sure, I just had a couple of points here that you know, 308 00:15:46.946 --> 00:15:49.248 there's some interesting stuff; but like Luke was saying, 309 00:15:49.348 --> 00:15:52.551 there's a bit of a pyramid to it, of like steps to go down. 310 00:15:52.551 --> 00:15:55.454 So like ShortCircuit first and then LTT videos. 311 00:15:55.454 --> 00:15:57.222 It takes a lot to get to the website state, 312 00:15:57.222 --> 00:15:59.825 where you need to have all the everything fixed in place 313 00:15:59.825 --> 00:16:03.429 and know how you're going to visualize and make sure that it makes sense to people. 314 00:16:03.562 --> 00:16:06.165 But right now I'm doing a lot of stuff with the USB-C power delivery app, 315 00:16:06.165 --> 00:16:06.899 and that's pretty interesting. 316 00:16:07.032 --> 00:16:10.903 A lot of interesting things there, and we're obviously getting a bit more into audio again; 317 00:16:10.903 --> 00:16:13.205 so that I think that is interesting as well. 318 00:16:14.773 --> 00:16:15.808 Lucas: Yeah, some good results there. 319 00:16:16.308 --> 00:16:18.110 Luke: I was just checking if there's anything else you need to go over; 320 00:16:18.110 --> 00:16:18.944 Luke: but yeah, that's fine. 321 00:16:18.944 --> 00:16:19.812 Nick: What about the- 322 00:16:19.812 --> 00:16:21.914 Lucas: Some random AI made these notes. So I don't- 323 00:16:21.914 --> 00:16:22.514 Luke: Yeah, yeah. 324 00:16:22.514 --> 00:16:23.248 Sammy: Whoa, whoa, whoa, whoa! 325 00:16:23.248 --> 00:16:25.551 Luke: The AI categorized them, these are your notes. 326 00:16:25.684 --> 00:16:29.154 Nick: Why don't you talk about the quark or like you did- 327 00:16:29.154 --> 00:16:31.290 Like your charging or like USB... 328 00:16:31.290 --> 00:16:32.491 Luke: Yeah, talk about the quark. 329 00:16:32.591 --> 00:16:34.560 Nick: You said we're doing cool things about USB-C. 330 00:16:34.560 --> 00:16:38.364 Nick: What are you doing that's cool about USB-C? Educate us. 331 00:16:38.364 --> 00:16:40.199 Me and Luke are ready to learn. 332 00:16:40.299 --> 00:16:41.667 John's on the edge of his seat. 333 00:16:42.134 --> 00:16:43.702 Lucas: I don't know, USB-C is kind of interesting. 334 00:16:43.802 --> 00:16:46.638 How we deliver so much power with like you know, the connector that usually isn't. 335 00:16:46.739 --> 00:16:50.676 But before we use this Quarch PAM power analysis module, 336 00:16:50.709 --> 00:16:54.713 which measures at the wall plug point, basically it simulates the wall plug; 337 00:16:54.713 --> 00:16:58.150 and you can measure the wattage through the charger and the phone. 338 00:16:58.183 --> 00:17:01.353 But now we have this new Infineon CY4500, 339 00:17:01.353 --> 00:17:04.757 which allows you to go on the USB-C line and read all the messages, 340 00:17:04.757 --> 00:17:08.293 and you can see all the negotiations and display those and how much power 341 00:17:08.427 --> 00:17:11.864 it's trying to draw from the power brick, 342 00:17:11.864 --> 00:17:15.234 and graph that along with the voltages negotiating in the current. 343 00:17:15.768 --> 00:17:22.041 Luke: Can you talk about your graphs and how you've done like both percentage and time? 344 00:17:22.074 --> 00:17:25.444 Lucas: We're able to measure the wattage over time that's delivered to the phones; 345 00:17:25.577 --> 00:17:29.782 and by integrating that over the full period we can measure the energy, 346 00:17:29.815 --> 00:17:33.018 the total energy delivered to the phone, and split that up to simulate, 347 00:17:33.018 --> 00:17:39.958 oh, 50% of the energy was delivered before this point or after this point at 25%, 50%, 75%. 348 00:17:40.059 --> 00:17:43.996 And simulate what might be like the charge percent in 20 minutes 349 00:17:43.996 --> 00:17:46.465 without having to go and manually check the phone each time. 350 00:17:46.465 --> 00:17:50.069 For, oh, is it at 50% now? And I think those are pretty interesting. 351 00:17:50.069 --> 00:17:53.138 It's still just an approximation because your phone will, you know, 352 00:17:53.138 --> 00:17:56.041 obviously report different values; but I think they're pretty good, 353 00:17:56.041 --> 00:17:59.378 and it tells a lot better story than just the total charge time. 354 00:17:59.378 --> 00:17:59.778 Luke: Yeah. 355 00:18:00.079 --> 00:18:01.313 Lucas: Maybe we'll insert a graph? 356 00:18:01.346 --> 00:18:04.483 I have some, maybe, do we have the ability for that? 357 00:18:04.483 --> 00:18:05.684 Nick: Look at this graph. 358 00:18:05.684 --> 00:18:07.052 Lucas: Yeah, doing this side. 359 00:18:07.453 --> 00:18:08.720 Sammy: You can point that camera. 360 00:18:09.721 --> 00:18:10.989 Sammy: Yep; show the graph from that camera. 361 00:18:12.458 --> 00:18:13.826 Sammy: Show the graph from that camera. 362 00:18:13.826 --> 00:18:18.564 Luke: I love that you both are just like, ugh! I got it, mission accomplished. 363 00:18:18.564 --> 00:18:22.768 Sammy: Have you used like the LTT cable with any of these testing so far or? 364 00:18:22.801 --> 00:18:25.637 Lucas: We have not, I have four on my desk. 365 00:18:26.004 --> 00:18:26.505 Sammy: Oh, exciting. 366 00:18:26.505 --> 00:18:29.108 Lucas: Because it was, you know to talk about like if there is a voltage drop, 367 00:18:29.108 --> 00:18:30.509 that's something you might want to see. 368 00:18:30.509 --> 00:18:32.311 Like with higher currents there can be a voltage drop 369 00:18:32.311 --> 00:18:35.347 depending on the quality of the cable and the conductors, or the length; 370 00:18:35.481 --> 00:18:36.582 so I wanted to test that. 371 00:18:36.715 --> 00:18:41.653 That's next but you know, everything takes time; I'm doing this video right now. 372 00:18:44.523 --> 00:18:47.659 Luke: And next up, Markbench and AI projects with Nick. 373 00:18:47.659 --> 00:18:48.160 Nick: AI. 374 00:18:49.995 --> 00:18:50.696 Nick: The future? 375 00:18:52.231 --> 00:18:52.664 Nick: No. 376 00:18:52.764 --> 00:18:53.432 Luke: Or the now. 377 00:18:53.565 --> 00:18:54.433 Nick: Or the now. 378 00:18:55.434 --> 00:18:56.034 Nick: I can't. 379 00:18:56.802 --> 00:19:02.040 Nick: Markbench is started as just a collection of test scripts, 380 00:19:02.040 --> 00:19:06.245 or we call them harnesses, primarily in Python. 381 00:19:06.311 --> 00:19:09.882 That's the open source repo you can actually go in and look at 382 00:19:09.882 --> 00:19:11.350 and try and run some of it yourself. 383 00:19:11.483 --> 00:19:17.823 Markbench has grown now into the core desktop UI is the orchestration 384 00:19:17.923 --> 00:19:21.160 that schedules and orchestrates running all the test harnesses. 385 00:19:21.193 --> 00:19:26.165 That's how we can queue up like an overnight run of like 14 games 386 00:19:26.198 --> 00:19:28.400 and run eight of them each time. 387 00:19:28.400 --> 00:19:33.272 But Markbench is now more of that like ecosystem we were talking about earlier; 388 00:19:33.305 --> 00:19:37.876 so there's a constellation of services that that kind of serve this. 389 00:19:38.577 --> 00:19:42.247 Nick: A lot of open source stuff we use like Grafana or Redash; 390 00:19:42.814 --> 00:19:45.150 Postgres is our at rest. 391 00:19:45.484 --> 00:19:49.821 Over the last year, the technicians and the developers have expanded the test suite; 392 00:19:49.821 --> 00:19:54.193 our harness suite or our collection of harnesses is like twofold. 393 00:19:54.226 --> 00:19:58.430 There's a whole bunch of games, some I didn't even know existed. 394 00:19:58.463 --> 00:20:01.767 Ages of the Singularity? Or Singularity Ages of? 395 00:20:01.967 --> 00:20:03.068 Luke: Ashes, there we go. 396 00:20:03.068 --> 00:20:04.269 Nick: Ashes of the Singularity, what is that called? 397 00:20:04.269 --> 00:20:06.171 Luke: I was like what is ages of the- Yeah, anyways. 398 00:20:06.171 --> 00:20:07.973 Luke: I've never heard of anyone actually play the game. 399 00:20:07.973 --> 00:20:08.273 Nick: Yeah. 400 00:20:08.273 --> 00:20:11.109 Luke: No offense to whoever made it. 401 00:20:11.109 --> 00:20:12.010 Nick: No offense to anyone that may play. 402 00:20:12.010 --> 00:20:16.648 Nick: So like our test suite right now is almost about 99% automated; 403 00:20:17.849 --> 00:20:20.819 so we can set up runs for like six to seven hours. 404 00:20:20.819 --> 00:20:24.089 So we can make use of time overnight and it allows the technician to do other stuff. 405 00:20:24.089 --> 00:20:28.260 So kind of talking about the constellation of services though 406 00:20:28.260 --> 00:20:30.362 and kind of going back to the automation you're saying; 407 00:20:30.929 --> 00:20:33.131 being able to flag problems earlier. 408 00:20:33.298 --> 00:20:38.804 We're using computer vision amongst other things. 409 00:20:38.804 --> 00:20:44.610 For one explicit example, we have a service called Game Settings Error Detection Service. 410 00:20:44.610 --> 00:20:49.147 It needs a better name; maybe we can workshop that on camera. 411 00:20:49.514 --> 00:20:50.115 I'm kidding. 412 00:20:51.183 --> 00:20:52.117 Luke: E-D-S? 413 00:20:52.117 --> 00:20:54.052 Nick: It just sounds like I didn't finish high school. 414 00:20:54.052 --> 00:20:54.786 Luke: G-SEDS. 415 00:20:55.954 --> 00:20:58.490 Nick: Or is it G-E-S-D? What's the high school equivalent? 416 00:20:58.490 --> 00:20:59.157 Luke: G-E-Ds. 417 00:20:59.391 --> 00:21:00.058 Nick: G-E-Ds? 418 00:21:00.926 --> 00:21:01.360 Luke: General- 419 00:21:01.360 --> 00:21:02.194 Lucas: I didn't finish high school. 420 00:21:04.229 --> 00:21:05.130 Sammy: You didn't finish high school!? 421 00:21:05.264 --> 00:21:05.897 Lucas: No, I did. 422 00:21:05.998 --> 00:21:07.065 Nick: He's kidding. 423 00:21:08.533 --> 00:21:13.338 Nick: Anyways, the idea with like using like AI in this instance is, 424 00:21:13.372 --> 00:21:16.041 if I upload like a hundred test runs, 425 00:21:16.174 --> 00:21:19.211 the harness will take pictures of the in-game settings; 426 00:21:19.211 --> 00:21:23.148 and then we have a control of what I expect the settings page to look like. 427 00:21:23.148 --> 00:21:32.958 So the algorithm or the workflow is that we use OCR to pick out all of the text groupings; 428 00:21:32.958 --> 00:21:37.162 because all settings menus are essentially kind of the same like property value, 429 00:21:37.162 --> 00:21:39.965 property value, property value, and they're all just menus right? 430 00:21:39.998 --> 00:21:44.369 So then we compare those text groupings, we hash them and compare them against the control. 431 00:21:44.369 --> 00:21:47.773 Then we can get like a confidence rating of like, hey I think this, 432 00:21:47.773 --> 00:21:51.576 you know this game settings is correct; or if it's not we can flag it, 433 00:21:51.710 --> 00:21:54.079 so that after we come in in the morning I can see like 434 00:21:54.079 --> 00:21:56.615 oh my God, I left rebar off for something. 435 00:21:56.715 --> 00:22:00.052 Luke: And that isn't like the only check that we have for that. 436 00:22:00.052 --> 00:22:03.455 We didn't remove human error checking with this system. 437 00:22:03.455 --> 00:22:06.792 Luke: This is just supposed to be able to try to help us catch it early. 438 00:22:06.825 --> 00:22:07.592 Nick: Fail fast. 439 00:22:07.693 --> 00:22:09.528 Luke: Sometimes when people hear that, 440 00:22:09.528 --> 00:22:12.964 they're like, they're not checking things themselves anymore; 441 00:22:12.964 --> 00:22:15.100 and it's like no, we're doing the same amount of that. 442 00:22:15.133 --> 00:22:18.103 We're just trying to like you said fail fast, 443 00:22:18.103 --> 00:22:21.907 so that if something is wrong we can correct it earlier on in the process 444 00:22:21.907 --> 00:22:23.575 instead of wasting a bunch of time. 445 00:22:23.575 --> 00:22:26.611 Nick: But that's the key, it's about not wasting time. 446 00:22:26.645 --> 00:22:30.248 Nick: You only make these mistakes once where you lose like four hours of test runs; 447 00:22:30.315 --> 00:22:32.050 and you're like, well never doing that again. 448 00:22:33.385 --> 00:22:35.320 Nick: But like the game error detection is like, 449 00:22:35.320 --> 00:22:39.891 it's just one of the- It's just an explicit example of sort of 450 00:22:39.891 --> 00:22:44.029 how we're trying to not replace the tester, but augment the tester. 451 00:22:44.062 --> 00:22:49.234 Soon we're just going to implant them with a Markbench RFID chips, 452 00:22:49.234 --> 00:22:51.069 so that when you walk up to the test bench, 453 00:22:51.103 --> 00:22:55.173 you can just kind of wave your ARM and it'll just download the information to you. 454 00:22:55.173 --> 00:22:56.775 Luke: Oh, do it to the person. 455 00:22:56.775 --> 00:22:57.342 Nick: Mhmm. 456 00:22:57.342 --> 00:22:57.943 Luke: Very good. 457 00:22:58.377 --> 00:22:59.177 Luke: That's not happening. 458 00:22:59.177 --> 00:22:59.544 Nick: No. 459 00:22:59.544 --> 00:23:00.112 Luke: That's stupid, yeah. 460 00:23:00.112 --> 00:23:00.579 Nick: No. 461 00:23:01.913 --> 00:23:04.983 Nick: So Linus tried vibe coding and you saw the results of that. 462 00:23:05.016 --> 00:23:07.452 Nick: Yeah; so without sharing too much of that, 463 00:23:07.486 --> 00:23:10.655 but we don't vibe code, but we definitely use like LLMs. 464 00:23:10.655 --> 00:23:12.624 And if you saw the ShortCircuit, we- 465 00:23:12.624 --> 00:23:18.897 By we, we mean kind of I, convinced Linus to buy the crazy 512 Gigabyte Mac studio. 466 00:23:18.897 --> 00:23:22.801 So we now have two Mac studios and we, 467 00:23:22.801 --> 00:23:29.141 we run an OLAMA and open web UI stack that is internally known as Labs Benchington. 468 00:23:29.141 --> 00:23:35.280 And it serves as our local on-premise LLM; so we use it to assist in coding. 469 00:23:35.280 --> 00:23:37.983 There's others around the company that are starting to use it 470 00:23:37.983 --> 00:23:40.752 for like summary tasks and things like that. 471 00:23:40.752 --> 00:23:43.955 We're not just shipping AI code to production. 472 00:23:43.989 --> 00:23:46.391 We're not like AI generating scripts. 473 00:23:46.391 --> 00:23:46.792 Nick: We're not- 474 00:23:46.792 --> 00:23:47.259 Luke: No. 475 00:23:47.359 --> 00:23:47.859 Nick: Yeah. 476 00:23:48.093 --> 00:23:51.663 Luke: But we're not blind to its influence on things either. 477 00:23:51.696 --> 00:23:56.735 I think the, the lab is, and you and you and Jesse in particular are kind of like 478 00:23:56.735 --> 00:24:01.673 the forefront of testing and experimentation with it in the company. 479 00:24:01.673 --> 00:24:05.277 But we've as a company I think been quite wary about it. 480 00:24:05.277 --> 00:24:09.614 Like I've had quite a few conversations with people about like not using its output, 481 00:24:09.748 --> 00:24:13.585 but using it to look at things that you've done and give you feedback. 482 00:24:13.618 --> 00:24:16.555 Using it as like a rubber ducky, like hey what do you think of this thing? 483 00:24:16.588 --> 00:24:18.089 Can you give me a sentiment analysis on this? 484 00:24:18.223 --> 00:24:20.459 Does this thing look correct, et cetera. 485 00:24:20.459 --> 00:24:24.796 And then again, you can't use it as like for sure fact. 486 00:24:25.230 --> 00:24:28.200 One of my favorite quotes around AI that I use all the time 487 00:24:28.200 --> 00:24:32.037 is that old IBM one that I'm going to probably slightly misquote, 488 00:24:32.037 --> 00:24:35.507 but it's from like the eighties; and it's something along the lines of 489 00:24:35.540 --> 00:24:37.175 \"a computer cannot be held responsible, 490 00:24:37.175 --> 00:24:39.511 therefore computers should never make management decisions\". 491 00:24:39.511 --> 00:24:42.314 Luke: And I think it's still true now where like we should be 492 00:24:42.314 --> 00:24:46.685 very careful about the output, but I don't think that means that we should never use it. 493 00:24:46.818 --> 00:24:49.154 It's a tool like many other tools; 494 00:24:49.921 --> 00:24:54.493 and that might actually feed us into the next section of AI experiments. 495 00:24:54.726 --> 00:24:55.260 Luke: Oh, my. 496 00:24:56.094 --> 00:24:56.495 Nick: Oh, my. 497 00:24:56.495 --> 00:24:57.162 Luke: It's you again. 498 00:24:57.162 --> 00:24:58.063 Nick: Oh, f***. 499 00:24:59.698 --> 00:25:02.400 Nick: Yeah, so like exploring AI, 500 00:25:02.400 --> 00:25:04.035 like there's a whole bunch of stuff that we're doing 501 00:25:04.035 --> 00:25:07.739 and a lot of it's really just experimentation, just kind of build our knowledge base. 502 00:25:07.739 --> 00:25:11.209 But actually, before I talked about some of the more fun experiments, 503 00:25:11.209 --> 00:25:16.882 we are looking, right now, like part of AI benchmarking right? Or sorry, GPU benchmarking. 504 00:25:16.882 --> 00:25:21.119 We don't benchmark the AI; we're using the AI to benchmark the GPU for it. 505 00:25:21.152 --> 00:25:24.422 And inference is usually what most people are after; 506 00:25:24.422 --> 00:25:27.325 and that's like how many tokens per second, time to first token. 507 00:25:27.325 --> 00:25:31.563 How many batches of images can I get from a stable diffusion come to UI workflow? 508 00:25:31.696 --> 00:25:39.271 But where I think is coming home a lot, or coming home sooner is training things, 509 00:25:39.271 --> 00:25:42.374 fine tuning; because I mean not even five years ago, 510 00:25:42.374 --> 00:25:45.243 like training something on your own on like- Or fine tuning something. 511 00:25:45.243 --> 00:25:50.115 You needed like H200s or you know, like some serious horsepower; 512 00:25:50.115 --> 00:25:56.555 but it's within reach to start training your own stuff at home, even just like tiny smaller models. 513 00:25:56.555 --> 00:26:02.160 Cause some of these like newer 7 billion to 12 billion small models are still super capable. 514 00:26:02.160 --> 00:26:04.563 Nick: Yeah, so we're adding that to our GPU testing suite. 515 00:26:04.563 --> 00:26:06.364 Nick: Jesse's been working really hard on that. 516 00:26:07.165 --> 00:26:10.969 Nick: Need some more time to \"cook\" as the kids say, 517 00:26:11.069 --> 00:26:16.374 but we also, we're hopefully going to share and get some community feedback just cause- 518 00:26:16.374 --> 00:26:18.310 Sammy: Sorry, we're going to stop there just cause of the train. 519 00:26:18.310 --> 00:26:21.846 Nick: Okay, so we're going to stop there, train choo-choo and get some community feedback. 520 00:26:23.448 --> 00:26:25.317 Nick: Hopefully soon in the next couple of weeks, 521 00:26:25.317 --> 00:26:29.387 we'll put some stuff out on the forum, get some community feedback; cause yeah. 522 00:26:29.387 --> 00:26:30.488 Nick: And I think it'd be really interesting. 523 00:26:30.622 --> 00:26:37.862 But the fun stuff is... I think I've shared some of the cursed Linus thumbnails? 524 00:26:38.296 --> 00:26:40.165 Luke: Yeah, we can probably show them here. 525 00:26:40.165 --> 00:26:40.732 Nick: Yeah. 526 00:26:43.969 --> 00:26:49.107 Nick: Jesse has created a data set of a couple hundred, 527 00:26:49.107 --> 00:26:51.776 if I'd not think a thousand of, of our own thumbnails 528 00:26:51.943 --> 00:26:58.516 and has tried training stable diffusion, various models and flux and SDXL and stuff like that. 529 00:26:58.516 --> 00:27:01.686 Well, SDXL is stable difusi- Anyway, she's tried training a bunch of things, 530 00:27:01.686 --> 00:27:04.189 just to kind of say like what kind of output we can. 531 00:27:04.189 --> 00:27:06.558 Cause like one of the things with AI, like trade- 532 00:27:06.558 --> 00:27:13.231 Like data is really what's really valuable these days; and we have petabytes of data, 533 00:27:13.231 --> 00:27:17.435 hordes of data, more data that can fit in Lucas's shower. 534 00:27:18.236 --> 00:27:19.137 Nick: I don't know what that was. 535 00:27:21.873 --> 00:27:27.012 Luke: Just Lucas's reaction to that just... was amazing. 536 00:27:27.212 --> 00:27:27.712 Nick: I don't- 537 00:27:27.812 --> 00:27:30.749 Lucas: Nick just starts metaphors without any plan of where they go. 538 00:27:31.650 --> 00:27:34.185 Nick: Data is like the water and you know how you can on the, 539 00:27:34.185 --> 00:27:38.189 on the head though you can change like the... like the different kinds of shower. 540 00:27:38.189 --> 00:27:38.790 Luke: Why is it? 541 00:27:38.823 --> 00:27:40.058 Lucas: Maybe it should be your shower. 542 00:27:40.058 --> 00:27:40.792 Luke: Yeah. 543 00:27:40.892 --> 00:27:42.160 Lucas: No, cause his is taller. 544 00:27:42.260 --> 00:27:43.662 Luke: Does that mean there's more? 545 00:27:43.662 --> 00:27:47.365 Nick: There's more volume to fit the data that is the water. 546 00:27:48.533 --> 00:27:51.136 Nick: The plumbing is PyTorch. 547 00:27:52.604 --> 00:27:54.773 Nick: We're, we're... choo choo. 548 00:27:57.375 --> 00:27:58.510 No, anyway, the data. 549 00:27:58.510 --> 00:28:02.013 Nick: Anyway, we have- We have oodles of data, oodles, tons; 550 00:28:02.781 --> 00:28:06.151 regarding not, we're not done- Not about your shower, but just lots. 551 00:28:06.151 --> 00:28:08.053 Nick: Jesse has, so we have a model; 552 00:28:08.053 --> 00:28:11.289 maybe we can open source that model so people can generate Linus. 553 00:28:11.990 --> 00:28:14.192 Luke: People can already generate Linus with open AI, so- 554 00:28:14.192 --> 00:28:16.628 Nick: But now they can generate worse Linus. 555 00:28:16.628 --> 00:28:17.128 Luke: Yeah. 556 00:28:18.029 --> 00:28:19.330 Nick: Official Linus. 557 00:28:20.065 --> 00:28:21.866 Nick: Officially licensed Linus. 558 00:28:21.900 --> 00:28:24.869 But now we're working, we're experimenting with like video generation; 559 00:28:24.869 --> 00:28:28.473 and actually one of my favorite examples of this is like 560 00:28:28.506 --> 00:28:31.643 Recorder Digital did like the rock paper scissors anime? 561 00:28:31.710 --> 00:28:34.145 So, and that was like what, two and a half years ago now? 562 00:28:34.145 --> 00:28:34.779 Nick: Three years ago? 563 00:28:34.846 --> 00:28:40.752 Anyways, we're doing a lot with cool stuff with AI; just f***ing around. 564 00:28:40.885 --> 00:28:43.688 Sammy: We should probably note that it will not replace anything we do 565 00:28:43.722 --> 00:28:44.656 production wise currently. 566 00:28:44.923 --> 00:28:45.457 Nick: Yeah. 567 00:28:45.457 --> 00:28:46.124 Nick: Yeah, yeah, yeah. 568 00:28:47.625 --> 00:28:50.462 Just like we're not pushing vibe coded stuff to production; 569 00:28:50.495 --> 00:28:54.499 like we're not generating thumbnails, we're not generating videos. 570 00:28:54.499 --> 00:28:59.604 Luke: This is largely an experiment to see how we can test and benchmark these things. 571 00:28:59.637 --> 00:29:02.774 Luke: Not, not to replace functions within the company. 572 00:29:02.774 --> 00:29:03.908 Nick: We're trying to build a knowledge base. 573 00:29:03.908 --> 00:29:04.442 Luke: Yeah. 574 00:29:04.542 --> 00:29:08.747 Nick: Because in- There've been comments from previous AI videos that it's like, 575 00:29:08.747 --> 00:29:10.682 these guys have no idea what they're talking about. 576 00:29:10.849 --> 00:29:13.251 Luke: It's like, okay let's figure out what we're talking about. 577 00:29:13.251 --> 00:29:15.053 Nick: We're trying not to be those guys anymore. 578 00:29:15.086 --> 00:29:15.487 Luke: Yeah. 579 00:29:15.620 --> 00:29:18.323 Luke: I think the biggest ongoing battle we have is with flies. 580 00:29:18.356 --> 00:29:21.025 Luke: My, my next step, which I've been discussing for a while 581 00:29:21.059 --> 00:29:24.829 is to ARM every desk with electric fly swatter. 582 00:29:24.829 --> 00:29:28.967 The plan is like... you know like, oh man, this is going to date me a little bit. 583 00:29:28.967 --> 00:29:34.072 But in Warcraft 3, the humans have an ability where you can ARM the militia, 584 00:29:34.072 --> 00:29:36.141 where you take the like civilian minor people. 585 00:29:36.141 --> 00:29:39.444 And for a temporary period of time, they become soldiers; 586 00:29:39.477 --> 00:29:41.546 and we're going to do that with Labs members, 587 00:29:41.579 --> 00:29:44.215 by giving every single Labs member of a fly swatter. 588 00:29:44.249 --> 00:29:46.985 Nick: So you heard it here, we're hiring for Labs militia. 589 00:29:47.018 --> 00:29:51.589 Luke: No, I think, I think basically over the last year the focus has been on, 590 00:29:52.791 --> 00:29:56.027 to be completely honest, slowing down in a lot of ways; 591 00:29:56.060 --> 00:30:01.499 and with the goal of that being improving the verticals that we have 592 00:30:01.533 --> 00:30:05.970 and working on how we develop verticals in the future, 593 00:30:05.970 --> 00:30:08.106 along with reducing error output. 594 00:30:08.173 --> 00:30:12.677 And I, I think it's pretty objective to say that the errors that we've had, 595 00:30:12.710 --> 00:30:16.648 like let's say within 2025, have all been pretty minor. 596 00:30:16.681 --> 00:30:21.419 I mean, one of the ones that stood out the most was technically an NVIDIA graphic 597 00:30:21.519 --> 00:30:22.620 Luke: that had an error in it. 598 00:30:22.620 --> 00:30:23.121 Nick: Yeah. 599 00:30:23.254 --> 00:30:26.925 Luke: And we've had our slew here or there, but they've all been pretty minor. 600 00:30:26.925 --> 00:30:30.195 Luke: Like when a graph will get through without an updated subtitle. 601 00:30:30.195 --> 00:30:30.695 Nick: Our graphs. 602 00:30:30.695 --> 00:30:32.397 Luke: So it's like better, better, better, better, or whatever. 603 00:30:32.397 --> 00:30:34.032 Nick: I was gonna say, they were better, better, better. 604 00:30:34.065 --> 00:30:37.302 Luke: The, the impact of the errors that we've been having lately are pretty low; 605 00:30:37.302 --> 00:30:40.405 it's not that they don't exist, it's that they're, they're within the like, 606 00:30:40.405 --> 00:30:44.976 you know, we're humans and we're going to screw up sometime range of things 607 00:30:44.976 --> 00:30:48.646 instead of like often and heavy. 608 00:30:48.646 --> 00:30:50.815 That should be the goal moving forward; 609 00:30:50.815 --> 00:30:53.585 Luke: and I think we've been doing a great job of that for quite a while now. 610 00:30:53.618 --> 00:30:54.319 Nick: We've been doing well. 611 00:30:54.319 --> 00:30:54.819 Luke: Yeah. 612 00:30:54.819 --> 00:30:56.120 Luke: But yeah; slowing down a little bit, 613 00:30:56.120 --> 00:31:00.225 error reduction was what we were focusing on for a while. 614 00:31:00.258 --> 00:31:01.993 It's not that we don't want to focus on that anymore; 615 00:31:02.026 --> 00:31:05.296 we obviously want to keep focusing on that, but we're trying to increase our scope and grow. 616 00:31:05.296 --> 00:31:09.934 So we're currently in a hire and training phase and then hopefully 617 00:31:09.934 --> 00:31:12.604 coming closer to the end of the year and moving into the next year, 618 00:31:12.604 --> 00:31:15.139 we're going to try to get some more verticals online on the website, 619 00:31:15.139 --> 00:31:21.613 and start increasing our capabilities to match the new team size 620 00:31:21.613 --> 00:31:22.780 that we'll have when we're done hiring. 621 00:31:22.780 --> 00:31:25.416 Luke: That seemed like a decent summary; did I miss anything? 622 00:31:26.484 --> 00:31:26.918 Nick: Choo choo. 623 00:31:27.085 --> 00:31:29.854 Sammy: Oh, I guess, I guess I have, I have some thoughts. 624 00:31:29.988 --> 00:31:32.190 Where do you see Labs in the year from now? 625 00:31:32.190 --> 00:31:35.293 Like what kind of verticals do you want to be, get into? 626 00:31:35.293 --> 00:31:38.563 I'm assuming a year from now you have most of your current verticals, 627 00:31:38.563 --> 00:31:41.866 like as high as they can go? And you also want to continue expanding on that? 628 00:31:41.866 --> 00:31:46.371 Luke: As high as they can go is honestly not even really a fair statement for verticals, 629 00:31:46.371 --> 00:31:50.141 because as the industry changes we'll have to change with the industry. 630 00:31:50.241 --> 00:31:55.246 So if we're at like a ceiling, that ceiling is inherently temporary. 631 00:31:55.246 --> 00:31:57.582 And in regards to upcoming verticals, 632 00:31:57.582 --> 00:32:00.151 I want to see CPU and headphones be on the website. 633 00:32:00.151 --> 00:32:03.454 Outside of that, there's a lot of different directions we could go, 634 00:32:03.454 --> 00:32:05.990 but I'm more interested in what you guys would like. 635 00:32:05.990 --> 00:32:06.925 Lucas: More power supply. 636 00:32:07.125 --> 00:32:09.627 Luke: More power- More power supplies! 637 00:32:09.794 --> 00:32:11.529 Luke: What would- I guess what would that look like? 638 00:32:11.529 --> 00:32:13.865 Luke: What, what more do you want to do with power supply? 639 00:32:13.898 --> 00:32:17.568 Lucas: Well, we do measure a lot of data right now that we don't publish; 640 00:32:17.602 --> 00:32:19.704 but also like we, it takes a while to you know, 641 00:32:19.737 --> 00:32:22.140 develop how we should actually even publish that data. 642 00:32:22.173 --> 00:32:25.009 Or is, is it interesting? Because there's some tests like, 643 00:32:25.009 --> 00:32:28.079 okay everything passes, like every power supply passes this test; 644 00:32:28.112 --> 00:32:30.248 it's not even worth publishing kind of thing. 645 00:32:30.615 --> 00:32:32.116 But, yeah. 646 00:32:32.116 --> 00:32:36.621 Luke: Yeah, that could be a situation where that's like a blog post that we work on. 647 00:32:36.654 --> 00:32:39.924 It's like, look, there's this test; this thing actually really matters. 648 00:32:41.159 --> 00:32:44.762 Every single power supply does it fine; but like this is a way that this, 649 00:32:44.896 --> 00:32:47.098 this thing that these power supplies are all capable of 650 00:32:47.198 --> 00:32:49.901 is keeping you safe or whatever else. 651 00:32:50.001 --> 00:32:54.939 Lucas: Yeah, I think actually less focusing on verticals, getting verticals up, 652 00:32:54.939 --> 00:32:59.377 like as a way to move forward; and more just increase our capabilities to test things, 653 00:32:59.410 --> 00:33:01.279 and for like testing for videos. 654 00:33:01.312 --> 00:33:03.881 Like oh yeah, we have to test this thermal paste for this video; 655 00:33:03.881 --> 00:33:06.751 so now we have some experience with testing thermal paste. 656 00:33:06.818 --> 00:33:09.754 Maybe we can explore that later in the future, if we get another video on that. 657 00:33:09.754 --> 00:33:14.258 Or we can test- We had built some contraption for measuring like 658 00:33:14.292 --> 00:33:15.893 temperature sensors in multiple points. 659 00:33:15.960 --> 00:33:18.663 So we can build up all these things so that when a video comes in, 660 00:33:18.696 --> 00:33:21.666 we already have some tests that we can slightly adapt to it. 661 00:33:21.699 --> 00:33:22.667 Lucas: Or, you know- 662 00:33:22.667 --> 00:33:23.601 Luke: I like that a lot. 663 00:33:23.668 --> 00:33:26.504 Luke: I, I, sorry for cutting you off; I thought you were kind of done. 664 00:33:26.504 --> 00:33:30.742 But something that we've been working on and I want us to keep working on 665 00:33:30.742 --> 00:33:34.812 is, is the kind of like stability and steady state of things. 666 00:33:34.812 --> 00:33:40.151 Because if we are super sporadic and running around like crazy all the time, 667 00:33:40.151 --> 00:33:43.688 that's where you get things like errors ending up in things. 668 00:33:43.721 --> 00:33:47.658 So if we can move at a more measured pace and move with more experience 669 00:33:47.658 --> 00:33:50.194 because we've done more things in the space. 670 00:33:50.194 --> 00:33:52.630 Whether that's testing thermal paste or whatever else. 671 00:33:52.630 --> 00:33:56.000 When we get some crazy curve ball it's not that weird. 672 00:33:56.000 --> 00:33:58.269 Because if we've already dealt with these types of things 673 00:33:58.302 --> 00:34:00.905 at least once in the past and it's been decently well documented, 674 00:34:00.905 --> 00:34:02.473 we can look back at what we did before, 675 00:34:02.473 --> 00:34:04.776 and apply that to what we're doing in the future. 676 00:34:05.777 --> 00:34:08.713 And this is, this is part of the like move slower thing; 677 00:34:08.713 --> 00:34:14.018 it's not do less, it's take on more at a slower rate 678 00:34:14.018 --> 00:34:15.787 while having the same amount of output. 679 00:34:15.787 --> 00:34:16.621 Luke: Does that make sense? 680 00:34:16.621 --> 00:34:17.255 Nick: No. 681 00:34:19.057 --> 00:34:22.126 Luke: It's like instead of trying to get into... 682 00:34:22.260 --> 00:34:26.030 uh the, the cooling vertical, and the memory vertical, 683 00:34:26.030 --> 00:34:31.035 and the storage vertical, and the GPU vertical, and the CPU vertical, 684 00:34:31.035 --> 00:34:32.570 and the keyboard vertical, and the mouse vertical, 685 00:34:32.570 --> 00:34:33.404 and the monitor vertical- 686 00:34:33.404 --> 00:34:38.543 All at the exact same time; we try to do like one of them at a time, 687 00:34:38.576 --> 00:34:41.979 or spend a pretty significant amount of time refining what we already have, 688 00:34:41.979 --> 00:34:43.548 and then going into what- 689 00:34:43.548 --> 00:34:46.984 Nick: What is that? Like slow is smooth and smooth as fast? 690 00:34:46.984 --> 00:34:47.785 Nick: Or something like that. 691 00:34:47.785 --> 00:34:48.219 Luke: Yeah. 692 00:34:48.219 --> 00:34:50.054 Nick: See, building on what you're saying, 693 00:34:50.054 --> 00:34:52.723 and like by building capabilities and like knowledge base 694 00:34:53.891 --> 00:34:58.529 we've been growingly increasingly better at being like topical, like a cream. 695 00:34:59.730 --> 00:35:01.399 Luke: Yeah, the whole metaphor thing you're talking about. 696 00:35:03.568 --> 00:35:06.504 Lucas: It's, it's more like, like bottom up learning instead of top down; 697 00:35:06.504 --> 00:35:07.872 instead of okay we want to test this vertical, 698 00:35:07.872 --> 00:35:09.807 so let's develop these tests for this vertical. 699 00:35:09.807 --> 00:35:10.174 Nick: Yeah. 700 00:35:10.174 --> 00:35:12.076 Lucas: More like, like USB-C testing. 701 00:35:12.076 --> 00:35:13.711 Lucas: Oh yeah, and now we have some experience with that. 702 00:35:13.711 --> 00:35:16.914 If we have a video that comes in like with Nintendo Switch dock 703 00:35:16.914 --> 00:35:18.149 that we want to look at; oh, USB-C. 704 00:35:18.149 --> 00:35:20.084 Oh, we know how to do that, and we've tested a few things. 705 00:35:20.118 --> 00:35:22.487 We have a bit of a procedure that can apply to it; 706 00:35:22.887 --> 00:35:26.491 or you know, we've tested CPU so we can test thermal paste pretty easily 707 00:35:26.524 --> 00:35:28.092 by just adapting that a little bit. 708 00:35:28.359 --> 00:35:31.028 Nick: Even being able to explore some of like, 709 00:35:31.028 --> 00:35:33.731 when I mean by topical, ditching the cream; 710 00:35:33.764 --> 00:35:36.501 is more of like random stuff from like Reddit threads. 711 00:35:36.501 --> 00:35:40.271 Nick: Like hey like does- If I change the seconds to like track every- 712 00:35:40.271 --> 00:35:41.139 Nick: Am I going to lose power? 713 00:35:41.272 --> 00:35:44.008 Nick: So we set it up and we did it in a week and we're like, 714 00:35:44.008 --> 00:35:46.644 Oh sh**, we kind of do see that. 715 00:35:46.677 --> 00:35:48.212 Nick: Like being able to just- What? 716 00:35:48.212 --> 00:35:49.947 Lucas: We actually just got some updated results with that. 717 00:35:49.947 --> 00:35:50.581 Nick: Actually. 718 00:35:50.581 --> 00:35:53.584 Lucas: We're like exactly opposite, so it's like- 719 00:35:53.584 --> 00:35:54.051 Nick: Oh, is it? 720 00:35:54.051 --> 00:35:57.855 Lucas: Just like basically within, basically just within variance of things. 721 00:35:57.855 --> 00:35:59.423 Like the setups, we did the battery setups; 722 00:35:59.423 --> 00:36:03.861 they were specifically set up as like generic, like somebody's laptop. 723 00:36:03.861 --> 00:36:06.164 We didn't try to lock them down, all the variables. 724 00:36:06.297 --> 00:36:09.700 But yeah, basically if you have Wi-Fi on and Bluetooth on 725 00:36:09.867 --> 00:36:12.470 it could like vary your battery life way more than anything. 726 00:36:12.470 --> 00:36:14.572 But we were able to kind of test it and kind of see something, 727 00:36:14.572 --> 00:36:16.541 because we have a similar test to that. 728 00:36:16.541 --> 00:36:19.343 We can you know, test, test, you know; and say we're interested in this, 729 00:36:19.343 --> 00:36:20.945 hopefully other people are interested in this, 730 00:36:20.945 --> 00:36:23.681 Lucas: or it'll help somebody in the future if they have another question about this. 731 00:36:23.681 --> 00:36:26.150 Nick: But the fact that we were able to engage the conversation, 732 00:36:26.517 --> 00:36:29.153 like that's, that's what I find more important. 733 00:36:29.253 --> 00:36:30.288 Just being able to engage. 734 00:36:30.288 --> 00:36:34.625 Like there's, there's users in Reddit that test more fast than we do. 735 00:36:34.692 --> 00:36:40.131 And like regression test, like kernel changes to Linux within like a week. 736 00:36:40.131 --> 00:36:44.769 And they're like, hey here's like the... the shower of data. 737 00:36:44.769 --> 00:36:45.136 Luke: Yeah. 738 00:36:45.436 --> 00:36:46.370 Luke: Yeah, it's wild, it's cool. 739 00:36:46.504 --> 00:36:47.939 Luke: The showe- Oh man. 740 00:36:49.207 --> 00:36:49.540 Luke: Yeah. 741 00:36:49.540 --> 00:36:49.974 Nick: Data. 742 00:36:49.974 --> 00:36:54.212 Luke: There's a descriptor that you gave a while ago that I really liked; 743 00:36:54.212 --> 00:36:58.282 which was that it's, it's supposed to be like a- Like we grew up... 744 00:36:58.349 --> 00:37:01.552 well I don't know, you guys are... you're a little younger than me. 745 00:37:02.286 --> 00:37:06.324 I don't know how old Nick is, but we grew up on Bill Nye the science guy? 746 00:37:06.324 --> 00:37:07.758 Did you guys watch that in school? 747 00:37:07.758 --> 00:37:10.328 Did they roll out an old CRT? That still happened? 748 00:37:10.328 --> 00:37:12.530 Nice, cool; not that old, got it. 749 00:37:12.530 --> 00:37:16.801 Luke: But yeah like those, the like fun kind of \"oh, that's interesting\". 750 00:37:16.801 --> 00:37:19.337 Let's try it, let's experiment, let's play with things. 751 00:37:19.337 --> 00:37:22.106 Luke: That's how we want the lab to feel; 752 00:37:22.106 --> 00:37:26.911 and I think a big win in that department was checking out 753 00:37:26.911 --> 00:37:29.747 that seconds battery usage thing, 754 00:37:29.747 --> 00:37:32.350 because it's an interesting question that the community has. 755 00:37:32.350 --> 00:37:36.354 And I want Lab to be there for those. 756 00:37:36.354 --> 00:37:37.054 Nick: Lab or Labs? 757 00:37:37.188 --> 00:37:37.989 Luke: Labs, whatever. 758 00:37:37.989 --> 00:37:38.456 Nick: Lab. 759 00:37:38.456 --> 00:37:39.056 Luke: I want the- 760 00:37:39.090 --> 00:37:40.258 Nick: We are lab. 761 00:37:40.258 --> 00:37:41.892 Lucas: It's actually LTT Labs. 762 00:37:41.993 --> 00:37:42.560 Luke: Yeah. 763 00:37:42.560 --> 00:37:43.961 Nick: Yeah, do you know your own branding? 764 00:37:43.961 --> 00:37:46.564 Luke: Well, some people call it Lab, The Lab? 765 00:37:46.564 --> 00:37:48.933 Luke: There is only one, why do we call it Labs? 766 00:37:49.267 --> 00:37:51.302 Nick: I don't know, there's more than one lab. 767 00:37:51.302 --> 00:37:54.071 Lucas: I think there's legal reasons, but it's legally LTT Labs, I think. 768 00:37:54.071 --> 00:37:54.672 Luke: Okay. 769 00:37:54.672 --> 00:37:55.606 Lucas: I think, I don't know. 770 00:37:56.374 --> 00:37:57.975 Nick: But we had it in thirds. 771 00:37:59.010 --> 00:38:00.511 Luke: Yeah, but it's just one now. 772 00:38:00.511 --> 00:38:04.081 Luke: Have we evolved backwards? Did it go from Labs to Lab? 773 00:38:04.181 --> 00:38:07.251 Nick: Evolving backwards is called just devolving, but okay. 774 00:38:07.251 --> 00:38:08.386 Lucas: This is devolving. 775 00:38:08.419 --> 00:38:09.687 Luke: Yeah, exactly. 776 00:38:10.121 --> 00:38:11.956 Luke: To try to get back on track, 777 00:38:11.989 --> 00:38:15.593 we want to be able to be ready to test more things, 778 00:38:15.593 --> 00:38:18.162 and to have more fun with testing. 779 00:38:18.195 --> 00:38:21.265 So responding to community inquiries like, 780 00:38:21.265 --> 00:38:23.768 hey, does enabling seconds tank my battery life? 781 00:38:23.768 --> 00:38:28.639 The fact that we did that, and we did that quickly, and we did that in public, 782 00:38:28.639 --> 00:38:31.575 and we did that while communicating in the post, 783 00:38:31.676 --> 00:38:33.544 hey look, this isn't ready; 784 00:38:33.678 --> 00:38:36.947 this is just a few tests that we've done, and it's interesting so far, 785 00:38:37.048 --> 00:38:40.384 and then updating that post over time is awesome. 786 00:38:40.384 --> 00:38:42.019 I like the way that we communicated that. 787 00:38:42.019 --> 00:38:45.523 We weren't like, hey we ran three tests, this is conclusive. 788 00:38:46.357 --> 00:38:49.960 We made it clear to the audience that this is still something that we were working on. 789 00:38:50.861 --> 00:38:54.165 But we had some fun with it and actually did test the off the wall thing 790 00:38:54.165 --> 00:38:56.334 that wasn't necessarily a request from the writers. 791 00:38:56.334 --> 00:39:00.071 It wasn't a demand for a ShortCircuit video, or an LTT video, 792 00:39:00.071 --> 00:39:01.038 or the website or whatever else; 793 00:39:01.038 --> 00:39:03.174 it was just an interesting question, and we responded to it. 794 00:39:03.207 --> 00:39:05.376 And I hope that we keep doing things like that in the future. 795 00:39:05.376 --> 00:39:06.544 Luke: Thanks for watching, everyone. 796 00:39:07.278 --> 00:39:09.647 Luke: Well, I'm sure we'll do another one of these at some point; 797 00:39:09.647 --> 00:39:11.615 maybe next year, maybe before then. 798 00:39:11.615 --> 00:39:14.218 There may even be one that's a little bit more structured 799 00:39:14.218 --> 00:39:17.855 on the public YouTube channel at some point; we'll see. 800 00:39:17.855 --> 00:39:21.292 But it's been nice actually just trying to 801 00:39:21.292 --> 00:39:23.227 communicate to you what's actually been happening. 802 00:39:23.227 --> 00:39:26.430 We've been trying to be at least somewhat quiet for a while now, 803 00:39:26.430 --> 00:39:29.934 so that we can actually get some things done before we really talk about the changes, 804 00:39:29.967 --> 00:39:32.370 and I think it is time. 805 00:39:32.503 --> 00:39:37.908 So if you want to see more cool exclusives of how the company works, 806 00:39:37.908 --> 00:39:41.078 maybe check out the Linus and Terren interview. 807 00:39:42.046 --> 00:39:43.381 Lucas: Or PSU circuit, you know. 808 00:39:43.647 --> 00:39:44.115 Luke: Or that. 809 00:39:44.115 --> 00:39:45.082 Lucas: You're always welcome. 810 00:39:45.149 --> 00:39:48.152 Luke: You can see the inner workings of power supplies. 811 00:39:48.519 --> 00:39:49.854 Luke: But anyways, bye!"}