Technology has always been a huge part of my life, but my other secret passion
is badminton, a fast-paced technical racket sport. And after years of
searching, I finally found the perfect property to satisfy my craving for both.
By day, it's Smash Champs, a state-of-the-art badminton club. But
just beneath the surface lurks a secret.
Because if you look closely, you'll see we invested tens of thousands of dollars
into making this the ultimate esports
land gaming venue. I'm talking stupid fast fiber optic networking, on-site
dedicated servers for any game you can imagine, and a crapload of power. And
with the city of Siri's blessing, we're going to be bringing together over 200
gamers for what we call Whale Land. Woo!
We are getting so close to the first event here, guys. But there's one small
giant problem. While the occasional badminton player might watch a YouTube
video on their phone now and then, we haven't stress tested any of the
infrastructure that we installed over the last couple of years. That means we
have no idea how everything's going to hold up under heavy use. So, I recruited
a few dozen of my closest paid friends to come together for the ultimate stress
test, an overnight caffeinefueled weekend gaming session. And it starts in
6 hours.
Guess what? It's Lime Day. This week on ltstore.com, we are squeezing the heck
out of our margins to bring you some really sweet deals.
Today only, get a stretchy and comfy sport button-up for 50% off. Or snag a
fun and colorful Macadress Oxford for 75% off while supplies last. Save $20 on
a mystery screwdriver and get one of five exclusive Lime Day colorways. Pick
up a commuter backpack and you'll get either an 8bit do controller with a
charging dock or an LTX Ridge wallet for
free. And best of all, you get free shipping on every order over $175
if you're shopping from the US and every order over $250 Canadian if you're
shopping from anywhere else. So, don't wait. Visit LMG.gg/lime
before it's too late. No, I'm not doing that again. Thankfully, there's not that
much to do. We just have to unload the truck, build all the cable harnesses,
set up all the tables. Not to mention this server, which is brought to you by
our sponsor, Kioxia, who sent over some of their top-of-the-line CM7 Gen 5 NVMe
SSDs that are going to be powering the fastest Steam cache that we have ever
shown you guys. See, the thing is, we've only got 3 GB internet here, which is a
lot even for 200 people. But as we've
observed in the past, the biggest source of traffic at lands tends to be things
like updates. So, if Microsoft decided to push a big patch or if Deadlock
updated for the third time that week, then everything else could slow to a
crawl. That is where our caching server comes in. In simple terms, it's a
computer that sits in between our internet connection and all of our
attendees so that rather than our internet maxing out and lagging
everyone's games, the update will get saved to the cache the first time it
gets downloaded. And then when every other person goes to grab that same
update, hey, we already have the files stored and we can serve them from our
cache, saving our precious internet bandwidth. And all of that happens
entirely transparently to the user. Meaning, unless they're a network nerd,
they can't even tell that it's happening, aside from their update downloading super fast, of course.
And by super fast, I mean super fast.
Thanks to the these four Kioxia CM7 PCIe
Gen 5 enterprise SSDs at the heart of our caching server. Each of these 15.36
TB SSDs can do, and I kid you not, 14 GB
per second read speed and an astonishing 2.4 million 4K random readiops. They're
available in both E3. S and U.2 form factors like we have here. And they or
any other SSD from Kopia's phenomenal lineup is a great choice for your next
server project, big or small. You can check them out at the link in the description and we'll also have links to
the rest of the parts we're using. Speaking of other parts, um, what do we
got here? Uh, how about the case? Sure.
Okay, thank you for that. Oh god.
Right on the motherboard side.
This is the Silverstone RM23502
Mini. It's not the fanciest rack mount case ever, but it's nice and compact,
relatively inexpensive. They sell a rail kit for it. And more importantly, it has
5 and a/4 in bays up front. A perfect spot to put our
optical drive. Nope. There wasn't a ready-made solution for
this. Nobody really makes a Gen 5 ready
5 and a/4 in U.2 bay. So, well, actually, Icy Do has one on their
website under concept projects. Oh, okay. They have like mockups and everything. I
just don't know if they've ever made one. And I didn't respond to the email. So, Jake designed our own. Yeah, it's
basically the same thing, just crappier and 3D printed. Okay.
But it works well. And these are just Gen 5 uh U.2 to MCIO cables. So, other
than that, it's just kind of a plastic holder. Builds its purpose.
And you printed this on the new Prusso, right? Yeah. Well, it's not really that new Prussa XL.
Looks nice though. Hey. Yeah. Oh, I love this carbon fiber reinforced filament.
Yeah. And uh it shouldn't melt. Shouldn't.
Exactly. Does it have any cooling? Uh it will. That's what that is. So it
just kind of goes like kind of like this with the cable
and it goes like that. And then we have one 40 mm knock to a fan.
Mhm. I have a couple more in just in case.
Okay. But um I'm kind of hoping it's maybe okay.
Okay. I guess now's a good time to talk about the motherboard that limits us in
that way. This is the ASRock Rack Epic
4000D4U. It's MATX, which fits perfectly in our
case. It has remote management, which it darn well should being a server. And
perhaps most importantly, it's got these 2x8 MCIO connectors, which are perfect
for our four Kioxia SSDs at full Gen 5
speeds. It's also compatible with the CPU that we're using, which
in spite of the name of the board, is not an Epic. Instead, we've gone with
AMD's latest and greatest desktop CPU, the Ryzen 9 9950 X3D. As for why we made
this choice, well, a number of reasons. Starting with the fact that it's a lot
less expensive than many of the full fat epic alternatives. And with the amounts
of PCIe Gen 5 lanes and the memory bandwidth they have on these top-end
desktop chips these days, it's a surprisingly viable option. And that 3D
vcache does help it perform better in caching workloads like this. I couldn't
help noticing that our cooler though. Is this really going to work for a
9950X3D? We're going to find out. The cool thing about AMD chips is even
with the power super limited, they still rip. And AMD has like an eco mode
functionality that limits it to a certain TDP if we have to. But I'm thinking we can just try undervolting
and seeing what happens. Okay. I mean, realistically, we didn't choose this because we like doing any of
that. It more comes down to the fact that in a 2U form factor,
there aren't really a lot of other options. Lionus decided he had a meeting
to attend. So, I've been trying to get the SSD holder thing assembled. I
realized that I kind of flubbed up a little bit and didn't make space for the
connectors on the back. So, I trimmed that with some snips, screwed in our cables, and in theory, this can now
mount in the bay. I say in theory because haven't tried. Oh, it looks like
the holes mostly line up. That's a good sign. While Ryzen 9000 desktop chips do
support ECC, I uh couldn't help noticing that we very much did not do that thing.
Well, look how cool this RAM is. And it's worth addressing that while yes,
you can run ECC memory on supported motherboards with Ryzen CPUs that
there's still some weirdness with certain boards, and I just didn't want
to deal with that. Uh, speaking of weirdness, I couldn't help noticing that this case uses just
like a standard ATX power supply. Yeah, this this would work great in a rack.
See? See all that air flow? That would definitely be there if you had a switch
or something on top. Yeah. So, how are we planning to get any air flow to our power supply? The FSP
Twins Pro. It's a a dual redundant power
supply, but in an ATX form factor. Nice. Okay. So, we don't need a big
intake here is what he's trying to say. Yeah. This just makes sense.
What doesn't make as much sense is our boot SSDs. They're made of plastic,
Jake. I guess so. a couple Patriot Burst Elite
480 gigs. Yeah, they're they're nothing special, but at least they're redundant, I guess.
Not really. We'll talk about that later. But our network connection is this
NVIDIA/Mlanox Connect X6 25 gig dualport nick is
perfect. It'll give us failover in the event that one of our cables or ports
barks. Okay, cool. But 25 gig though, I mean, that seems kind of pinner for a cache of
this magnitude. Well, this wasn't exactly my first
choice. I wanted to run this a dual port
connect x600 gig nick instead so we could really push those well I mean to
be clear not really push those SSDs but push them a little bit. In fact I even
had Sean and the boys swap out the normal UniFi aggregation switch with
their like 48 port 25 gig/und gig
enterprise campus switch. Okay well that's cool. So why aren't we
using it? However, this motherboard you
see right here, my friend, while it does make use of all of the PCIe on this
chip, it shares half of the 16x lanes from this slot with the MCIO port right
here, which we need for our SSDs. There is a setting in the BIOS to change that.
Yeah. Where you could just bifrocate it, right? Yeah. We we really just want to use
eight lanes for the nick. But if there's any card in here that has all 16 lanes,
it just ignores the setting and doesn't work. However, if you have an 8x slot in
here, this doesn't work until you set that setting. So clearly it does something.
Okay. So what we could do maybe is just
put nail polish on. I was thinking Dremel,
but then that seemed pretty bad. So captain tape. It's like micro electrical
tape. So we just need to cover up exactly by eight of the lanes. That should give
us about 16 GB a second. And that's more
than double what we'd be able to do with that dinkier 25 gig card.
Which is like 6 GB a second. Yeah. We have a lot more to do before that goes in though.
All right, let's get this 5 and a/4 inch bay thing in here. Uh oh boy.
Uh-oh. Uh-oh. Okay, this might actually
No, she's in. She's in. Don't worry. There's extra room. I have an idea. I am
taking off these screws. Those are the wrong screws.
Don't worry about it. You want these ones? Don't worry about it. See, I'm just going to disassemble some more
of the case. But for why? For why? Everything is fine. For whence?
For to Why don't you just slide it in from this side?
Cuz it won't. Why? Unless I do this. No, that won't go.
Okay, there we go. Oh, that's the problem.
These little ears on this piece don't sit on the front. They go in from the
back. I didn't need to take any of that out. What happened? Don't worry about it. Everything's good.
Everything's great. Wow, that looks great. Does look great. I put little threaded
holes so that we could have a little screw that went in and kind of like friction held these in. Yeah,
they're kind of just like in there good enough. This is really tight, dude.
What? What are we talking about? This right here. Oh, it's tight.
Damn. It's tight as a tiger. What? So, what time is it, by the way?
Oh my gosh, it's almost 4.
I have Wancho in 45 minutes at the studio. Okay, let's be real. You don't actually
do Wancho at 4:30. Literally no air flow for this RAM. But who needs it, right?
Wait, did you not add the fans? What fans? You have to add fans to the case, you
door. What fans? The fans on the table.
Well, where the heck am I in the front there? There's two fan mounts there, bud.
Dude, there's no room. Yes, there is. There is exactly 25 mm.
He's over here playing with Falcon Northwest gaming computers.
Oh, that looks really cool. It does look cool. Kelt it up like a
special one for me to game at the land today with. It got a little bonked in shipping and
one of the fans was like scratching against something. So, we had to take
the fan out and receat it. But now it's good. And I'm just putting the cable
management back to where it was. Like, look at this. Look at how anal they are.
That is mint. Do we only have three fan
screws? This is why I hate doing builds outside of the studio. Okay, I'm done my
part of the job. Where are you at? Just don't start saying numbers.
numbers. 236 75.
Dude, I'm ADHD enough that that would actually trip me up. Okay, mark the thing.
My god. Well, you keep saying 38. Oh, you wrote it down.
Man knows himself. Honestly, I think there's there's definitely worse ways we could have done
this. Oh, yeah. 100%. Mint. 100%. Okay, here we go.
Snickers. Yo, this might be too nice for us. Wait,
wait, wait, whoa, whoa, whoa. You're breaking the law.
Don't do it. You're not wrong. Don't do it. We don't need the bad GB's today. Oh
god. No, we don't. We literally don't have time for bad GB's.
Okay. Do we have a monitor? No, we have IPMI.
Oh, we have a remote management interface. Are we just going to take this straight
up to the rack? Kevin's looking at me. You see him? Look at him. He's looking at me like that
because he gave me a monitor. But I don't need your stinking American
monitor. Elbows up, baby. Oh my god. Stop.
Oh god, that's awesome. I need another power cable. No, you don't. Here, I'll solve that for
you. Ah, fixed. That CPU fan really goes, eh?
It's because the IPMI is not moving up yet.
Oh, I was like, why can't I feel any air from the fan that goes down there?
Yeah, it's impossible. Stick your finger in the hole.
No, I don't need Do it. Just do it. Stick it. Stick it. You feel it?
Yeah. You didn't even You did his finger did not penetrate. I just want the cable management done.
While Jake sets up that server, why don't I talk layout with Chase?
This is only a a network and infrastructure test right now. So, we're
only expecting about 50 to 60 people to show up. So, it's not the entire land,
but that's plenty for us to stress test the physical elements, like our floor
protectors, for example, which this is my first time actually seeing. It's from
Facility Armor. It seems like they did a great job. These just go down. They've
got these Velcro strips that hold everything in place and seems like it
won't be damaged. Now, one of the main things because this is a badminton
center first and foremost is that we need to be able to roll in and roll out
as fast as possible. So, we've got a pretty cool plan for how to handle our
cabling. Instead of distributing hundreds of Ethernet cables to each
attendee to plug in, we've made these power and Ethernet whips that will just
be rolled out as part of our setup. We measure the length of our cables using
our schematics. Then we had Infinite Cables make a whole bunch of custom
length Cat 6A Ethernet cables and threeprong extension power cords. Huge
shout out to those guys. Again, it really does seem like their supply of
cables is infinite. We'll link some of their stuff in the video description cuz
they provided literally all the cabling for the entire facility, including
the fiber. More on that in a sec. Then all of the cables run back to these. On
the bottom, we've got these sick 50 amp 28vt power distribution units. And then
we use this Chungus California standard
power connector that's pretty standard in event spaces. And then on top of
them, we actually bolted our network switches. Each of our rows gets one of
these Ubiquiti Enterprise 48 PoE switches. These are the same ones that
we used back at Whale at LTX in 2023.
RIP LTX. I still can't believe they sent all of these anyway because they did.
Each seat will get at least a two and a half gig connection. And who knows,
maybe we could bump it up to 10 gig for the VIPs or the whales. Each row gets
two 10 gig single mode fiber uplinks into those switches for a total of 20 GB
aggregate. And then you can actually see all the power connections spaced out
along here. Once we're ready to do a real event, this entire space will be
filled with desks and chairs. Okay, we got the system booted up here,
plugged into our network switches for one of the rows just for now. I want to
make sure everything's working before we actually install it in the rack. And it
does look like the network card is running at PCIe Gen 4 by8, which is
great. We taped off the extra pins, and that happens to have worked. However, I
only see two of the four SSDs we have installed, which means the BIOS setting
I talked about earlier is not set. Moment of truth. I'm in the BIOS.
Sick. Okay, it is set to the wrong setting. I know that's weird to be like
excited about something being wrong. Hit save and let's see if it works. Fingers
crossed. Oh, actually, I should mention this thing. It's uh one of Eaton's new
trip light cloud connected UPS's. Um,
I'm running the cache off it right now because we were kind of screwing around
with the power situation during the setup. And these are pretty sweet because rather than having like a
management weird thing that runs off of them, you just plug it into the internet
and adopt it to their cloud software and you can monitor and control these UPS's
and they're like small, compact, and not crazy expensive. We'll have links to
those down in the description along with all the other parts like the bits for
our computers and our Kio SSDs down there. Got back into the BIOS and the
SSDs are all there. Our network card is all there. The system works, baby. Now
we just have to put it in the rack, do all the software setup, and there's
already people here. It's now I don't have a watch. It's 7:00 p.m. We're late.
There we go. Perfect. I know we said earlier we were going to mount this on
rails, but the rails that come in the rail kit look awful. And uh it just has
this nice spot where it can sit right here and be future Sean's problem. Screw
that guy. Look at that. We got both of our 100 gig links in an aggregate. We're
ready to go once we set it up. Except I was smart
enough to already have done that at the office. And here's how you do it with
two really cool pieces of software. First, LANC cache. That's the heart of
how this whole operation works. It takes
the incoming data stream, which is like your Steam game downloads, stores them
on the storage locally so that the next time you download something, it gets
pulled from the cache instead of the internet, which is what saves us
bandwidth. and then the operating system called flat car container Linux. Now,
because LANC cache runs in a Docker container, which is kind of like a mini
operating system that runs within your main operating system, and it's the only
thing we're going to really be running, we don't want the main operating system
to be like this big cludgy mess like Windows that needs updates all the time.
So, we're going to be using flat car container Linux, which is specifically
built to be as simple and minimal as possible just for running containers.
And what's really cool about flat car, the main reason that we're using it is
because the entire setup gets distilled down into a text file. That's how you
configure LANC cache. We've got our four Kioxia SSDs are going to be in RAID
zero. All of that good stuff. And then you install it to the computer. And the
text file tells it how to set everything up. So if you have a boot drive die or
you have a a new server or or whatever it is, you just reinstall it with a text
file and it resets it up. It's perfectly repeatable. It's super easy to maintain
and it automatically updates which is a really good thing for us. We're not
going to be running these events all the time and um having to worry about that
sort of stuff. Not something I want to deal with. Now, this is a pretty
advanced way to set this up, but if you're curious, we're going to post a
simplified version of this flat car configuration in the video description
so you can play around with it yourself. Uh it's also the same thing we used for the DIYVPN video. That was pretty fun
and it worked pretty good. It's just that this time we're installing to our own physical hardware and not through a
cloud provider. So for this time we're going to use the more old school manual
install tool. It's literally called flat car-install. So download that and
install any missing dependencies in the current version of Ubuntu server. We
just needed bzzip 2. So pseudoapp install bzzip 2. Once you have that, you
have to download your flat car config file but the system readable version.
And if you don't know how to make that, check the description. and we'll have some instructions on how to do that.
Make flat car install executable, figure out what drive you're installing to.
Ours is dev/ sda and then run the command. It's going to download the flat
car image, do all the installing, and then throw your config into it. And once
it's done, you should be able to take your USB out, reboot your server, and
you'll have a setup LANC cache. Cool tutorial, Jake. Now, let's see how
it worked, cuz our network test is in full swing. We'll start with a look at
my setup. My poolcooled computer's a little tough to move, but thankfully
Falcon Northwest just sent over this frag box for us to check out
editorially. And I was like, "Oh, good timing, boys. I guess I'll use this for
the land." They actually did a sick custom print on it for us as well. That
looks amazing. And uh it's running a Ryzen 9 9950X 3D and an RTX 5090 with a
whopping 96 gigs of DDR5 6000. This is
as fast as it gets. And it is shockingly
compact and totable. Of course, that's not going to help us if Jake's network
cache doesn't work. And she works freaking awesome. Now, the only question
is what happens if somebody else tries to download a game at the same time.
Luke, Luke, download a game. Lionus
wants to battle. So, let's see what kind of download speed you get and if it
affects mine. 1.8.
1.9. Okay. But what am I at? Dude
hasn't moved at all. What are you at? 2.5. So, we are already exceeding our
total internet connection speed here. That means the caching is absolutely
working. Now, we just got to have everybody download a game at the same
time. Think we can coordinate that? Okay. 3 2 1 click download. Go.
Okay. Let's check this out. 15 GB. Wow, that's pretty sick, actually. The SSDs
are like, "Sorry, are you like doing something? I I I didn't even notice. Did
you breathe next to me? This is freaking awesome.
It doesn't even care, dude. The CPU usage is at 10%.
How's our internet usage? Do we know that? The inbound, which would be
Yeah. Well, there'll be some axe there, but realistically about a gig to play.
So, a couple people managed to find a game that we don't already have cached.
Yeah, that's freaking awesome. So, it works. Woohoo.
Hey, great job, Kyia making SSDs that are way way way way way way
faster than my CPU can decompress. My CPU is at 100%.
Yeah, it sure is, buddy. That's probably more our bottleneck right now than the
actual caching server. Oh, 100%. In conclusion, even though this test was
only with 40 or 50 people once we expand
to the entire facility, we've got plenty of headroom left over. If you guys
enjoyed this one, why don't you check out the time I roasted people's PCs at
Whale Land back in 2023. Editor note, put this in.
Thank