Wait. What? Okay. That was a reenactment. But a few weeks ago, my computer didn't die
just like that right in the middle of a gaming session. Turns out, my boot SSD catastrophically
failed, giving me an excuse to rebuild my gaming rig and my kids' machines and every
other system in my house with no SSD, no hard drive, no storage at all. And yet, look at this.
I'm gaming, I've got Windows, I've even got a ton of games installed. How is this possible?
I mean, I know you can download RAM, but downloading storage?
Well, kind of. The storage still needs to be somewhere. And luckily for you,
Kyoksia, who sponsored this video, provided some of their CD8 enterprise SSDs. But
what he's saying is true. None of them are in the gaming PCs. Instead, it's all in here.
That is freaking amazing. Did you just pull one out so we got redundancy and everything?
Yeah. I mean, it's cool. But there are some disadvantages. Like, you kind of made every
computer in your house a bit. We'll get to that later. Roll the thing.
I'm really excited about this because in spite of the fact that network booting has existed for
longer than Jake here, it's not something we've ever really played around with much and certainly
never for a home gaming setup. So how does it wake?
Well, the current way to network boot from something like a NAS, Network Attached Storage,
is ice-guzzy. There is also newer stuff like NVMe over fabric. But as far as I know, it doesn't
really exist, at least in Windows Land yet. And certainly it's not within the scope of this project.
But as silly as it sounds to boot from a NAS, is it? I mean, you can video edit on files from your
NAS. We do that all the time. You can game using files on a NAS. We do that all the time. So why
not Windows from your NAS? It comes with some key advantages. For one thing, it centralizes your
storage, which means redundancy can be more economical to safeguard against the drive failure.
Instead of having to put at least two drives in every one of your systems, you can just put,
I don't know, six drives in here, plus one extra for redundancy. And it can make for easy
management of backups and lightning fast system restores. Which I'm sure is great for kids PCs
because I bet they mess stuff up all the time. Mine are actually pretty good so far,
but I'm sure that's coming. My dad would be happy just because he could get rid of MSN
Messenger faster. I wasn't allowed. I did it anyways. What I am allowed to use is de-duplication
or compression with ZFS as the back end. That's super cool because it gives us even more efficiency
out of the storage that we have. But it's not all sunshine and rainbows. Like what about the lack of
system? Okay, we're going to talk about that later. First, let's build a NAS. Okay. Why don't you tell
us about this weird platform you decided to use for this project? I hate this computer. This is the
worst server I have ever played with, at least in like the last five years. In fairness to Intel,
it's a reference platform. Software development platform. It is never meant to go to an end user,
but it has been a little annoying to get working. The good news is it's a super cool system. It's
dual 144 core processors, which is hilarious overkill. It's not even great for this setup,
actually, because iSCSI not super multi-threaded. But he's been looking for an excuse to tool
around with it, and now we have it. But aside from the CPUs, we've loaded this thing up with an
NVIDIA Mellanoff ConnectX 6 dual port 100 gigabit per second, Nick. We don't actually need the fastest
network cards in the client PCs. But what we have to consider is that because this one machine is
going to serve all of them, it needs to be as fast as all of their network cards combined. That is
assuming that they're all trying to hit it at the same time. I mean, you wouldn't want your
computers to be slow, right? No, no, I really wouldn't. And with the speed of these CD8 Gen4
drives that Kyoksia provided for this little thought experiment, we can use as fast a Nick as we can
possibly get our hands on and never have to worry about our drives being the bottleneck. Now, what
are we using for networking? Because I don't have a 100 gig switch. Yeah, and ubiquity doesn't make
one yet. So instead, hey, our buddies for Microtik. This is the CRS 51816XS 2XQ. Not only does it have
two 100 gig ports, that's the 2XQ, but it has 1625 gig ports, which is perfect. We plug our
server in here, and then we can plug in all the other things downstream to the 25 gig ports. At
least we could if we had 25 gig networking in any of our systems. Well, we're going to. The funny
thing is as silly as it feels to use these CD8 drives for gaming, this kind of an application
is not that far off of what drives like this are designed for. They've got die failure recovery as
well as power loss protection and parity protection to make them extraordinarily reliable. They're
available in capacities much higher than the 3.84 terabytes that we're deploying here. So if you've
got simulation work or you're storing gigantic high performance databases, something like this
would be perfect. And we're going to put games on it. And Windows. So we've allocated about 10
terabytes of storage for a shared game drive and about one terabyte for the boot drives for each
of our systems. We've got network, we've got stores, that's all we really need. You're going to take us
to that. I got the whole thing. Don't drop it. All right. Oh my god. What the hell? Oh, there wasn't
a drive in that tray. We're good. What the hell? Well, Jake gets things hooked up. Let's talk about
the long term plan because obviously I'm not going to be keeping a 288 core Intel reference
server in my house to boot my computers. What I actually want to do is take my two loud NASs
and consolidate them into one quiet one. So take that 45 drives chassis, rip out the hard drive
mounts, put in NVMe mounts, and then run them all off of that NAS software that I invested in.
Whether or not we boot the computers from them, that kind of depends on how this proof of concept
goes. Why are you taking the drives out? We just put them in. Well, I told you this server is so
great, right? It doesn't boot with SSDs in it. What? So you have to like let it boot and then you
hot plug the SSDs back. Tried so many things that Intel Rep was like, it's my last day. I'm not even
kidding. That is legitimately what happened. Good luck at wherever you go. But I took the
voluntary payout. I'm leaving. All right, let's go look at the software stack. Now my assumption was
we were just going to use TrueNAS for this because we love TrueNAS. Yeah, TrueNAS is great. But we
didn't. Yeah, I used it for Ice Cozy like a couple months ago and it was pretty slow. There's also
this thing called Starwind vSAN, which there's a free version that they let you use forever
on even multiple machines. You get a high availability for free and it's like purpose
built for Ice Cozy. So it's a lot easier to just set up and use for this exact workload. So why
not, right? There is a bare metal install, but I didn't notice until I'd already set up the like
Proxmox image version. So that's what we're using instead. It should still be performant. I just took
our network card and our six SSDs and just PCIe pass through them. Right. So if there's any performance
hit for virtualization, it should be pretty negligible. This system has 512 gigs. Right.
But you have to like load it. So I just picked 256 because that's faster. And I selected 64 cores.
I mean, this type of server is really meant for like a web host or like virtualization web hosts.
Yeah. Like something that just parallelizes like crazy. Yeah. Oh, we can see we have our
Kyoxy SSDs. Look at that. All six of them. Wow. And then I went ahead and created a ZFS pool. It
does give you the option to like just make a raid pool. It seems to be slower for some reason.
The ZFS is cool because it has RAM caching. And that'll, it'll just put your files in memory.
And memory is of course super fast. So with RAID Z1, that means that one of our drives is
not contributing to our overall storage, but it also means that we can lose any one of our drives
just like I did when I was in the middle of my game. And none of our systems should go down.
We're going to show the real world results of that live test where Jake ripped a drive out in the
intro a little bit later. After that, you make a volume, which is kind of just like a folder. So
we've got one for our game drive here. This really reminds me of the TrueNAS interface a lot.
It's like the TrueNAS interface, but like less crappy. You notice how like responsive it is?
Yeah. And then for iSCSI, we have to make a LUN, which is kind of like a virtual disk. Now
some of it is just needlessly complicated. Now Starwind does a good job of making it less
complicated. Let's create one. iSCSI standalone because we're not high availability. It's just
one system. We'll make it in our Linus smells thing. We'll call it Linus really smells.
And 100 gigs and done. That's all you have to do, right? So now I have a 100 gig drive and it
lives on my network. But how do you use it? That's the annoying part. This is the overly
complicated BS I'm talking. I'm sure there's a reason for it, but this is like the address of
the storage. I don't know why the date needs to be in there. It's not supposed to be the
current date or the date of creation. It's like the date you registered your domain. As far as I
can tell, entirely arbitrary. You can set it to whatever you want. One way or another,
like any other network resource, it needs an address. But what is the difference between
an iSCSI storage drive and an SMB shared folder? Well, one is block storage and one is file
storage. And if you don't care what that means, it's basically imagine iSCSI like an SSD. It's
the whole drive. It's meant to plug right into a computer or not be shared. And you can imagine
an SMB is like, here's a folder. You know, multiple people can use one folder. And they won't conflict
with each other if they both try to read a file or write to a folder. Yeah, because it's designed
to handle that. Luckily, Starwind can handle both. You see, SMB share, that'll be for our games. Since
most games, you install them to an SMB share, you can play them on multiple computers and it just
works. And then they're going to use their local my documents or games folder for any save data or
whatever else, hopefully. Some games not so good. Yeah. So this is going to be a little hit or
miss, but we can at least try it. Yeah. And worst case, you can install a game locally.
Question for you. For this test, did you swap out my network card for the fastest one that we've
got? No. I mean, the one you have right now, an X540T2 dual 10 gig one should in theory work just
fine for this. Yeah. I mean, if you think about it, 10 gig, that's way faster than say a three.
Yeah. Yeah. But we can go faster. Let's go faster. I mean, how fast do you want your computer to be?
I don't know. I just want to see what this is capable of. I think we should do the 100 gig
Nick. There is a PCIe slot limitation. Your system only has a by four. Yeah, that's the thing about
high end desktop platforms being effectively dead these days is if you want to get into kind of
weird homelab stuff like this on a consumer board, you're not going to have enough PCIe lanes, but
this one's cool. It still has a second gen four by four link to the CPU, not through the chipset.
But this is a gen three card. So you're going to get four gigabytes a second max. Boom.
Card installation. Let's go. Yeah. Dude, this is a buy one slot though. No, it's by four. No, it's by one.
Oh, well, where's the, what board is this? Well, I got my Thunderbolt card in the buy four. Why?
Because that it needs it. Don't you have an iKron thing anyways? Yeah, but I also have a Thunderbolt
dock. That like one Gigabyte a second is all you're going to get out of that. Oh, 10 gig then. At
that point, I might as well just leave the 10 gig card in there. Well, we could use this one. This
is a connect six card. It's only 25 gig ports, but it's gen four. So we'll get two gigabytes a second.
All right. How big do you want your drive to be? Toerite? Oh, okay. I was expecting more. Yeah,
sure. Fine. No, that's fine. So the first thing we have to do, of course, is make the drive. Boom,
done. Okay. And now we're done. You're officially ice goes eat. Okay, we should probably configure
the BIOS then. Yep. Since our computer doesn't have a disk to boot from, we have to tell it
where to boot from. Right. And this is potentially more complicated and annoying than one might think.
Um, the easy way is to set up what we're going to set up on a USB stick and just plug it into the
computer and boot off the USB stick. But then we'd be adding storage and we said we were getting
rid of that, right? I guess. So instead, we're going to use our router. Now you might be familiar
with a DHCP server. It gives devices on your network and IP address. Yeah. It can also give
them other things, specifically where to look for storage. Interesting. Okay. Now the router you
have a ubiquity dream machine doesn't have the most advanced DHCP configuration. So you can only
really tell it one location to look, which obviously doesn't work great if we have 10 computers trying
to boot off of it, right? Because then they'd all be getting the same drive. Right. So we made it
a little bit more complicated and I'm running a web server that it talks to and tells it its MAC
address. And then based on the Mac Address, it responds with the disk that it's supposed to use.
It sounds complicated, but honestly, it's like 30 lines in a script here. You can look at this.
Like this is the name of the device and this is the disk it's about to go to. And this is the MAC
address. All right. I'm going to link all of the like scripty things we used in the description
if for whatever reason you want to try to do this. What we're going to be using to install
Windows is Windows PE or the Windows pre-installation environment. It's like many Windows that most of
the time is used for network installing Windows, which is what we're doing. You have to take the
image that you make and modify it slightly, namely just to add the network drivers. In our case,
that would be Melanox drivers and I've already done that. Okay. But it also means I need to
modify this boot script because which Mac Address do you want? Oh boy. Oh, wait, wait, yeah, right,
it should be 98. Checking media presence, media present. Okay, we're good. All right.
Oh, it worked. Wow. That's, that's great. We are glossing over a lot of this, but you'll see the
scripts down below. This would be the one that you would use to boot Windows PE. We're glossing
over so much. This is like the glossiness. It's so shiny. Editor, can you make this shot just
like really shiny? Ew. Is that like a filter? Why does it look like wet? Set boot URL. That's like
the file storage location. Initiator IQN is like the name of your machine. And then sandhook,
that attaches the disk. You can see his name Linus. And this is the bits it needs to download
for Windows PE and then boot. And then when you go to actually boot, you just change the sandhook
to sand boot and then just comment all this out. And in theory, it should just boot. Okay. Yeah,
wow. Look at that. We're in. So we're going to mount a network drive that has our Windows
installer on it. I don't know if it's strictly necessary, but I like injected the proper drivers
into the Windows installer. Okay. You can do that while you're like selecting your disk. It'll
say install driver. Yeah. We'll just prompt you. I just did it because I didn't want to have to screw
with that. It's every time. Every time. Oh, boy. Oh, no, it's going to do that thing. Why? Okay,
well, no, I can just use this one. I think you need to unplug the SSDs. This is the step that
usually fails. Please God, please, please. God can't help you where we're going. This is the step
that installs drivers. I think it's good. When I was researching this, everyone talked about how
this is just like an absolute nightmare to do and like disable Windows update and all this stuff.
It seems to just work as long as you have a Nick that supports it. We ran into one very modern Nick
that just didn't support it. Those 10 gig Aquanta ones that are in the LAN PCs. That's why when
we were talking about this at the beginning of the video, we were saying two and a half gig
because we were using the onboard real tech. I tried for days to get it to work, but their
driver just refuses to launch at boot time. We're going to boot into NPE one more time because
we have to make a registry edit. I'm not 100% sure if it's 100% necessary, but there's a ton
of people that mentioned it causes blue screens if you don't. We're just going to disable the page
file. To be clear, disabling your page file could cause more stability problems than it solves,
so it's not something that we would recommend under normal circumstances. This is just
experimental for now. Wait, is it just working? Is this it? I think it's just working. This
shouldn't be that cool to me, but I'm not going to lie. It kind of is. I'm just using a computer
like I would normally use a computer. I'm going to install a program like I would normally install
a program. There are definitely some red flags. You can see the active time of my boot drive here
is 100% and my average response time, while it's usually pretty reasonable in the sub-20
milliseconds range, occasionally it'll pop up to half a second. That's a yikes. You don't want to
see that. This is not a CPU issue, how long it's taking to extract this Steam package. I have a
7950X. Oh wow, do you see that? Average response time as it's testing random writes is getting into
the multiple seconds. That bad. Also, even these sequential results are a fraction of what I was
expecting to be getting, even with the PCIe slot limitations. Oh wow. Even before the game starts
downloading, okay, there it finally goes. Just the reservation process was taking up 100% of the
active time of the drive. Still is actually. Network is doing nothing. And to be clear,
this is not the fault of those Kyoksia drives. They're just like, yeah, we'll send over storage
if you guys are doing a wild project. Those are really fast drives. So this comes down to the
limitations of our network infrastructure and our implementation. How's it going? Oh, you've got
four of them working? That's sick. How's that? Oh, it's terrible. Really? Yeah. Why? Just everything
is terrible. Like slow? Oh yeah. Even just launching Steam took a while. Jake, I'm logging into Steam
on this system and I just got a system leg spike that was so intense, I couldn't move the mouse
cursor. Not good. There's something very wrong with the permissions on the SMB share and it just
like, yeah, okay. It's not very happy. It was an interesting experiment. I want to know now if
you can make it go fast. Me too. But I think that's an experiment for another day. I want to go unplug
one SSD. I'm going to do it right now. Note that we've got four systems running right now, one of
which is actively verifying the integrity of... Oh, you did it already. Okay. Well, that's one
thing it does well. Was I supposed to wait? But I think it might finally be time to talk about
the disadvantages. Which one? You made every computer in your house way slower. You added a
single point of failure. So if that server has a problem, which you probably will,
none of your computers work. And how are you going to fix that server? You don't have a computer to
manage it with to try to fix it. It's great. And you've added a whole bunch of costs that
realistically you could put two consumer grade drives in every one of your machines and run
RAID 1. And complexity and management headaches. But I'm just stoked that this works. I think it's
cool. And the fact that it would work with just about any computer, even if we wouldn't recommend
doing this at home. Like, man, okay. The fact that the real techniques work at all. If we could
tune it, something like a LAN cafe. Oh, I could totally see this like that. From a manageability
standpoint, where you actually have an IT admin who's working on it instead of just a home user?
Usually when you have a big install base of hundreds of computers, you might use folder
redirection. You have a local disk and then you just like redirect the user folders to a network
share or something like that. That way you're optimizing your storage use through things like
deduplication and compression. You're also making it way easier for IT to manage how much space
everybody's using and make sure that if something gets accidentally deleted or intentionally deleted.
Defreeze. It can be restored. But this is dumb. I wouldn't recommend booting Windows over iSCSI.
It does seemingly work, but. But I would recommend checking out Kyoksia's CD8s. They're PCIe Gen4
drives with NVMe 1.4 support. They use their 112 layer 3D TLC flash. They're very fast,
they're very great, and they have a whole lineup of other drives if they don't specifically
meet your need. And you can check them all out at the link down below. Thank you Kyoksia
for making it possible for us to deploy Jake on Fool's errands like this for extended periods
of time because it's pretty cool. And look, now we can take those drives and build them into a new
NAS. Yeah, and do something useful with those. If you guys enjoyed this video, maybe go check out
the one where Jake and I built a NAS for your house, which is no longer here.