{"video_id":"HaMjPs66cTs","title":"Building a 100TB Folding@Home Server!","channel":"Linus Tech Tips","show":"Linus Tech Tips","published_at":"2020-05-05T14:53:29Z","duration_s":1031,"segments":[{"start_s":0.16,"end_s":6.0,"text":"this is day two of isolation with a sore throat and you're gonna be able to tell","speaker":null,"is_sponsor":0},{"start_s":3.76,"end_s":10.24,"text":"how long i've been at home by how long my facial hair gets but that doesn't","speaker":null,"is_sponsor":0},{"start_s":8.08,"end_s":14.24,"text":"mean we're gonna stop making videos so we did a folding at home call to arms","speaker":null,"is_sponsor":0},{"start_s":12.32,"end_s":18.0,"text":"last week where we had people contribute computing power from their desktop","speaker":null,"is_sponsor":0},{"start_s":15.839,"end_s":23.439,"text":"machines to help run protein folding simulations to help in the fight against","speaker":null,"is_sponsor":0},{"start_s":20.0,"end_s":25.359,"text":"covid19 among other diseases but as we","speaker":null,"is_sponsor":0},{"start_s":23.439,"end_s":28.8,"text":"mentioned in that video one of the main problems the folding project is having","speaker":null,"is_sponsor":0},{"start_s":27.119,"end_s":32.96,"text":"right now is not necessarily that there's not enough willing volunteers to","speaker":null,"is_sponsor":0},{"start_s":31.039,"end_s":37.52,"text":"contribute their compute power but that there's not enough servers to intake all","speaker":null,"is_sponsor":0},{"start_s":35.36,"end_s":42.32,"text":"the data so we said we were going to work with the folding guys to build","speaker":null,"is_sponsor":0},{"start_s":39.2,"end_s":45.04,"text":"ourselves a folding at home target","speaker":null,"is_sponsor":0},{"start_s":42.32,"end_s":49.12,"text":"server and put it in our server room that's what i'm going to be doing here","speaker":null,"is_sponsor":0},{"start_s":46.64,"end_s":53.44,"text":"at home today and i've got one of our old decommissioned servers along with a","speaker":null,"is_sponsor":0},{"start_s":51.52,"end_s":57.76,"text":"care package from Jake that i'm going to be opening up using to upgrading it and","speaker":null,"is_sponsor":0},{"start_s":55.68,"end_s":62.559,"text":"then lysoling the heck out of this thing and sending it back to him and the video","speaker":null,"is_sponsor":0},{"start_s":60.0,"end_s":66.32,"text":"is brought to you by glasswire instantly see your current and past network","speaker":null,"is_sponsor":1},{"start_s":64.08,"end_s":69.92,"text":"activity detect malware and block badly behaving apps on your pc or Android","speaker":null,"is_sponsor":1},{"start_s":67.92,"end_s":74.479,"text":"device with glasswire use offer code Linus to get 25 off glass wire at the","speaker":null,"is_sponsor":1},{"start_s":72.479,"end_s":77.68,"text":"link in the video description so it's pretty obvious what all the volunteer","speaker":null,"is_sponsor":1},{"start_s":75.92,"end_s":82.4,"text":"contributors like you are doing for the folding at home project but many of you","speaker":null,"is_sponsor":0},{"start_s":80.0,"end_s":87.04,"text":"probably won't know what role the server plays in all this so here's the thing","speaker":null,"is_sponsor":0},{"start_s":84.479,"end_s":92.32,"text":"when you're folding your machine says hey i need a job to work on what the","speaker":null,"is_sponsor":0},{"start_s":89.6,"end_s":96.799,"text":"server does is it says all right i've got a spot available go ahead and","speaker":null,"is_sponsor":0},{"start_s":94.799,"end_s":101.119,"text":"connect to me i'm going to generate a job for you send it off to you then you","speaker":null,"is_sponsor":0},{"start_s":99.36,"end_s":105.119,"text":"go ahead you crunch those numbers and the job is ready to submit and the","speaker":null,"is_sponsor":0},{"start_s":102.64,"end_s":108.72,"text":"server says okay i'm ready for you you send it back and it stores it until the","speaker":null,"is_sponsor":0},{"start_s":107.28,"end_s":113.439,"text":"researchers who are working on the folding at home project are ready to","speaker":null,"is_sponsor":0},{"start_s":110.56,"end_s":118.64,"text":"grab it and do something with it so for that to work you actually need a few","speaker":null,"is_sponsor":0},{"start_s":115.6,"end_s":120.56,"text":"things one is a decent amount of CPU","speaker":null,"is_sponsor":0},{"start_s":118.64,"end_s":124.96,"text":"power folding at home recommends an eight core CPU which we actually do have","speaker":null,"is_sponsor":0},{"start_s":123.119,"end_s":129.52,"text":"but uh for a couple of reasons i'll get into later we're gonna be upgrading the","speaker":null,"is_sponsor":0},{"start_s":126.64,"end_s":133.599,"text":"one in here it's a little pinner they also recommend about 64 gigabytes","speaker":null,"is_sponsor":0},{"start_s":132.0,"end_s":137.44,"text":"of RAM that's gonna let us handle anywhere from a thousand to twelve","speaker":null,"is_sponsor":0},{"start_s":135.599,"end_s":141.44,"text":"hundred clients connected to our machine the other things we're gonna need are a","speaker":null,"is_sponsor":0},{"start_s":139.28,"end_s":146.64,"text":"buttload of storage in order to hold those completed jobs and of course a","speaker":null,"is_sponsor":0},{"start_s":144.319,"end_s":151.04,"text":"fast internet connection in order to connect to all those clients so because","speaker":null,"is_sponsor":0},{"start_s":149.12,"end_s":154.959,"text":"we've got 10 gigabit Ethernet already built into this motherboard we don't","speaker":null,"is_sponsor":0},{"start_s":152.72,"end_s":159.44,"text":"need to make any upgrades there but we do need to swap out the 32 gigs of RAM","speaker":null,"is_sponsor":0},{"start_s":157.12,"end_s":163.92,"text":"we've got in here that eight core CPU and of course we're going to need to add","speaker":null,"is_sponsor":0},{"start_s":161.599,"end_s":170.72,"text":"some storage let's start with our CPU the xeon e3 2618 lv3 that we've got in","speaker":null,"is_sponsor":0},{"start_s":167.2,"end_s":172.64,"text":"here was just fine for basic file server","speaker":null,"is_sponsor":0},{"start_s":170.72,"end_s":175.92,"text":"duties especially given that we were running a raid card in here when we","speaker":null,"is_sponsor":0},{"start_s":174.16,"end_s":180.16,"text":"originally deployed it so the CPU didn't even have to handle any storage parity","speaker":null,"is_sponsor":0},{"start_s":178.08,"end_s":185.28,"text":"calculations or anything like that but it's only got a 2.3 gigahertz base clock","speaker":null,"is_sponsor":0},{"start_s":183.44,"end_s":189.92,"text":"which means that if we're actually loaded up with you know somewhere over a","speaker":null,"is_sponsor":0},{"start_s":188.0,"end_s":195.84,"text":"thousand clients that thing is going to be running darn near base speeds not to","speaker":null,"is_sponsor":0},{"start_s":193.36,"end_s":200.8,"text":"mention that folding at home recommends a gigabit internet connection and we are","speaker":null,"is_sponsor":0},{"start_s":199.04,"end_s":205.68,"text":"actually going to be running 10 gig and we're hoping to provision off anywhere","speaker":null,"is_sponsor":0},{"start_s":202.48,"end_s":207.92,"text":"between about 4 gigabit and 5 gigabit of","speaker":null,"is_sponsor":0},{"start_s":205.68,"end_s":213.28,"text":"our internet connection to run to this machine so we want to see this thing","speaker":null,"is_sponsor":0},{"start_s":210.64,"end_s":217.76,"text":"melt if we can get away with it let's pull out this chip and now seems like as","speaker":null,"is_sponsor":0},{"start_s":215.68,"end_s":223.12,"text":"good a time as any to open up my care package from jakku","speaker":null,"is_sponsor":0},{"start_s":220.72,"end_s":226.56,"text":"what do we got in here this is looking like pretty good stuff","speaker":null,"is_sponsor":0},{"start_s":224.959,"end_s":230.64,"text":"here's our two power cables for the redundant power supply that's already","speaker":null,"is_sponsor":0},{"start_s":228.159,"end_s":234.799,"text":"built into this machine a vga cable so we can test it make sure it actually","speaker":null,"is_sponsor":0},{"start_s":232.0,"end_s":238.4,"text":"powers on after our upgrade oh shoot these RAM sticks got kicked around a","speaker":null,"is_sponsor":0},{"start_s":236.48,"end_s":242.439,"text":"little bit 32 gig registered ecc memory modules so that's","speaker":null,"is_sponsor":0},{"start_s":240.72,"end_s":247.68,"text":"going to give us a total of 128 gigs of RAM in our completed config","speaker":null,"is_sponsor":0},{"start_s":245.599,"end_s":253.2,"text":"i will explain why we need so much a little bit later i've got some","speaker":null,"is_sponsor":0},{"start_s":250.319,"end_s":253.2,"text":"thermal paste","speaker":null,"is_sponsor":0},{"start_s":253.36,"end_s":263.72,"text":"some screws and this seems like there's more than just an optane SSD in it ah there it is","speaker":null,"is_sponsor":0},{"start_s":260.16,"end_s":267.84,"text":"all right so we are upgrading to an e5","speaker":null,"is_sponsor":0},{"start_s":263.72,"end_s":270.96,"text":"2697 v3 that is a high performance 14","speaker":null,"is_sponsor":0},{"start_s":267.84,"end_s":273.6,"text":"core 28 thread processor or at least it","speaker":null,"is_sponsor":0},{"start_s":270.96,"end_s":276.96,"text":"was a few years back nowadays it's not really anything special but it should be","speaker":null,"is_sponsor":0},{"start_s":275.759,"end_s":281.04,"text":"more than enough for what we're doing here today go ahead get that installed","speaker":null,"is_sponsor":0},{"start_s":279.52,"end_s":285.36,"text":"it's so nice he included a little cleaning pad but he also sent me a clean","speaker":null,"is_sponsor":0},{"start_s":282.8,"end_s":289.44,"text":"CPU so i'm not gonna bother with that i love these thick boy thermal paste","speaker":null,"is_sponsor":0},{"start_s":287.6,"end_s":291.919,"text":"tubes look at that it's a hand for scale i mean it's a small hand but you know","speaker":null,"is_sponsor":0},{"start_s":290.96,"end_s":296.24,"text":"how it is oh wow that's a lot uh uh well the good","speaker":null,"is_sponsor":0},{"start_s":294.8,"end_s":299.44,"text":"news is we're not likely to run out anytime soon","speaker":null,"is_sponsor":0},{"start_s":298.0,"end_s":303.04,"text":"this is a very early unit of this motherboard super micro actually sent us","speaker":null,"is_sponsor":0},{"start_s":301.28,"end_s":307.44,"text":"i think it was one of the first ones off the line for this thing because it was","speaker":null,"is_sponsor":0},{"start_s":304.96,"end_s":312.8,"text":"one of their first matx server boards that had 10 gig Ethernet built in 10 gig","speaker":null,"is_sponsor":0},{"start_s":310.4,"end_s":316.32,"text":"used to be more of like uh well you why wouldn't you just use an add-in card for","speaker":null,"is_sponsor":0},{"start_s":314.32,"end_s":321.6,"text":"that and it's become over the last few years an option to have just","speaker":null,"is_sponsor":0},{"start_s":318.88,"end_s":326.479,"text":"pre-built into the motherboard and our CPU upgrade is done now let's","speaker":null,"is_sponsor":0},{"start_s":324.56,"end_s":331.68,"text":"talk about the storage folding at home recommends about 50 terabytes of storage","speaker":null,"is_sponsor":0},{"start_s":329.52,"end_s":336.24,"text":"for one of these servers but because our network connection and our CPU not to","speaker":null,"is_sponsor":0},{"start_s":333.759,"end_s":340.56,"text":"mention our memory are all beefed up we could quite possibly need more than","speaker":null,"is_sponsor":0},{"start_s":338.08,"end_s":344.96,"text":"that so we're configuring ours with eight of these 12 terabyte ironwolf pro","speaker":null,"is_sponsor":0},{"start_s":342.96,"end_s":350.88,"text":"nas drives from seagate that's going to give us about 96 terabytes of raw space","speaker":null,"is_sponsor":0},{"start_s":348.08,"end_s":356.24,"text":"or 72 once we give up two of our drives for parity and then once we've formatted","speaker":null,"is_sponsor":0},{"start_s":353.039,"end_s":359.12,"text":"about 60 terabytes of usable storage","speaker":null,"is_sponsor":0},{"start_s":356.24,"end_s":361.68,"text":"fortunately this case is super easy to install drives in it's actually one of","speaker":null,"is_sponsor":0},{"start_s":360.639,"end_s":366.479,"text":"the only things that i really really like","speaker":null,"is_sponsor":0},{"start_s":364.0,"end_s":371.36,"text":"about it you just pop these open slide them in","speaker":null,"is_sponsor":0},{"start_s":367.68,"end_s":373.919,"text":"no tools no sleds no nothing just","speaker":null,"is_sponsor":0},{"start_s":371.36,"end_s":379.759,"text":"boom just like that that takes care of our capacity but these are mechanical","speaker":null,"is_sponsor":0},{"start_s":376.0,"end_s":382.24,"text":"drives and even in zfs raid they're not","speaker":null,"is_sponsor":0},{"start_s":379.759,"end_s":386.72,"text":"going to be particularly fast so believe it or not that's where our overkill","speaker":null,"is_sponsor":0},{"start_s":384.72,"end_s":391.52,"text":"memory upgrade comes in so we're going to be using 64 gigs of","speaker":null,"is_sponsor":0},{"start_s":389.039,"end_s":396.88,"text":"our RAM to meet the recommended specs from folding at home but we're going to","speaker":null,"is_sponsor":0},{"start_s":393.28,"end_s":400.479,"text":"be using the other 64 gigs to actually","speaker":null,"is_sponsor":0},{"start_s":396.88,"end_s":402.4,"text":"act as an arc or a or a cache for our","speaker":null,"is_sponsor":0},{"start_s":400.479,"end_s":405.759,"text":"mechanical drives these are going to be running in a quad","speaker":null,"is_sponsor":0},{"start_s":403.919,"end_s":410.88,"text":"channel configuration so we should have plenty of bandwidth and uh i don't know","speaker":null,"is_sponsor":0},{"start_s":409.199,"end_s":414.4,"text":"whatever i haven't shouted out kingston for sending these to us in a long time","speaker":null,"is_sponsor":0},{"start_s":413.039,"end_s":419.44,"text":"they actually sent them originally for the six workstations one CPU project but","speaker":null,"is_sponsor":0},{"start_s":417.44,"end_s":423.919,"text":"we have used them for tons of things and they've worked basically anything we've","speaker":null,"is_sponsor":0},{"start_s":421.36,"end_s":428.24,"text":"put them in so good on them for that you guys probably noticed though that","speaker":null,"is_sponsor":0},{"start_s":425.68,"end_s":437.12,"text":"inside that optane box there was also an actual Intel optane 905p 960 gig SSD","speaker":null,"is_sponsor":0},{"start_s":433.919,"end_s":439.759,"text":"this is gonna act as a level 2 arc so","speaker":null,"is_sponsor":0},{"start_s":437.12,"end_s":444.4,"text":"between our RAM and this SSD we are hoping to accelerate our mechanical","speaker":null,"is_sponsor":0},{"start_s":441.52,"end_s":448.639,"text":"storage array quite significantly alleviating any performance bottlenecks","speaker":null,"is_sponsor":0},{"start_s":446.479,"end_s":452.319,"text":"that we could run into there where's my screwdriver so we're just","speaker":null,"is_sponsor":0},{"start_s":450.24,"end_s":456.56,"text":"going to take one of these open pci express","speaker":null,"is_sponsor":0},{"start_s":453.44,"end_s":458.08,"text":"gen 3 8x slots i was going to put it in","speaker":null,"is_sponsor":0},{"start_s":456.56,"end_s":462.8,"text":"the next slot over but i realized that puts it right up against the edge of the chassis and there's no cooling fan there","speaker":null,"is_sponsor":0},{"start_s":461.199,"end_s":467.039,"text":"so probably better off sitting next to the hba and having a cooling fan near it","speaker":null,"is_sponsor":0},{"start_s":464.96,"end_s":470.4,"text":"as opposed to being farther away from another heat source but not having any","speaker":null,"is_sponsor":0},{"start_s":468.72,"end_s":474.72,"text":"direct cooling one thing we're not changing is this hba the difference","speaker":null,"is_sponsor":0},{"start_s":472.8,"end_s":479.28,"text":"between an hba and a raid card is that an hba doesn't have a CPU built into it","speaker":null,"is_sponsor":0},{"start_s":477.28,"end_s":483.28,"text":"to handle raid calculations it's just a card that adds more ports to your","speaker":null,"is_sponsor":0},{"start_s":481.28,"end_s":486.8,"text":"motherboard in this case eight 12 gigabit per second sas ports even though","speaker":null,"is_sponsor":0},{"start_s":485.44,"end_s":490.56,"text":"we're only running them at SATA six gigabit per second through this back","speaker":null,"is_sponsor":0},{"start_s":488.8,"end_s":494.639,"text":"plane over here that's it then for the hardware upgrades and it all comes down","speaker":null,"is_sponsor":0},{"start_s":492.56,"end_s":498.8,"text":"to software configuration to decide how best to use it so one of the things we","speaker":null,"is_sponsor":0},{"start_s":496.4,"end_s":502.96,"text":"could do is instead of using 64 gigs of our memory to accelerate our hard drives","speaker":null,"is_sponsor":0},{"start_s":500.8,"end_s":507.12,"text":"we might cordon off just 32 gigs for that giving ourselves the capability to","speaker":null,"is_sponsor":0},{"start_s":504.879,"end_s":510.479,"text":"handle over 1500 clients that's something that those extra CPU cores","speaker":null,"is_sponsor":0},{"start_s":508.8,"end_s":514.399,"text":"might come in handy for or we might realize that we're better off using our","speaker":null,"is_sponsor":0},{"start_s":512.08,"end_s":518.959,"text":"extra CPU horsepower to enable real-time data compression to our hard drives like","speaker":null,"is_sponsor":0},{"start_s":516.479,"end_s":522.64,"text":"maybe we run into a network limitation or some other system limitation and","speaker":null,"is_sponsor":0},{"start_s":521.12,"end_s":526.72,"text":"realize we just don't have enough storage guys that's something that you","speaker":null,"is_sponsor":0},{"start_s":524.399,"end_s":530.64,"text":"can do with cfs for now though all that remains","speaker":null,"is_sponsor":0},{"start_s":530.88,"end_s":535.839,"text":"close this puppy up there we go screw it together","speaker":null,"is_sponsor":0},{"start_s":536.8,"end_s":545.04,"text":"and give the whole thing a good lysol wipe down i like Jake he's a good guy he","speaker":null,"is_sponsor":0},{"start_s":541.92,"end_s":546.64,"text":"can have the uh lemon scented wipes","speaker":null,"is_sponsor":0},{"start_s":545.04,"end_s":549.839,"text":"this is gonna be like the best smelling server ever","speaker":null,"is_sponsor":0},{"start_s":550.24,"end_s":554.72,"text":"hopefully matthias is here to pick it up and we'll send it back to the office","speaker":null,"is_sponsor":0},{"start_s":556.64,"end_s":561.36,"text":"okay let me open this okay so i don't actually have the rails","speaker":null,"is_sponsor":0},{"start_s":560.16,"end_s":566.24,"text":"for this server so we're just going to sit it on top of this storinator","speaker":null,"is_sponsor":0},{"start_s":564.32,"end_s":569.68,"text":"it's kind of ghetto but whatever i also don't have a screwdriver so i'm not","speaker":null,"is_sponsor":0},{"start_s":567.519,"end_s":574.959,"text":"going to look at what Linus did but i kind of know what it is already","speaker":null,"is_sponsor":0},{"start_s":573.04,"end_s":578.56,"text":"and for a little server she sure is heavy there we go","speaker":null,"is_sponsor":0},{"start_s":580.16,"end_s":587.36,"text":"okay hey he gave me the cables back noise","speaker":null,"is_sponsor":0},{"start_s":584.72,"end_s":593.44,"text":"he stole the thermal face though what the","speaker":null,"is_sponsor":0},{"start_s":588.839,"end_s":593.44,"text":"heck okay we got power","speaker":null,"is_sponsor":0},{"start_s":595.92,"end_s":601.839,"text":"plug one in for ipmi so we can remote control this thing","speaker":null,"is_sponsor":0},{"start_s":600.88,"end_s":607.2,"text":"uh oh right i need a display","speaker":null,"is_sponsor":0},{"start_s":605.12,"end_s":610.399,"text":"do we even have spaces left on this thing","speaker":null,"is_sponsor":0},{"start_s":608.64,"end_s":617.8,"text":"sweet okay i gotta try and find this back here so this is p","speaker":null,"is_sponsor":0},{"start_s":613.279,"end_s":617.8,"text":"where do these cables even come from","speaker":null,"is_sponsor":0},{"start_s":618.959,"end_s":625.44,"text":"should probably label this right yeah","speaker":null,"is_sponsor":0},{"start_s":623.2,"end_s":629.76,"text":"i'll leave that for the next person hey it's working","speaker":null,"is_sponsor":0},{"start_s":627.279,"end_s":634.079,"text":"let's make sure everything's here 110","speaker":null,"is_sponsor":0},{"start_s":631.2,"end_s":636.8,"text":"28 gigs of RAM everything else it's working","speaker":null,"is_sponsor":0},{"start_s":635.68,"end_s":641.76,"text":"yes like Linus mentioned earlier we're going to be using something called zfs for our","speaker":null,"is_sponsor":0},{"start_s":640.16,"end_s":645.44,"text":"hard drive array so not only does it do the work of a file system by controlling","speaker":null,"is_sponsor":0},{"start_s":643.6,"end_s":649.279,"text":"how your data is stored and retrieved but it also handles how all the physical","speaker":null,"is_sponsor":0},{"start_s":647.2,"end_s":652.8,"text":"or virtual disks are partitioned and works to ensure the integrity of the","speaker":null,"is_sponsor":0},{"start_s":650.88,"end_s":655.92,"text":"data on said drives now that we've installed zfs and verified that it's","speaker":null,"is_sponsor":0},{"start_s":654.399,"end_s":660.88,"text":"actually working we're going to create our z pool now we're going to be using something called raid z2 which is","speaker":null,"is_sponsor":0},{"start_s":658.88,"end_s":664.959,"text":"actually based on raid 5 but it's going to allow us to lose two total drives","speaker":null,"is_sponsor":0},{"start_s":662.64,"end_s":667.6,"text":"before we lose any data i've already kind of pre-configured the command","speaker":null,"is_sponsor":0},{"start_s":666.32,"end_s":672.32,"text":"written out all the disk name so we don't have to wait for that and i've also defined our cache in here which","speaker":null,"is_sponsor":0},{"start_s":670.88,"end_s":677.839,"text":"means we're going to have that 905p working right off the bat so just copy","speaker":null,"is_sponsor":0},{"start_s":674.88,"end_s":680.56,"text":"this command in there paste enter and yeah","speaker":null,"is_sponsor":0},{"start_s":678.88,"end_s":684.88,"text":"okay let's check our z pool list now that it's should be in there we see 87","speaker":null,"is_sponsor":0},{"start_s":683.2,"end_s":690.24,"text":"terabytes which is a little lower than 96 but you know drive partitioning and","speaker":null,"is_sponsor":0},{"start_s":687.04,"end_s":691.68,"text":"what have you and then if we run dfh","speaker":null,"is_sponsor":0},{"start_s":690.24,"end_s":696.16,"text":"we can see we actually end up with 60 terabytes of usable space that's a","speaker":null,"is_sponsor":0},{"start_s":693.68,"end_s":698.56,"text":"little bit down from 96 but it should be plenty for what we're trying to do","speaker":null,"is_sponsor":0},{"start_s":697.44,"end_s":704.0,"text":"there's still a couple more things we need to check off the list before we can install our folding software namely i","speaker":null,"is_sponsor":0},{"start_s":701.76,"end_s":706.8,"text":"want to set our arc cache to a max of 64 gigs and then i'm going to enable","speaker":null,"is_sponsor":0},{"start_s":705.44,"end_s":710.72,"text":"compression because i think it's going to be fine with the 14 cores we have i","speaker":null,"is_sponsor":0},{"start_s":709.04,"end_s":714.72,"text":"think realistically our network is going to be our bottleneck but we'll enable it","speaker":null,"is_sponsor":0},{"start_s":713.279,"end_s":718.0,"text":"for now and if down the road we have to turn it off no big deal","speaker":null,"is_sponsor":0},{"start_s":716.399,"end_s":722.32,"text":"now as a sort of last step i'm just going to kind of look at our z pool so","speaker":null,"is_sponsor":0},{"start_s":719.519,"end_s":725.92,"text":"we're going to do z pool status folding and we can see all eight of our drives","speaker":null,"is_sponsor":0},{"start_s":724.32,"end_s":729.279,"text":"one two three four five six seven eight and our cash drive are all there and","speaker":null,"is_sponsor":0},{"start_s":727.519,"end_s":733.12,"text":"ready to go and we have no errors which is perfect","speaker":null,"is_sponsor":0},{"start_s":731.36,"end_s":736.639,"text":"in order for folders to actually be able to access our folding server i've","speaker":null,"is_sponsor":0},{"start_s":734.88,"end_s":740.48,"text":"created some firewall rules now that was pretty easy but we're also going to create something called a traffic","speaker":null,"is_sponsor":0},{"start_s":739.04,"end_s":743.519,"text":"limiter because we don't want this folding server to eat up all of our 10","speaker":null,"is_sponsor":0},{"start_s":742.16,"end_s":748.48,"text":"gigs of bandwidth so we're going to limit it to 5 gigabit but i'm going to show you guys how this works on the","speaker":null,"is_sponsor":0},{"start_s":746.56,"end_s":751.68,"text":"laptop as a baseline i'm going to run a speed test on the system before we","speaker":null,"is_sponsor":0},{"start_s":750.32,"end_s":755.2,"text":"enable the traffic limiter so you can actually see the difference","speaker":null,"is_sponsor":0},{"start_s":753.68,"end_s":760.32,"text":"okay we have our results in and we're looking at 230 down and 200 up so i","speaker":null,"is_sponsor":0},{"start_s":758.56,"end_s":763.92,"text":"think i'm going to limit it to 5 megabit which will be a very stark change and","speaker":null,"is_sponsor":0},{"start_s":762.16,"end_s":767.44,"text":"you should be able to see it right away so we're going to go into our two rules","speaker":null,"is_sponsor":0},{"start_s":765.44,"end_s":773.92,"text":"here it's already set to megabit so we'll do 5 and then we'll go to our in","speaker":null,"is_sponsor":0},{"start_s":770.88,"end_s":775.6,"text":"set that to five and then apply and then","speaker":null,"is_sponsor":0},{"start_s":773.92,"end_s":780.56,"text":"we just need to find our ip which i already did so","speaker":null,"is_sponsor":0},{"start_s":777.72,"end_s":784.88,"text":"10.20.3.35 for this laptop so i'll just go add the rule in so here's our results","speaker":null,"is_sponsor":0},{"start_s":783.6,"end_s":788.72,"text":"and yeah five megabit right on the dot ish","speaker":null,"is_sponsor":0},{"start_s":787.36,"end_s":792.639,"text":"so now we're gonna change this over to five gigabit and then set it up on our","speaker":null,"is_sponsor":0},{"start_s":790.8,"end_s":796.079,"text":"actual server now that we've got our z pool configured and working we've got","speaker":null,"is_sponsor":0},{"start_s":794.24,"end_s":798.959,"text":"our bandwidth limiter and our firewall rules set up it's just a matter of","speaker":null,"is_sponsor":0},{"start_s":797.68,"end_s":804.88,"text":"calling the folding guys and getting them to install the software so we can get this work server going","speaker":null,"is_sponsor":0},{"start_s":802.32,"end_s":810.32,"text":"well it's been a week and as you can see my isolation facial hair is quite long","speaker":null,"is_sponsor":0},{"start_s":807.6,"end_s":814.8,"text":"and this is great i've got jakku on the line and we're going to be checking out","speaker":null,"is_sponsor":0},{"start_s":812.16,"end_s":818.8,"text":"how well our folding server is running so Jake give me the good news well the","speaker":null,"is_sponsor":0},{"start_s":816.959,"end_s":822.72,"text":"good news is there's enough work units for most people now which is great did","speaker":null,"is_sponsor":0},{"start_s":821.2,"end_s":826.24,"text":"we help with that or was that just serendipitous yes but it's not only us","speaker":null,"is_sponsor":0},{"start_s":824.639,"end_s":830.48,"text":"helping so they've also got microsoft azure i think they donated some servers","speaker":null,"is_sponsor":0},{"start_s":828.24,"end_s":834.0,"text":"um oracle you know the guys that make java and lots of other things they're","speaker":null,"is_sponsor":0},{"start_s":831.92,"end_s":837.76,"text":"doing stuff too and then there's us but uh if we look on the stats here we can","speaker":null,"is_sponsor":0},{"start_s":835.6,"end_s":842.8,"text":"definitely see that our server does have jobs available which means work units","speaker":null,"is_sponsor":0},{"start_s":840.0,"end_s":846.399,"text":"are available nice 19 000 jobs i mean for the first like five days it was up","speaker":null,"is_sponsor":0},{"start_s":844.959,"end_s":850.8,"text":"there was basically nothing because they just couldn't create enough projects it","speaker":null,"is_sponsor":0},{"start_s":848.8,"end_s":854.56,"text":"was no longer the servers but the people that were the ben or the bottleneck so","speaker":null,"is_sponsor":0},{"start_s":852.88,"end_s":859.279,"text":"how we doing now how much usage have we got i was kind of expecting more to be","speaker":null,"is_sponsor":0},{"start_s":856.56,"end_s":863.6,"text":"honest but if we look at the past history you can see like","speaker":null,"is_sponsor":0},{"start_s":861.68,"end_s":870.0,"text":"they spike up quite a bit so like right there is half a gigabit and then that's","speaker":null,"is_sponsor":0},{"start_s":866.56,"end_s":873.92,"text":"600 and then if we scroll out actually","speaker":null,"is_sponsor":0},{"start_s":870.0,"end_s":876.16,"text":"so that's like 1.6 gigabit wow so from","speaker":null,"is_sponsor":0},{"start_s":873.92,"end_s":880.32,"text":"my understanding these actual like spikes a lot of the time are actually","speaker":null,"is_sponsor":0},{"start_s":878.16,"end_s":884.639,"text":"when the different work servers are like communicating with each other the actual","speaker":null,"is_sponsor":0},{"start_s":882.079,"end_s":888.079,"text":"like network bandwidth of individual clients connecting isn't too much but","speaker":null,"is_sponsor":0},{"start_s":886.959,"end_s":893.04,"text":"it's when they're trying to talk together that it really starts to hurt and how much storage have we used so far","speaker":null,"is_sponsor":0},{"start_s":891.36,"end_s":896.8,"text":"last i checked this morning it was already like two and a half or three","speaker":null,"is_sponsor":0},{"start_s":894.639,"end_s":900.639,"text":"terabytes of storage used that's awesome and we can look at our zfs stats here","speaker":null,"is_sponsor":0},{"start_s":898.959,"end_s":905.839,"text":"which actually paints a little bit more of a picture so our RAM cache i set to","speaker":null,"is_sponsor":0},{"start_s":903.44,"end_s":910.399,"text":"64 gigs is completely full you can see it's it's ramping up slowly filling so","speaker":null,"is_sponsor":0},{"start_s":908.079,"end_s":913.6,"text":"as things are being removed from the RAM cache when they would normally get","speaker":null,"is_sponsor":0},{"start_s":912.079,"end_s":919.04,"text":"dumped back to the disks they actually go to the level two cache instead","speaker":null,"is_sponsor":0},{"start_s":915.839,"end_s":920.56,"text":"so this is already up to 64 or 40 gigs","speaker":null,"is_sponsor":0},{"start_s":919.04,"end_s":923.92,"text":"and it's only been like an hour or something since i reset it so it does","speaker":null,"is_sponsor":0},{"start_s":922.48,"end_s":929.279,"text":"take a few days to fill obviously because it's only filling up with stuff that's being removed from the primary","speaker":null,"is_sponsor":0},{"start_s":926.56,"end_s":933.76,"text":"cash but um fantastic and hey i've actually got an update for you so we","speaker":null,"is_sponsor":0},{"start_s":931.279,"end_s":936.959,"text":"have created an exclusive folding at home shirt design","speaker":null,"is_sponsor":0},{"start_s":935.44,"end_s":940.0,"text":"and all profits from the shirts are going to be contributed to causes within","speaker":null,"is_sponsor":0},{"start_s":938.56,"end_s":943.36,"text":"canada around the research relief and treatment of covet 19. so you guys are","speaker":null,"is_sponsor":0},{"start_s":942.32,"end_s":948.32,"text":"going to have to check that out at lttstore.com and","speaker":null,"is_sponsor":0},{"start_s":946.079,"end_s":951.68,"text":"what's really cool is Intel is going to match","speaker":null,"is_sponsor":0},{"start_s":949.279,"end_s":955.199,"text":"every dollar raised dollar for dollar to a maximum of 40 grand and so that 40","speaker":null,"is_sponsor":0},{"start_s":953.6,"end_s":958.8,"text":"grand is actually going to go directly to the folding project so they can get","speaker":null,"is_sponsor":0},{"start_s":956.88,"end_s":962.8,"text":"servers and stuff right yes that's correct oh apparently you already knew","speaker":null,"is_sponsor":0},{"start_s":960.399,"end_s":966.72,"text":"about this i did already know yes this is actually like one of my favorite","speaker":null,"is_sponsor":0},{"start_s":964.72,"end_s":972.24,"text":"shirt designs and the qr code on the back takes you to our our team stats on","speaker":null,"is_sponsor":0},{"start_s":969.199,"end_s":973.68,"text":"extreme overclocking dot","speaker":null,"is_sponsor":0},{"start_s":972.24,"end_s":978.48,"text":"so join the cause guys we're gonna leave this server running for the foreseeable","speaker":null,"is_sponsor":0},{"start_s":975.36,"end_s":980.56,"text":"future i'll let you go now Jake and uh","speaker":null,"is_sponsor":0},{"start_s":978.48,"end_s":986.72,"text":"just share a word from our sponsors with the good viewers out there drop.com the","speaker":null,"is_sponsor":1},{"start_s":983.519,"end_s":988.48,"text":"cost gmr 54x iso gaming headset is the","speaker":null,"is_sponsor":1},{"start_s":986.72,"end_s":992.56,"text":"featured product this time around these are audiophile approved and based on a","speaker":null,"is_sponsor":1},{"start_s":990.24,"end_s":996.16,"text":"popular cost headset that was custom engineered for immersive 3d sound giving","speaker":null,"is_sponsor":1},{"start_s":994.72,"end_s":999.839,"text":"you positional cues for where your enemies are coming from there are some","speaker":null,"is_sponsor":1},{"start_s":998.0,"end_s":1002.48,"text":"changes made from the original so they've reduced the tension in the","speaker":null,"is_sponsor":1},{"start_s":1001.12,"end_s":1006.079,"text":"lightweight headband for improved comfort they include a cord splitter","speaker":null,"is_sponsor":1},{"start_s":1004.56,"end_s":1010.639,"text":"inline microphone with remote and a detachable boom mic and the boom mic","speaker":null,"is_sponsor":1},{"start_s":1007.92,"end_s":1015.12,"text":"works with the ps4 xbox nintendo switch and more without hassle get it today and","speaker":null,"is_sponsor":1},{"start_s":1013.12,"end_s":1019.36,"text":"new users who sign up on drop.com will get 20 off on this headset we're gonna","speaker":null,"is_sponsor":1},{"start_s":1017.68,"end_s":1024.319,"text":"have it linked in the video description so thanks for watching guys if you enjoyed this video maybe check out our","speaker":null,"is_sponsor":1},{"start_s":1022.8,"end_s":1028.72,"text":"previous part of this video where we built a monster client folding machine","speaker":null,"is_sponsor":0},{"start_s":1026.88,"end_s":1032.0,"text":"as opposed to a server that's some really good stuff too","speaker":null,"is_sponsor":0}],"full_text":"this is day two of isolation with a sore throat and you're gonna be able to tell how long i've been at home by how long my facial hair gets but that doesn't mean we're gonna stop making videos so we did a folding at home call to arms last week where we had people contribute computing power from their desktop machines to help run protein folding simulations to help in the fight against covid19 among other diseases but as we mentioned in that video one of the main problems the folding project is having right now is not necessarily that there's not enough willing volunteers to contribute their compute power but that there's not enough servers to intake all the data so we said we were going to work with the folding guys to build ourselves a folding at home target server and put it in our server room that's what i'm going to be doing here at home today and i've got one of our old decommissioned servers along with a care package from Jake that i'm going to be opening up using to upgrading it and then lysoling the heck out of this thing and sending it back to him and the video is brought to you by glasswire instantly see your current and past network activity detect malware and block badly behaving apps on your pc or Android device with glasswire use offer code Linus to get 25 off glass wire at the link in the video description so it's pretty obvious what all the volunteer contributors like you are doing for the folding at home project but many of you probably won't know what role the server plays in all this so here's the thing when you're folding your machine says hey i need a job to work on what the server does is it says all right i've got a spot available go ahead and connect to me i'm going to generate a job for you send it off to you then you go ahead you crunch those numbers and the job is ready to submit and the server says okay i'm ready for you you send it back and it stores it until the researchers who are working on the folding at home project are ready to grab it and do something with it so for that to work you actually need a few things one is a decent amount of CPU power folding at home recommends an eight core CPU which we actually do have but uh for a couple of reasons i'll get into later we're gonna be upgrading the one in here it's a little pinner they also recommend about 64 gigabytes of RAM that's gonna let us handle anywhere from a thousand to twelve hundred clients connected to our machine the other things we're gonna need are a buttload of storage in order to hold those completed jobs and of course a fast internet connection in order to connect to all those clients so because we've got 10 gigabit Ethernet already built into this motherboard we don't need to make any upgrades there but we do need to swap out the 32 gigs of RAM we've got in here that eight core CPU and of course we're going to need to add some storage let's start with our CPU the xeon e3 2618 lv3 that we've got in here was just fine for basic file server duties especially given that we were running a raid card in here when we originally deployed it so the CPU didn't even have to handle any storage parity calculations or anything like that but it's only got a 2.3 gigahertz base clock which means that if we're actually loaded up with you know somewhere over a thousand clients that thing is going to be running darn near base speeds not to mention that folding at home recommends a gigabit internet connection and we are actually going to be running 10 gig and we're hoping to provision off anywhere between about 4 gigabit and 5 gigabit of our internet connection to run to this machine so we want to see this thing melt if we can get away with it let's pull out this chip and now seems like as good a time as any to open up my care package from jakku what do we got in here this is looking like pretty good stuff here's our two power cables for the redundant power supply that's already built into this machine a vga cable so we can test it make sure it actually powers on after our upgrade oh shoot these RAM sticks got kicked around a little bit 32 gig registered ecc memory modules so that's going to give us a total of 128 gigs of RAM in our completed config i will explain why we need so much a little bit later i've got some thermal paste some screws and this seems like there's more than just an optane SSD in it ah there it is all right so we are upgrading to an e5 2697 v3 that is a high performance 14 core 28 thread processor or at least it was a few years back nowadays it's not really anything special but it should be more than enough for what we're doing here today go ahead get that installed it's so nice he included a little cleaning pad but he also sent me a clean CPU so i'm not gonna bother with that i love these thick boy thermal paste tubes look at that it's a hand for scale i mean it's a small hand but you know how it is oh wow that's a lot uh uh well the good news is we're not likely to run out anytime soon this is a very early unit of this motherboard super micro actually sent us i think it was one of the first ones off the line for this thing because it was one of their first matx server boards that had 10 gig Ethernet built in 10 gig used to be more of like uh well you why wouldn't you just use an add-in card for that and it's become over the last few years an option to have just pre-built into the motherboard and our CPU upgrade is done now let's talk about the storage folding at home recommends about 50 terabytes of storage for one of these servers but because our network connection and our CPU not to mention our memory are all beefed up we could quite possibly need more than that so we're configuring ours with eight of these 12 terabyte ironwolf pro nas drives from seagate that's going to give us about 96 terabytes of raw space or 72 once we give up two of our drives for parity and then once we've formatted about 60 terabytes of usable storage fortunately this case is super easy to install drives in it's actually one of the only things that i really really like about it you just pop these open slide them in no tools no sleds no nothing just boom just like that that takes care of our capacity but these are mechanical drives and even in zfs raid they're not going to be particularly fast so believe it or not that's where our overkill memory upgrade comes in so we're going to be using 64 gigs of our RAM to meet the recommended specs from folding at home but we're going to be using the other 64 gigs to actually act as an arc or a or a cache for our mechanical drives these are going to be running in a quad channel configuration so we should have plenty of bandwidth and uh i don't know whatever i haven't shouted out kingston for sending these to us in a long time they actually sent them originally for the six workstations one CPU project but we have used them for tons of things and they've worked basically anything we've put them in so good on them for that you guys probably noticed though that inside that optane box there was also an actual Intel optane 905p 960 gig SSD this is gonna act as a level 2 arc so between our RAM and this SSD we are hoping to accelerate our mechanical storage array quite significantly alleviating any performance bottlenecks that we could run into there where's my screwdriver so we're just going to take one of these open pci express gen 3 8x slots i was going to put it in the next slot over but i realized that puts it right up against the edge of the chassis and there's no cooling fan there so probably better off sitting next to the hba and having a cooling fan near it as opposed to being farther away from another heat source but not having any direct cooling one thing we're not changing is this hba the difference between an hba and a raid card is that an hba doesn't have a CPU built into it to handle raid calculations it's just a card that adds more ports to your motherboard in this case eight 12 gigabit per second sas ports even though we're only running them at SATA six gigabit per second through this back plane over here that's it then for the hardware upgrades and it all comes down to software configuration to decide how best to use it so one of the things we could do is instead of using 64 gigs of our memory to accelerate our hard drives we might cordon off just 32 gigs for that giving ourselves the capability to handle over 1500 clients that's something that those extra CPU cores might come in handy for or we might realize that we're better off using our extra CPU horsepower to enable real-time data compression to our hard drives like maybe we run into a network limitation or some other system limitation and realize we just don't have enough storage guys that's something that you can do with cfs for now though all that remains close this puppy up there we go screw it together and give the whole thing a good lysol wipe down i like Jake he's a good guy he can have the uh lemon scented wipes this is gonna be like the best smelling server ever hopefully matthias is here to pick it up and we'll send it back to the office okay let me open this okay so i don't actually have the rails for this server so we're just going to sit it on top of this storinator it's kind of ghetto but whatever i also don't have a screwdriver so i'm not going to look at what Linus did but i kind of know what it is already and for a little server she sure is heavy there we go okay hey he gave me the cables back noise he stole the thermal face though what the heck okay we got power plug one in for ipmi so we can remote control this thing uh oh right i need a display do we even have spaces left on this thing sweet okay i gotta try and find this back here so this is p where do these cables even come from should probably label this right yeah i'll leave that for the next person hey it's working let's make sure everything's here 110 28 gigs of RAM everything else it's working yes like Linus mentioned earlier we're going to be using something called zfs for our hard drive array so not only does it do the work of a file system by controlling how your data is stored and retrieved but it also handles how all the physical or virtual disks are partitioned and works to ensure the integrity of the data on said drives now that we've installed zfs and verified that it's actually working we're going to create our z pool now we're going to be using something called raid z2 which is actually based on raid 5 but it's going to allow us to lose two total drives before we lose any data i've already kind of pre-configured the command written out all the disk name so we don't have to wait for that and i've also defined our cache in here which means we're going to have that 905p working right off the bat so just copy this command in there paste enter and yeah okay let's check our z pool list now that it's should be in there we see 87 terabytes which is a little lower than 96 but you know drive partitioning and what have you and then if we run dfh we can see we actually end up with 60 terabytes of usable space that's a little bit down from 96 but it should be plenty for what we're trying to do there's still a couple more things we need to check off the list before we can install our folding software namely i want to set our arc cache to a max of 64 gigs and then i'm going to enable compression because i think it's going to be fine with the 14 cores we have i think realistically our network is going to be our bottleneck but we'll enable it for now and if down the road we have to turn it off no big deal now as a sort of last step i'm just going to kind of look at our z pool so we're going to do z pool status folding and we can see all eight of our drives one two three four five six seven eight and our cash drive are all there and ready to go and we have no errors which is perfect in order for folders to actually be able to access our folding server i've created some firewall rules now that was pretty easy but we're also going to create something called a traffic limiter because we don't want this folding server to eat up all of our 10 gigs of bandwidth so we're going to limit it to 5 gigabit but i'm going to show you guys how this works on the laptop as a baseline i'm going to run a speed test on the system before we enable the traffic limiter so you can actually see the difference okay we have our results in and we're looking at 230 down and 200 up so i think i'm going to limit it to 5 megabit which will be a very stark change and you should be able to see it right away so we're going to go into our two rules here it's already set to megabit so we'll do 5 and then we'll go to our in set that to five and then apply and then we just need to find our ip which i already did so 10.20.3.35 for this laptop so i'll just go add the rule in so here's our results and yeah five megabit right on the dot ish so now we're gonna change this over to five gigabit and then set it up on our actual server now that we've got our z pool configured and working we've got our bandwidth limiter and our firewall rules set up it's just a matter of calling the folding guys and getting them to install the software so we can get this work server going well it's been a week and as you can see my isolation facial hair is quite long and this is great i've got jakku on the line and we're going to be checking out how well our folding server is running so Jake give me the good news well the good news is there's enough work units for most people now which is great did we help with that or was that just serendipitous yes but it's not only us helping so they've also got microsoft azure i think they donated some servers um oracle you know the guys that make java and lots of other things they're doing stuff too and then there's us but uh if we look on the stats here we can definitely see that our server does have jobs available which means work units are available nice 19 000 jobs i mean for the first like five days it was up there was basically nothing because they just couldn't create enough projects it was no longer the servers but the people that were the ben or the bottleneck so how we doing now how much usage have we got i was kind of expecting more to be honest but if we look at the past history you can see like they spike up quite a bit so like right there is half a gigabit and then that's 600 and then if we scroll out actually so that's like 1.6 gigabit wow so from my understanding these actual like spikes a lot of the time are actually when the different work servers are like communicating with each other the actual like network bandwidth of individual clients connecting isn't too much but it's when they're trying to talk together that it really starts to hurt and how much storage have we used so far last i checked this morning it was already like two and a half or three terabytes of storage used that's awesome and we can look at our zfs stats here which actually paints a little bit more of a picture so our RAM cache i set to 64 gigs is completely full you can see it's it's ramping up slowly filling so as things are being removed from the RAM cache when they would normally get dumped back to the disks they actually go to the level two cache instead so this is already up to 64 or 40 gigs and it's only been like an hour or something since i reset it so it does take a few days to fill obviously because it's only filling up with stuff that's being removed from the primary cash but um fantastic and hey i've actually got an update for you so we have created an exclusive folding at home shirt design and all profits from the shirts are going to be contributed to causes within canada around the research relief and treatment of covet 19. so you guys are going to have to check that out at lttstore.com and what's really cool is Intel is going to match every dollar raised dollar for dollar to a maximum of 40 grand and so that 40 grand is actually going to go directly to the folding project so they can get servers and stuff right yes that's correct oh apparently you already knew about this i did already know yes this is actually like one of my favorite shirt designs and the qr code on the back takes you to our our team stats on extreme overclocking dot so join the cause guys we're gonna leave this server running for the foreseeable future i'll let you go now Jake and uh just share a word from our sponsors with the good viewers out there drop.com the cost gmr 54x iso gaming headset is the featured product this time around these are audiophile approved and based on a popular cost headset that was custom engineered for immersive 3d sound giving you positional cues for where your enemies are coming from there are some changes made from the original so they've reduced the tension in the lightweight headband for improved comfort they include a cord splitter inline microphone with remote and a detachable boom mic and the boom mic works with the ps4 xbox nintendo switch and more without hassle get it today and new users who sign up on drop.com will get 20 off on this headset we're gonna have it linked in the video description so thanks for watching guys if you enjoyed this video maybe check out our previous part of this video where we built a monster client folding machine as opposed to a server that's some really good stuff too"}