{"video_id":"gSrnXgAmK8k","title":"All of our data is GONE!","channel":"Linus Tech Tips","show":"Linus Tech Tips","published_at":"2016-05-06T14:53:45Z","duration_s":1378,"segments":[{"start_s":0.0,"end_s":1.5,"text":"Well, hello YouTube","speaker":null,"is_sponsor":0},{"start_s":1.5,"end_s":8.42,"text":"We have a bit of a problem today on this section of the moving, but we've been moved in for months moving vlog","speaker":null,"is_sponsor":0},{"start_s":9.96,"end_s":11.4,"text":"Something is going on with the server","speaker":null,"is_sponsor":0},{"start_s":11.4,"end_s":17.4,"text":"and we haven't been able to work on a project for more than like 30 minutes at a time before the server crashes and","speaker":null,"is_sponsor":0},{"start_s":17.48,"end_s":22.44,"text":"Our computers all freeze and then we're all sad because we can't work on them. Most of the sh** we do is on the server","speaker":null,"is_sponsor":0},{"start_s":22.88,"end_s":26.52,"text":"Well, I'm doing I'm actually doing some writing right now because I can do that","speaker":null,"is_sponsor":0},{"start_s":26.52,"end_s":30.98,"text":"I'd love to be working on my next video. Anything that doesn't need the server we can do now","speaker":null,"is_sponsor":0},{"start_s":30.98,"end_s":34.26,"text":"No, that's not much, but it always seems to go down","speaker":null,"is_sponsor":0},{"start_s":34.26,"end_s":40.56,"text":"Yeah, at the worst when there's nothing else to do. No kidding, but you know YOLO you only","speaker":null,"is_sponsor":0},{"start_s":41.42,"end_s":43.71,"text":"Liao once","speaker":null,"is_sponsor":0},{"start_s":45.27,"end_s":48.03,"text":"See let's see the stress that Linus is going through right now","speaker":null,"is_sponsor":0},{"start_s":48.03,"end_s":53.51,"text":"I'm only getting about 25 minutes per second, 30 minutes per second","speaker":null,"is_sponsor":0},{"start_s":53.51,"end_s":56.89,"text":"It's probably gonna crash like while you're doing this","speaker":null,"is_sponsor":0},{"start_s":57.99,"end_s":59.07,"text":"Yes","speaker":null,"is_sponsor":0},{"start_s":59.07,"end_s":61.39,"text":"How do we fix servers? How do you fix it?","speaker":null,"is_sponsor":0},{"start_s":62.05,"end_s":64.47,"text":"Tell Linus he will fix it","speaker":null,"is_sponsor":0},{"start_s":65.29,"end_s":67.29,"text":"That's your go-to. Yeah","speaker":null,"is_sponsor":0},{"start_s":67.81,"end_s":77.26,"text":"Whether you're running a small business or a","speaker":null,"is_sponsor":0},{"start_s":77.6,"end_s":84.22,"text":"One billion dollar or even multi-billion dollar enterprise Rackspace has your dedicated storage environments covered","speaker":null,"is_sponsor":0},{"start_s":84.3,"end_s":87.28,"text":"Check out the link in the video description to learn more","speaker":null,"is_sponsor":0},{"start_s":87.8,"end_s":88.61,"text":"All right","speaker":null,"is_sponsor":0},{"start_s":88.61,"end_s":89.99,"text":"So here's the situation","speaker":null,"is_sponsor":0},{"start_s":89.99,"end_s":95.17,"text":"Wonix server over the course of the last couple of days has been spontaneously going offline","speaker":null,"is_sponsor":0},{"start_s":95.67,"end_s":101.49,"text":"And turning off in spite of my best efforts to turn it back on and","speaker":null,"is_sponsor":0},{"start_s":101.87,"end_s":106.99,"text":"Complete the backup that I'm in the middle of trying to do to our new unraid vault","speaker":null,"is_sponsor":0},{"start_s":107.78,"end_s":109.78,"text":"While I was standing in front of it. I","speaker":null,"is_sponsor":0},{"start_s":110.96,"end_s":113.11,"text":"observed","speaker":null,"is_sponsor":0},{"start_s":113.11,"end_s":115.83,"text":"One of my raid controllers giving up the ghost","speaker":null,"is_sponsor":0},{"start_s":116.05,"end_s":120.21,"text":"So as a reminder this server is running three raid fives","speaker":null,"is_sponsor":0},{"start_s":120.67,"end_s":124.45,"text":"Striped together in Windows of a total of 24 SSDs","speaker":null,"is_sponsor":0},{"start_s":124.57,"end_s":127.01,"text":"So if one of the raid fives drops out in time","speaker":null,"is_sponsor":0},{"start_s":127.11,"end_s":134.11,"text":"Entirely all the data is gone only about 10% of the backup that I was just in the middle of","speaker":null,"is_sponsor":0},{"start_s":134.73,"end_s":136.39,"text":"remaking is","speaker":null,"is_sponsor":0},{"start_s":136.39,"end_s":138.52,"text":"saved","speaker":null,"is_sponsor":0},{"start_s":138.52,"end_s":139.87,"text":"so","speaker":null,"is_sponsor":0},{"start_s":139.87,"end_s":141.87,"text":"It's time to","speaker":null,"is_sponsor":0},{"start_s":142.61,"end_s":146.2,"text":"Investigate. All right. So here's wonix server","speaker":null,"is_sponsor":0},{"start_s":146.92,"end_s":153.74,"text":"So far troubleshooting steps that I've tried you never want to you never want to rebuild something like this if there's a possibility","speaker":null,"is_sponsor":0},{"start_s":153.86,"end_s":158.92,"text":"You're gonna have to go to a data recovery service because that can make it much much worse","speaker":null,"is_sponsor":0},{"start_s":158.92,"end_s":159.5,"text":"so","speaker":null,"is_sponsor":0},{"start_s":159.5,"end_s":166.3,"text":"In terms of troubleshooting steps, I tried transplanting it into another case so that I could use a different","speaker":null,"is_sponsor":0},{"start_s":167.8,"end_s":173.32,"text":"SATA backplane that didn't work. I tried a different power supply a better power supply","speaker":null,"is_sponsor":0},{"start_s":174.04,"end_s":175.04,"text":"That didn't work","speaker":null,"is_sponsor":0},{"start_s":175.04,"end_s":178.14,"text":"None of that fixed it and I even went as far as to put in","speaker":null,"is_sponsor":0},{"start_s":178.48,"end_s":182.7,"text":"Another LSI raid card that I had and try to import the array","speaker":null,"is_sponsor":0},{"start_s":182.74,"end_s":188.88,"text":"But while it did detect my drives all of them as unconfigured good and a foreign array","speaker":null,"is_sponsor":0},{"start_s":188.88,"end_s":191.48,"text":"It was not able to import it","speaker":null,"is_sponsor":0},{"start_s":191.68,"end_s":197.72,"text":"So there is some good news there though while I'm getting firmware errors kicked out by this card right here when I try to boot","speaker":null,"is_sponsor":0},{"start_s":198.12,"end_s":201.64,"text":"All the drives are detected by another card","speaker":null,"is_sponsor":0},{"start_s":201.64,"end_s":206.06,"text":"So hopefully it didn't write too much garbled data to the array","speaker":null,"is_sponsor":0},{"start_s":209.08,"end_s":211.98,"text":"Okay, so this is basically what I've been doing for the last","speaker":null,"is_sponsor":0},{"start_s":212.8,"end_s":215.86,"text":"14 hours is sending emails contacting different","speaker":null,"is_sponsor":0},{"start_s":216.64,"end_s":218.64,"text":"Data recovery services","speaker":null,"is_sponsor":0},{"start_s":219.14,"end_s":220.58,"text":"There's some that are local","speaker":null,"is_sponsor":0},{"start_s":220.58,"end_s":226.34,"text":"there's some that are not local, but I'm not ruling those ones out. In fact, right","speaker":null,"is_sponsor":0},{"start_s":226.34,"end_s":232.58,"text":"now the most likely solution looks like WeRecoverData.com. They've got some","speaker":null,"is_sponsor":0},{"start_s":232.58,"end_s":238.2,"text":"custom tools that they've created that will allow them to SSH into the system","speaker":null,"is_sponsor":0},{"start_s":238.2,"end_s":245.83,"text":"and actually potentially recover the missing RAID 5 and then the","speaker":null,"is_sponsor":0},{"start_s":245.83,"end_s":251.93,"text":"other two RAID 5s and assemble them all together and export them as one","speaker":null,"is_sponsor":0},{"start_s":251.93,"end_s":257.53,"text":"gigantic pile of data to one of our servers, all without me actually having","speaker":null,"is_sponsor":0},{"start_s":257.53,"end_s":262.6,"text":"to send the drives away, which would be pretty ideal. Now obviously an approach","speaker":null,"is_sponsor":0},{"start_s":262.6,"end_s":266.5,"text":"like that would not work in the case of physical damage to the drives, but","speaker":null,"is_sponsor":0},{"start_s":266.5,"end_s":272.24,"text":"because it is only a RAID controller issue, they're saying, hey, there's a shot","speaker":null,"is_sponsor":0},{"start_s":272.24,"end_s":274.72,"text":"at this, so let's give it a try.","speaker":null,"is_sponsor":0},{"start_s":274.72,"end_s":276.0,"text":"So I just","speaker":null,"is_sponsor":0},{"start_s":276.0,"end_s":284.14,"text":"downloaded their remote recovery client thing here. So right now I'm waiting for","speaker":null,"is_sponsor":0},{"start_s":284.14,"end_s":291.76,"text":"their custom Linux-based tool to load on a bootable external SSD so that they can","speaker":null,"is_sponsor":0},{"start_s":291.76,"end_s":300.03,"text":"SSH into the server. I'm not having a good couple days. Right now I can't get","speaker":null,"is_sponsor":0},{"start_s":300.03,"end_s":310.69,"text":"the utility to detect my USB drive. It's called tech support. Oh bloody hell, I just","speaker":null,"is_sponsor":0},{"start_s":310.69,"end_s":311.29,"text":"figured it out.","speaker":null,"is_sponsor":0},{"start_s":311.31,"end_s":316.37,"text":"This is Linus calling, by the way. I just realized I have to run the WRD disk","speaker":null,"is_sponsor":0},{"start_s":316.37,"end_s":325.18,"text":"image as administrator. You guys should put that in the documentation. It never","speaker":null,"is_sponsor":0},{"start_s":325.18,"end_s":329.24,"text":"fails. This is why it's a good idea to call tech support, because the second you","speaker":null,"is_sponsor":0},{"start_s":329.24,"end_s":332.18,"text":"call them, you'll solve the problem on your own.","speaker":null,"is_sponsor":0},{"start_s":332.18,"end_s":345.52,"text":"Alright, so in theory we now have a bootable drive. Okay, so step one is","speaker":null,"is_sponsor":0},{"start_s":345.52,"end_s":353.34,"text":"getting this unnecessary extraneous server out of the way. I'm grabbing a UPS to use.","speaker":null,"is_sponsor":0},{"start_s":353.5,"end_s":363.26,"text":"Now, in a perfect world, I would have a motherboard or an HBA card, a host bus","speaker":null,"is_sponsor":0},{"start_s":363.26,"end_s":369.14,"text":"adapter card, that could allow me to plug in all 24 of the drives for them to do","speaker":null,"is_sponsor":0},{"start_s":369.14,"end_s":374.4,"text":"their data recovery magic on. But what we're gonna have to settle for is having","speaker":null,"is_sponsor":0},{"start_s":374.4,"end_s":379.06,"text":"eight of them plugged into the motherboard at a time as each RAID 5 is","speaker":null,"is_sponsor":0},{"start_s":379.06,"end_s":384.64,"text":"rebuilt. RAID cards do all kinds of funny, funny nonsense and they don't give other","speaker":null,"is_sponsor":0},{"start_s":384.64,"end_s":385.26,"text":"software","speaker":null,"is_sponsor":0},{"start_s":385.26,"end_s":390.96,"text":"direct access to the drives at a bit level in some cases. So we need to remove","speaker":null,"is_sponsor":0},{"start_s":390.96,"end_s":398.27,"text":"all these drives and plug them in directly to the motherboard. Okay, now it","speaker":null,"is_sponsor":0},{"start_s":398.27,"end_s":403.22,"text":"should boot to that, in theory. Let's just make sure all the drives are detected in","speaker":null,"is_sponsor":0},{"start_s":403.22,"end_s":407.88,"text":"the BIOS and everything. Oh, that's a problem. I need to pull out the other","speaker":null,"is_sponsor":0},{"start_s":407.88,"end_s":410.56,"text":"RAID controllers as well, otherwise they're gonna be looking for their","speaker":null,"is_sponsor":0},{"start_s":410.56,"end_s":417.36,"text":"drives and I don't have them powered up right now. Okay. Oh, actually that doesn't","speaker":null,"is_sponsor":0},{"start_s":417.36,"end_s":419.5,"text":"look very good.","speaker":null,"is_sponsor":0},{"start_s":419.7,"end_s":434.58,"text":"DCI-E port errors. What? So let's make sure all of our drives are even showing","speaker":null,"is_sponsor":0},{"start_s":434.58,"end_s":452.22,"text":"up. Uh-oh. That is extremely worrying. Oh, that is extremely worrying. Oh, that's not","speaker":null,"is_sponsor":0},{"start_s":452.22,"end_s":458.14,"text":"good. That means potentially three of the drives in a RAID 5 are dead, which means","speaker":null,"is_sponsor":0},{"start_s":458.14,"end_s":460.77,"text":"this data is not coming back. None of it.","speaker":null,"is_sponsor":0},{"start_s":460.77,"end_s":464.84,"text":"Not this particular RAID?","speaker":null,"is_sponsor":0},{"start_s":464.84,"end_s":466.16,"text":"No, none of it.","speaker":null,"is_sponsor":0},{"start_s":466.16,"end_s":466.66,"text":"None of the...","speaker":null,"is_sponsor":0},{"start_s":468.1,"end_s":469.1,"text":"But I thought we have three RAID 5s.","speaker":null,"is_sponsor":0},{"start_s":469.1,"end_s":480.35,"text":"None of it will be coming back if we lost three drives from one of the RAID 5s.","speaker":null,"is_sponsor":0},{"start_s":480.35,"end_s":486.07,"text":"So let's try moving things around. I've never tested the SATA ports on this board.","speaker":null,"is_sponsor":0},{"start_s":486.07,"end_s":491.09,"text":"It's possible three of them are dead, however extraordinarily unlikely that","speaker":null,"is_sponsor":0},{"start_s":491.09,"end_s":501.39,"text":"would be. Five, six. Okay, we've got six now. Okay, that gives some hope. I'm just","speaker":null,"is_sponsor":0},{"start_s":501.39,"end_s":506.21,"text":"moving SATA ports around. I was just gonna say right now I'm even having difficulty","speaker":null,"is_sponsor":0},{"start_s":506.55,"end_s":514.2,"text":"in this system to post. So I'm at a whole new level of concern. Yeah, I'll let you","speaker":null,"is_sponsor":0},{"start_s":514.2,"end_s":522.03,"text":"know. I'll send an email. Okay. So I'm at the point now where I don't want to put the","speaker":null,"is_sponsor":0},{"start_s":522.03,"end_s":528.43,"text":"drives on this system. Let's go with this one. I didn't like all those PCI Express errors","speaker":null,"is_sponsor":0},{"start_s":528.43,"end_s":533.95,"text":"that the bootable USB was kicking out. I don't like that some of these drives aren't showing","speaker":null,"is_sponsor":0},{"start_s":533.95,"end_s":538.94,"text":"up. I'm starting to wonder if that's what corrupted the LSI card. This royal pain in","speaker":null,"is_sponsor":0},{"start_s":538.94,"end_s":539.94,"text":"the ass.","speaker":null,"is_sponsor":0},{"start_s":539.94,"end_s":546.73,"text":"A thousand things I can think of that I would rather be doing right now. Okay, I got seven","speaker":null,"is_sponsor":0},{"start_s":546.73,"end_s":553.69,"text":"right now. Really? Yeah. Wow, maybe it was bad PCIe. Linus speaking. Well, I can tell","speaker":null,"is_sponsor":0},{"start_s":553.69,"end_s":558.71,"text":"you what's happening so far is I put it on the system that I was running the RAID card","speaker":null,"is_sponsor":0},{"start_s":558.71,"end_s":566.75,"text":"and the drives in and that baby started spitting out PCIe errors all over the place. So I got","speaker":null,"is_sponsor":0},{"start_s":566.75,"end_s":572.17,"text":"everything pulled out of that system and I have plugged all the drives now into a different","speaker":null,"is_sponsor":0},{"start_s":572.37,"end_s":574.95,"text":"disk bench and I'm trying to see because I wasn't even getting all the drives showing","speaker":null,"is_sponsor":0},{"start_s":574.95,"end_s":580.12,"text":"up. Two, three, four, five, six, seven, eight. Okay. Hey, you guys are on the phone for the","speaker":null,"is_sponsor":0},{"start_s":580.12,"end_s":584.1,"text":"magic moment where all eight drives are detected by a new motherboard. So I'm beginning to","speaker":null,"is_sponsor":0},{"start_s":584.1,"end_s":588.1,"text":"think there may be a motherboard issue that caused the firmware on that LSI controller","speaker":null,"is_sponsor":0},{"start_s":588.1,"end_s":592.3,"text":"to corrupt in the first place. So I'm actually pressing enter and I am booting up to your","speaker":null,"is_sponsor":0},{"start_s":592.3,"end_s":601.02,"text":"USB and it is loading correctly on this. I think that board is dead. Okay. Well, that's","speaker":null,"is_sponsor":0},{"start_s":601.02,"end_s":602.64,"text":"I don't know if that makes it worse.","speaker":null,"is_sponsor":0},{"start_s":603.46,"end_s":608.3,"text":"Probably does, doesn't it? You can tell me. I can take it. It says waiting up to 60 more","speaker":null,"is_sponsor":0},{"start_s":608.3,"end_s":614.52,"text":"seconds for network configuration. Good timing with the follow-up call. If you called one","speaker":null,"is_sponsor":0},{"start_s":614.52,"end_s":617.88,"text":"minute earlier, I've been cursing you for dialing my number. So I have to answer my","speaker":null,"is_sponsor":0},{"start_s":617.88,"end_s":621.96,"text":"phone because I was in the middle of thinking only five of my drives were being detected.","speaker":null,"is_sponsor":0},{"start_s":621.96,"end_s":625.66,"text":"You guys know what you're doing, right? I had a local shop that I was going to physically","speaker":null,"is_sponsor":0},{"start_s":625.66,"end_s":630.18,"text":"take it to and they basically, they were like, they kind of freaked out when I said I was","speaker":null,"is_sponsor":0},{"start_s":630.18,"end_s":633.02,"text":"having someone use their remote tool. They were like, well, unless these guys really","speaker":null,"is_sponsor":0},{"start_s":633.02,"end_s":633.44,"text":"know what they're doing.","speaker":null,"is_sponsor":0},{"start_s":633.46,"end_s":636.94,"text":"That's pretty dangerous. And I was like, okay, well I'll, I'll check with them. I'll make","speaker":null,"is_sponsor":0},{"start_s":636.94,"end_s":641.68,"text":"sure they, they know what they're doing. So, so you guys know what you're doing. So basically","speaker":null,"is_sponsor":0},{"start_s":641.68,"end_s":646.6,"text":"they're explaining that they're going to virtually Mount the volumes. So we're not doing any","speaker":null,"is_sponsor":0},{"start_s":646.6,"end_s":653.9,"text":"writing to my discs. Um, so even in the event that we cannot recover it by this means, uh,","speaker":null,"is_sponsor":0},{"start_s":653.9,"end_s":665.39,"text":"that still leaves other options open to us. Okay. They're in. Okay. Well, that's, that's","speaker":null,"is_sponsor":0},{"start_s":665.39,"end_s":669.93,"text":"very wonderful to hear, but I'm trying not to get too excited right now. They sound pretty","speaker":null,"is_sponsor":0},{"start_s":669.93,"end_s":675.09,"text":"confident. This is like the ultimate roller coaster right now. So to put this in the appropriate","speaker":null,"is_sponsor":0},{"start_s":675.09,"end_s":683.01,"text":"context for the viewers right now, one X server has several in progress projects stored on","speaker":null,"is_sponsor":0},{"start_s":683.01,"end_s":687.55,"text":"it that do not have a backup anywhere else, including a fully shot channel, super fun","speaker":null,"is_sponsor":0},{"start_s":687.55,"end_s":693.41,"text":"that involved a thousand dollars in equipment rental, um, including, uh, lots of footage","speaker":null,"is_sponsor":0},{"start_s":693.41,"end_s":699.93,"text":"from Linus tech tips videos. Like it is going to be a huge, huge, devastating loss for us.","speaker":null,"is_sponsor":0},{"start_s":699.93,"end_s":705.45,"text":"To lose this. I mean, it's got a lot of our, our templates for editing, uh, editing videos","speaker":null,"is_sponsor":0},{"start_s":705.45,"end_s":710.03,"text":"for, for scripting templates for employee reviews, like all kinds of stuff. Like this","speaker":null,"is_sponsor":0},{"start_s":710.03,"end_s":715.55,"text":"is our main working server. No, the offsite backup server has not ever been built yet.","speaker":null,"is_sponsor":0},{"start_s":715.55,"end_s":720.51,"text":"The vault was the backup server. If I ever have to like act in a, in a video where I","speaker":null,"is_sponsor":0},{"start_s":720.51,"end_s":724.89,"text":"have to look for Lauren, I'm going to think back to this moment. Okay. So I do need to","speaker":null,"is_sponsor":0},{"start_s":724.89,"end_s":729.17,"text":"reboot it once in order to add that eight terabyte drive. So anyway, right. Okay. Thank","speaker":null,"is_sponsor":0},{"start_s":729.17,"end_s":729.91,"text":"you very much, gentlemen.","speaker":null,"is_sponsor":0},{"start_s":729.93,"end_s":738.9,"text":"I hope to speak to you, uh, with good news very soon, 12 SATA connectors. Ah, ah, ah.","speaker":null,"is_sponsor":0},{"start_s":738.9,"end_s":746.14,"text":"Okay. So let's make sure everything is showing up 1, 2, 3, 4, 5, 6, 7, 8 of those are boot","speaker":null,"is_sponsor":0},{"start_s":746.14,"end_s":754.7,"text":"device and our eight terabyte recovery media. We are ready to rock. Okay. So within about","speaker":null,"is_sponsor":0},{"start_s":754.7,"end_s":758.38,"text":"10 minutes, they contacted me back. They said, okay, yeah, we're ready to have all the drives","speaker":null,"is_sponsor":0},{"start_s":758.38,"end_s":764.03,"text":"in there. So that means we are going to have to do some janky ass.","speaker":null,"is_sponsor":0},{"start_s":764.09,"end_s":773.67,"text":"Right now. So basically what you're looking at here is these drives are going to be powered","speaker":null,"is_sponsor":0},{"start_s":773.67,"end_s":779.31,"text":"off this power supply with a jumper. These cards and all these drives are going to be","speaker":null,"is_sponsor":0},{"start_s":779.31,"end_s":783.84,"text":"powered off of this rig. So now we can power all those drives without powering up this","speaker":null,"is_sponsor":0},{"start_s":783.84,"end_s":785.5,"text":"motherboard or any of that nonsense.","speaker":null,"is_sponsor":0},{"start_s":785.5,"end_s":793.22,"text":"Okay? So we have a grand total of 25 drives connected to this test bench. 16 of which","speaker":null,"is_sponsor":0},{"start_s":793.22,"end_s":794.58,"text":"are running in Bay's.","speaker":null,"is_sponsor":0},{"start_s":794.58,"end_s":799.58,"text":"of this enclosure and eight of which are spread out here.","speaker":null,"is_sponsor":0},{"start_s":801.34,"end_s":802.48,"text":"So there we go.","speaker":null,"is_sponsor":0},{"start_s":802.48,"end_s":804.06,"text":"There's one of my LSI controllers.","speaker":null,"is_sponsor":0},{"start_s":804.06,"end_s":806.3,"text":"There's its eight drives.","speaker":null,"is_sponsor":0},{"start_s":806.3,"end_s":808.04,"text":"There's my other one.","speaker":null,"is_sponsor":0},{"start_s":808.04,"end_s":814.92,"text":"Whoa, where are your virtual drives?","speaker":null,"is_sponsor":0},{"start_s":814.92,"end_s":817.02,"text":"I think I remember a troubleshooting step","speaker":null,"is_sponsor":0},{"start_s":817.02,"end_s":820.32,"text":"where I tried swapping all the drives to different bays","speaker":null,"is_sponsor":0},{"start_s":820.32,"end_s":822.12,"text":"where I was trying to see if it was the back plane","speaker":null,"is_sponsor":0},{"start_s":822.12,"end_s":823.0,"text":"or the cables.","speaker":null,"is_sponsor":0},{"start_s":823.86,"end_s":828.94,"text":"Wow, panic moment there, but it's okay.","speaker":null,"is_sponsor":0},{"start_s":828.94,"end_s":834.5,"text":"So now we should see two virtual drives here on the screen.","speaker":null,"is_sponsor":0},{"start_s":834.5,"end_s":836.56,"text":"That's one set.","speaker":null,"is_sponsor":0},{"start_s":836.56,"end_s":839.57,"text":"Show me the second set.","speaker":null,"is_sponsor":0},{"start_s":839.57,"end_s":846.91,"text":"Oh, I thought we lost another RAID 5 there.","speaker":null,"is_sponsor":0},{"start_s":846.91,"end_s":849.53,"text":"Okay, it's good.","speaker":null,"is_sponsor":0},{"start_s":849.53,"end_s":851.47,"text":"Feeling good.","speaker":null,"is_sponsor":0},{"start_s":851.47,"end_s":854.97,"text":"Reality TV ain't got nothing on IT work.","speaker":null,"is_sponsor":0},{"start_s":854.97,"end_s":857.57,"text":"Show me all the devices.","speaker":null,"is_sponsor":0},{"start_s":857.57,"end_s":859.73,"text":"Two RAID adapters, one eight terabyte drive,","speaker":null,"is_sponsor":0},{"start_s":859.73,"end_s":861.43,"text":"one, two, three, four, five, six, seven,","speaker":null,"is_sponsor":0},{"start_s":861.43,"end_s":863.74,"text":"eight Kingston drives.","speaker":null,"is_sponsor":0},{"start_s":863.74,"end_s":865.54,"text":"Boom.","speaker":null,"is_sponsor":0},{"start_s":865.54,"end_s":866.38,"text":"Okay.","speaker":null,"is_sponsor":0},{"start_s":866.38,"end_s":867.0,"text":"I will let...","speaker":null,"is_sponsor":0},{"start_s":867.0,"end_s":870.26,"text":"the data recovery specialists know.","speaker":null,"is_sponsor":0},{"start_s":870.26,"end_s":871.1,"text":"All right.","speaker":null,"is_sponsor":0},{"start_s":871.1,"end_s":872.34,"text":"So all that's left now then I think,","speaker":null,"is_sponsor":0},{"start_s":872.34,"end_s":875.28,"text":"I got in touch with them and I asked if we can put","speaker":null,"is_sponsor":0},{"start_s":875.28,"end_s":877.7,"text":"a 10 gigabit network card in","speaker":null,"is_sponsor":0},{"start_s":877.7,"end_s":881.06,"text":"so we can accelerate the offloading of all of our files.","speaker":null,"is_sponsor":0},{"start_s":881.06,"end_s":882.36,"text":"So all we're waiting for now","speaker":null,"is_sponsor":0},{"start_s":882.36,"end_s":884.22,"text":"is for them to reboot the machine","speaker":null,"is_sponsor":0},{"start_s":884.22,"end_s":887.4,"text":"and set me up with a network share that I can access","speaker":null,"is_sponsor":0},{"start_s":887.4,"end_s":890.9,"text":"and fingers crossed,","speaker":null,"is_sponsor":0},{"start_s":890.9,"end_s":892.72,"text":"just start pulling off all the footage","speaker":null,"is_sponsor":0},{"start_s":892.72,"end_s":894.71,"text":"and well, everything else.","speaker":null,"is_sponsor":0},{"start_s":894.71,"end_s":895.53,"text":"All right.","speaker":null,"is_sponsor":0},{"start_s":895.53,"end_s":896.95,"text":"So check this out here, around here.","speaker":null,"is_sponsor":0},{"start_s":896.95,"end_s":898.59,"text":"So I've got the instructions","speaker":null,"is_sponsor":0},{"start_s":898.59,"end_s":902.59,"text":"for how to access our data.","speaker":null,"is_sponsor":0},{"start_s":902.59,"end_s":908.05,"text":"So I go into the Tritium share","speaker":null,"is_sponsor":0},{"start_s":908.05,"end_s":909.99,"text":"and there it is.","speaker":null,"is_sponsor":0},{"start_s":909.99,"end_s":913.41,"text":"Bunch of EULAs, a bunch of stuff.","speaker":null,"is_sponsor":0},{"start_s":913.41,"end_s":918.41,"text":"We've got what looks like the file structure.","speaker":null,"is_sponsor":0},{"start_s":918.41,"end_s":920.11,"text":"Let's test it.","speaker":null,"is_sponsor":0},{"start_s":920.11,"end_s":922.41,"text":"How about one that I,","speaker":null,"is_sponsor":0},{"start_s":922.41,"end_s":924.57,"text":"that we haven't finished releasing yet?","speaker":null,"is_sponsor":0},{"start_s":924.57,"end_s":926.87,"text":"Like something from the, the melee battle here.","speaker":null,"is_sponsor":0},{"start_s":926.87,"end_s":929.21,"text":"Let's just, let's watch it.","speaker":null,"is_sponsor":0},{"start_s":929.81,"end_s":931.3,"text":"Hmm.","speaker":null,"is_sponsor":0},{"start_s":931.3,"end_s":936.86,"text":"Okay. XA footage for sure.","speaker":null,"is_sponsor":0},{"start_s":936.86,"end_s":948.42,"text":"Oh, okay.","speaker":null,"is_sponsor":0},{"start_s":948.42,"end_s":949.86,"text":"Here we can copy one of these.","speaker":null,"is_sponsor":0},{"start_s":949.86,"end_s":952.0,"text":"Let's copy one of these clips to my local machine","speaker":null,"is_sponsor":0},{"start_s":952.0,"end_s":954.87,"text":"and see if that helps.","speaker":null,"is_sponsor":0},{"start_s":954.87,"end_s":955.71,"text":"No.","speaker":null,"is_sponsor":0},{"start_s":955.71,"end_s":960.5,"text":"Oh boy.","speaker":null,"is_sponsor":0},{"start_s":960.5,"end_s":961.34,"text":"Let's have a look.","speaker":null,"is_sponsor":0},{"start_s":961.34,"end_s":963.22,"text":"I wonder if everything's even here.","speaker":null,"is_sponsor":0},{"start_s":963.22,"end_s":965.72,"text":"Like I don't even see the Linus tech tips folder.","speaker":null,"is_sponsor":0},{"start_s":965.72,"end_s":969.89,"text":"One and two are missing.","speaker":null,"is_sponsor":0},{"start_s":969.89,"end_s":972.05,"text":"Useful applications is empty.","speaker":null,"is_sponsor":0},{"start_s":972.05,"end_s":974.86,"text":"That should be full of things.","speaker":null,"is_sponsor":0},{"start_s":974.86,"end_s":980.51,"text":"Okay. Well, I guess we'd better get in touch with them.","speaker":null,"is_sponsor":0},{"start_s":980.51,"end_s":981.51,"text":"So thanks for watching.","speaker":null,"is_sponsor":0},{"start_s":981.51,"end_s":986.51,"text":"Subscribe, follow all that good stuff and see you next time.","speaker":null,"is_sponsor":0},{"start_s":986.51,"end_s":988.25,"text":"I just got an email back from them.","speaker":null,"is_sponsor":0},{"start_s":988.25,"end_s":991.67,"text":"Please stand by addressing your emails with the engineer.","speaker":null,"is_sponsor":0},{"start_s":991.67,"end_s":994.89,"text":"It's possible the RAID definitions were changed","speaker":null,"is_sponsor":0},{"start_s":994.89,"end_s":996.93,"text":"from one reboot to the next","speaker":null,"is_sponsor":0},{"start_s":996.93,"end_s":999.39,"text":"because of drive letter assignments in Linux.","speaker":null,"is_sponsor":0},{"start_s":999.39,"end_s":1001.23,"text":"It's 26, so it's 26 drive limit.","speaker":null,"is_sponsor":0},{"start_s":1001.23,"end_s":1005.31,"text":"We have, I think we have more than 26 devices in there.","speaker":null,"is_sponsor":0},{"start_s":1005.31,"end_s":1006.65,"text":"I got another email back.","speaker":null,"is_sponsor":0},{"start_s":1006.65,"end_s":1009.76,"text":"Linus, we recreated the block devices.","speaker":null,"is_sponsor":0},{"start_s":1009.76,"end_s":1010.64,"text":"Please go ahead and check the videos in context.","speaker":null,"is_sponsor":0},{"start_s":1010.64,"end_s":1012.84,"text":"That you were looking at again.","speaker":null,"is_sponsor":0},{"start_s":1012.84,"end_s":1016.12,"text":"No way.","speaker":null,"is_sponsor":0},{"start_s":1016.12,"end_s":1017.5,"text":"Oh, the suspense.","speaker":null,"is_sponsor":0},{"start_s":1019.36,"end_s":1021.97,"text":"I'm not enjoying this.","speaker":null,"is_sponsor":0},{"start_s":1021.97,"end_s":1022.81,"text":"Is nothing there?","speaker":null,"is_sponsor":0},{"start_s":1022.81,"end_s":1025.93,"text":"I'll let them know the chair's empty again.","speaker":null,"is_sponsor":0},{"start_s":1025.93,"end_s":1030.86,"text":"This is just like the wildest roller coaster ever.","speaker":null,"is_sponsor":0},{"start_s":1030.86,"end_s":1031.84,"text":"I don't even, yeah.","speaker":null,"is_sponsor":0},{"start_s":1031.84,"end_s":1034.59,"text":"I don't even have anything else to say actually.","speaker":null,"is_sponsor":0},{"start_s":1034.59,"end_s":1036.23,"text":"It's almost funny.","speaker":null,"is_sponsor":0},{"start_s":1036.23,"end_s":1039.52,"text":"I hope to some of you it's funny.","speaker":null,"is_sponsor":0},{"start_s":1039.52,"end_s":1041.4,"text":"Here's the email I got 17 minutes ago.","speaker":null,"is_sponsor":0},{"start_s":1041.4,"end_s":1042.56,"text":"I was stuck on a call.","speaker":null,"is_sponsor":0},{"start_s":1042.56,"end_s":1044.14,"text":"We got everything remounted.","speaker":null,"is_sponsor":0},{"start_s":1044.14,"end_s":1045.8,"text":"So you can now go ahead and view the data.","speaker":null,"is_sponsor":0},{"start_s":1045.8,"end_s":1046.96,"text":"Once more.","speaker":null,"is_sponsor":0},{"start_s":1046.96,"end_s":1050.99,"text":"This is the moment.","speaker":null,"is_sponsor":0},{"start_s":1050.99,"end_s":1052.23,"text":"Do we not have a two?","speaker":null,"is_sponsor":0},{"start_s":1052.23,"end_s":1053.99,"text":"I don't think we ever had a two.","speaker":null,"is_sponsor":0},{"start_s":1053.99,"end_s":1055.65,"text":"Oh, okay.","speaker":null,"is_sponsor":0},{"start_s":1058.29,"end_s":1061.13,"text":"Okay, so let's go to that Channel Super Fun video","speaker":null,"is_sponsor":0},{"start_s":1061.13,"end_s":1062.57,"text":"that we were looking at.","speaker":null,"is_sponsor":0},{"start_s":1062.57,"end_s":1064.01,"text":"Yes, yes, Swagway jousting.","speaker":null,"is_sponsor":0},{"start_s":1064.01,"end_s":1071.3,"text":"Okay, and it was the delivery.","speaker":null,"is_sponsor":0},{"start_s":1071.3,"end_s":1074.87,"text":"I'm getting like tingling.","speaker":null,"is_sponsor":0},{"start_s":1074.87,"end_s":1076.69,"text":"Okay, okay, you gotta check the source files though","speaker":null,"is_sponsor":0},{"start_s":1076.69,"end_s":1078.07,"text":"because that was what we know was corrupted.","speaker":null,"is_sponsor":0},{"start_s":1078.07,"end_s":1079.47,"text":"Okay, okay, okay, okay.","speaker":null,"is_sponsor":0},{"start_s":1079.47,"end_s":1081.97,"text":"So untranscoded footage.","speaker":null,"is_sponsor":0},{"start_s":1081.97,"end_s":1090.26,"text":"XA, they did it.","speaker":null,"is_sponsor":0},{"start_s":1090.26,"end_s":1093.03,"text":"It's back.","speaker":null,"is_sponsor":0},{"start_s":1093.03,"end_s":1095.09,"text":"My God.","speaker":null,"is_sponsor":0},{"start_s":1095.09,"end_s":1096.31,"text":"It's back, here's another file.","speaker":null,"is_sponsor":0},{"start_s":1096.31,"end_s":1097.15,"text":"It's back.","speaker":null,"is_sponsor":0},{"start_s":1097.15,"end_s":1100.64,"text":"Do we have everything?","speaker":null,"is_sponsor":0},{"start_s":1100.64,"end_s":1102.64,"text":"In theory, yes.","speaker":null,"is_sponsor":0},{"start_s":1102.64,"end_s":1104.42,"text":"Really?","speaker":null,"is_sponsor":0},{"start_s":1104.42,"end_s":1107.33,"text":"Holy .","speaker":null,"is_sponsor":0},{"start_s":1107.33,"end_s":1109.84,"text":"Oh.","speaker":null,"is_sponsor":0},{"start_s":1109.84,"end_s":1111.98,"text":"I think it's finally over.","speaker":null,"is_sponsor":0},{"start_s":1111.98,"end_s":1112.82,"text":"Woo hoo.","speaker":null,"is_sponsor":0},{"start_s":1114.91,"end_s":1117.9,"text":"That is fantastic.","speaker":null,"is_sponsor":0},{"start_s":1117.9,"end_s":1122.27,"text":"Hey guys, guess what happened with the recovery?","speaker":null,"is_sponsor":0},{"start_s":1122.27,"end_s":1123.45,"text":"We got everything back.","speaker":null,"is_sponsor":0},{"start_s":1123.45,"end_s":1125.45,"text":"Oh, thank goodness.","speaker":null,"is_sponsor":0},{"start_s":1125.45,"end_s":1126.29,"text":"Woo hoo.","speaker":null,"is_sponsor":0},{"start_s":1126.29,"end_s":1129.81,"text":"Like it's just been such a stressful few weeks","speaker":null,"is_sponsor":0},{"start_s":1129.81,"end_s":1133.6,"text":"and that was just such a huge, huge thing.","speaker":null,"is_sponsor":0},{"start_s":1133.6,"end_s":1138.04,"text":"All of our pitches and proposals and stuff are on that server.","speaker":null,"is_sponsor":0},{"start_s":1138.04,"end_s":1142.1,"text":"Okay, Wanik's back.","speaker":null,"is_sponsor":0},{"start_s":1142.1,"end_s":1142.94,"text":"Bully?","speaker":null,"is_sponsor":0},{"start_s":1142.94,"end_s":1143.78,"text":"Really?","speaker":null,"is_sponsor":0},{"start_s":1143.78,"end_s":1146.12,"text":"Everything is back as far as I can tell.","speaker":null,"is_sponsor":0},{"start_s":1146.12,"end_s":1148.9,"text":"I have played back video files.","speaker":null,"is_sponsor":0},{"start_s":1148.9,"end_s":1151.18,"text":"They are not corrupted.","speaker":null,"is_sponsor":0},{"start_s":1151.18,"end_s":1153.56,"text":"Wanik is officially back.","speaker":null,"is_sponsor":0},{"start_s":1153.56,"end_s":1157.68,"text":"Swagway jousting footage for the melee battle should be fine.","speaker":null,"is_sponsor":0},{"start_s":1157.68,"end_s":1161.32,"text":"And we'll have a better backup scheme in the future, I promise.","speaker":null,"is_sponsor":0},{"start_s":1161.32,"end_s":1162.44,"text":"That is all.","speaker":null,"is_sponsor":0},{"start_s":1162.44,"end_s":1163.44,"text":"Yay.","speaker":null,"is_sponsor":0},{"start_s":1163.44,"end_s":1164.44,"text":"Yay.","speaker":null,"is_sponsor":0},{"start_s":1164.44,"end_s":1165.44,"text":"Yay.","speaker":null,"is_sponsor":0},{"start_s":1165.44,"end_s":1166.44,"text":"Video shot.","speaker":null,"is_sponsor":0},{"start_s":1166.44,"end_s":1170.44,"text":"We should physically print all of our videos, every frame of every video on that paper.","speaker":null,"is_sponsor":0},{"start_s":1170.44,"end_s":1171.44,"text":"I don't think so.","speaker":null,"is_sponsor":0},{"start_s":1171.44,"end_s":1173.12,"text":"Did you hear the good news?","speaker":null,"is_sponsor":0},{"start_s":1173.12,"end_s":1174.12,"text":"That Wanik's back?","speaker":null,"is_sponsor":0},{"start_s":1174.12,"end_s":1175.12,"text":"Wanik is back.","speaker":null,"is_sponsor":0},{"start_s":1175.12,"end_s":1177.12,"text":"Yay for hiring other people to do our jobs.","speaker":null,"is_sponsor":0},{"start_s":1177.12,"end_s":1181.06,"text":"No, I don't think it's back, but it is.","speaker":null,"is_sponsor":0},{"start_s":1181.06,"end_s":1182.06,"text":"It's back.","speaker":null,"is_sponsor":0},{"start_s":1182.06,"end_s":1183.54,"text":"What are you doing?","speaker":null,"is_sponsor":0},{"start_s":1183.54,"end_s":1185.62,"text":"I'm just letting everyone know that Taren, it's back.","speaker":null,"is_sponsor":0},{"start_s":1185.62,"end_s":1186.62,"text":"The server is back.","speaker":null,"is_sponsor":0},{"start_s":1186.62,"end_s":1187.62,"text":"It's back.","speaker":null,"is_sponsor":0},{"start_s":1187.62,"end_s":1188.62,"text":"The server is back.","speaker":null,"is_sponsor":0},{"start_s":1188.62,"end_s":1189.62,"text":"Okay?","speaker":null,"is_sponsor":0},{"start_s":1189.62,"end_s":1203.12,"text":"So it's not always that our sponsors tie so perfectly into our content, but today it happened.","speaker":null,"is_sponsor":0},{"start_s":1203.12,"end_s":1207.9,"text":"Today's sponsor is Rackspace, the top tier managed cloud computing company.","speaker":null,"is_sponsor":0},{"start_s":1207.9,"end_s":1211.72,"text":"They pride themselves on best in class service across all platforms.","speaker":null,"is_sponsor":0},{"start_s":1211.72,"end_s":1217.02,"text":"They've got over 300,000 customers in 120 countries with 10 worldwide data centers.","speaker":null,"is_sponsor":0},{"start_s":1217.02,"end_s":1223.06,"text":"They've got Red Hat, Cisco, Microsoft, VMware certifications, and all of that amounts to,","speaker":null,"is_sponsor":0},{"start_s":1223.06,"end_s":1227.82,"text":"whether you're running a small business or a billion dollar enterprise, whatever your","speaker":null,"is_sponsor":0},{"start_s":1227.82,"end_s":1233.56,"text":"needs are in terms of capacity storage or flash-based high performance storage, they","speaker":null,"is_sponsor":0},{"start_s":1233.56,"end_s":1242.18,"text":"can take care of it without you going gray or bald because you did something wrong and","speaker":null,"is_sponsor":0},{"start_s":1242.18,"end_s":1246.56,"text":"screwed up a backup and went and lost a RAID array or whatever else.","speaker":null,"is_sponsor":0},{"start_s":1246.56,"end_s":1252.06,"text":"They've got dedicated storage to meet your performance, security, network capacity, and","speaker":null,"is_sponsor":0},{"start_s":1252.06,"end_s":1253.06,"text":"compliance needs.","speaker":null,"is_sponsor":0},{"start_s":1253.06,"end_s":1257.78,"text":"Everything from direct attached storage, so that gives you the flexibility and scalability","speaker":null,"is_sponsor":0},{"start_s":1257.78,"end_s":1263.0,"text":"of just a simple, okay, let's attach some drives to this thing, it's redundant, cool,","speaker":null,"is_sponsor":0},{"start_s":1263.0,"end_s":1268.34,"text":"off you go to the races, to SAN, so that's high availability and reliability, fully redundant","speaker":null,"is_sponsor":0},{"start_s":1268.34,"end_s":1273.5,"text":"for business, to even NAS with support for demanding workloads like virtualization, file","speaker":null,"is_sponsor":0},{"start_s":1273.5,"end_s":1275.0,"text":"sharing, and rich media.","speaker":null,"is_sponsor":0},{"start_s":1275.0,"end_s":1280.7,"text":"In fact, their NAS stuff can scale up to 20 petabytes of capacity.","speaker":null,"is_sponsor":0},{"start_s":1280.7,"end_s":1283.0,"text":"They've got public cloud and private cloud options.","speaker":null,"is_sponsor":0},{"start_s":1283.0,"end_s":1288.0,"text":"As well, you can get your own server, a scalable private cloud in the data center, in your","speaker":null,"is_sponsor":0},{"start_s":1288.0,"end_s":1293.06,"text":"data center, in their data center, and it's all supported by Rackspace and VMware.","speaker":null,"is_sponsor":0},{"start_s":1293.06,"end_s":1298.0,"text":"They call their support fanatical because these people are available 24 hours a day,","speaker":null,"is_sponsor":0},{"start_s":1298.0,"end_s":1304.64,"text":"seven days a week, 365 days a year, and they've got industry leading service level agreements,","speaker":null,"is_sponsor":0},{"start_s":1304.64,"end_s":1306.7,"text":"both managed and intensive.","speaker":null,"is_sponsor":0},{"start_s":1306.7,"end_s":1311.84,"text":"So if you need dedicated storage, they got you covered.","speaker":null,"is_sponsor":0},{"start_s":1311.84,"end_s":1312.84,"text":"Go to rackspace.com.","speaker":null,"is_sponsor":0},{"start_s":1312.84,"end_s":1318.8,"text":"de to learn more, and not go through what I did.","speaker":null,"is_sponsor":0},{"start_s":1318.8,"end_s":1324.05,"text":"I just, yeah, I don't really have anything to say","speaker":null,"is_sponsor":0},{"start_s":1324.05,"end_s":1325.05,"text":"Thanks for watching guys.","speaker":null,"is_sponsor":0},{"start_s":1325.05,"end_s":1329.09,"text":"If you just like this video, then hit the dislike button, I guess.","speaker":null,"is_sponsor":0},{"start_s":1329.09,"end_s":1330.09,"text":"But come on.","speaker":null,"is_sponsor":0},{"start_s":1330.09,"end_s":1332.71,"text":"This is about as real as it gets around here.","speaker":null,"is_sponsor":0},{"start_s":1332.71,"end_s":1335.09,"text":"If you liked the video, hit that like button.","speaker":null,"is_sponsor":0},{"start_s":1335.09,"end_s":1340.51,"text":"Get subscribed, maybe even consider checking out our Amazon affiliate, instructions for","speaker":null,"is_sponsor":0},{"start_s":1340.51,"end_s":1341.51,"text":"how to use it.","speaker":null,"is_sponsor":0},{"start_s":1341.51,"end_s":1342.37,"text":"Whenever you buy stuff there, instructions are up there.","speaker":null,"is_sponsor":0},{"start_s":1342.37,"end_s":1343.47,"text":"Buy and share with worksheets.","speaker":null,"is_sponsor":0},{"start_s":1343.47,"end_s":1343.97,"text":"That's it.","speaker":null,"is_sponsor":0},{"start_s":1344.31,"end_s":1348.07,"text":"cool shirt like this one or even giving us a monthly contribution through our","speaker":null,"is_sponsor":0},{"start_s":1348.07,"end_s":1351.33,"text":"forum which gives you a little contributor badge now that you're done","speaker":null,"is_sponsor":0},{"start_s":1351.33,"end_s":1354.11,"text":"doing all that stuff hey maybe you want to check out that Channel Super Fun","speaker":null,"is_sponsor":0},{"start_s":1354.11,"end_s":1359.25,"text":"video that we just discovered is back and that'll definitely be worth your","speaker":null,"is_sponsor":0},{"start_s":1359.25,"end_s":1365.48,"text":"while we joust with each other on swagways it's pretty awesome so see you","speaker":null,"is_sponsor":0},{"start_s":1365.48,"end_s":1367.96,"text":"next time","speaker":null,"is_sponsor":0}],"full_text":"Well, hello YouTube We have a bit of a problem today on this section of the moving, but we've been moved in for months moving vlog Something is going on with the server and we haven't been able to work on a project for more than like 30 minutes at a time before the server crashes and Our computers all freeze and then we're all sad because we can't work on them. Most of the sh** we do is on the server Well, I'm doing I'm actually doing some writing right now because I can do that I'd love to be working on my next video. Anything that doesn't need the server we can do now No, that's not much, but it always seems to go down Yeah, at the worst when there's nothing else to do. No kidding, but you know YOLO you only Liao once See let's see the stress that Linus is going through right now I'm only getting about 25 minutes per second, 30 minutes per second It's probably gonna crash like while you're doing this Yes How do we fix servers? How do you fix it? Tell Linus he will fix it That's your go-to. Yeah Whether you're running a small business or a One billion dollar or even multi-billion dollar enterprise Rackspace has your dedicated storage environments covered Check out the link in the video description to learn more All right So here's the situation Wonix server over the course of the last couple of days has been spontaneously going offline And turning off in spite of my best efforts to turn it back on and Complete the backup that I'm in the middle of trying to do to our new unraid vault While I was standing in front of it. I observed One of my raid controllers giving up the ghost So as a reminder this server is running three raid fives Striped together in Windows of a total of 24 SSDs So if one of the raid fives drops out in time Entirely all the data is gone only about 10% of the backup that I was just in the middle of remaking is saved so It's time to Investigate. All right. So here's wonix server So far troubleshooting steps that I've tried you never want to you never want to rebuild something like this if there's a possibility You're gonna have to go to a data recovery service because that can make it much much worse so In terms of troubleshooting steps, I tried transplanting it into another case so that I could use a different SATA backplane that didn't work. I tried a different power supply a better power supply That didn't work None of that fixed it and I even went as far as to put in Another LSI raid card that I had and try to import the array But while it did detect my drives all of them as unconfigured good and a foreign array It was not able to import it So there is some good news there though while I'm getting firmware errors kicked out by this card right here when I try to boot All the drives are detected by another card So hopefully it didn't write too much garbled data to the array Okay, so this is basically what I've been doing for the last 14 hours is sending emails contacting different Data recovery services There's some that are local there's some that are not local, but I'm not ruling those ones out. In fact, right now the most likely solution looks like WeRecoverData.com. They've got some custom tools that they've created that will allow them to SSH into the system and actually potentially recover the missing RAID 5 and then the other two RAID 5s and assemble them all together and export them as one gigantic pile of data to one of our servers, all without me actually having to send the drives away, which would be pretty ideal. Now obviously an approach like that would not work in the case of physical damage to the drives, but because it is only a RAID controller issue, they're saying, hey, there's a shot at this, so let's give it a try. So I just downloaded their remote recovery client thing here. So right now I'm waiting for their custom Linux-based tool to load on a bootable external SSD so that they can SSH into the server. I'm not having a good couple days. Right now I can't get the utility to detect my USB drive. It's called tech support. Oh bloody hell, I just figured it out. This is Linus calling, by the way. I just realized I have to run the WRD disk image as administrator. You guys should put that in the documentation. It never fails. This is why it's a good idea to call tech support, because the second you call them, you'll solve the problem on your own. Alright, so in theory we now have a bootable drive. Okay, so step one is getting this unnecessary extraneous server out of the way. I'm grabbing a UPS to use. Now, in a perfect world, I would have a motherboard or an HBA card, a host bus adapter card, that could allow me to plug in all 24 of the drives for them to do their data recovery magic on. But what we're gonna have to settle for is having eight of them plugged into the motherboard at a time as each RAID 5 is rebuilt. RAID cards do all kinds of funny, funny nonsense and they don't give other software direct access to the drives at a bit level in some cases. So we need to remove all these drives and plug them in directly to the motherboard. Okay, now it should boot to that, in theory. Let's just make sure all the drives are detected in the BIOS and everything. Oh, that's a problem. I need to pull out the other RAID controllers as well, otherwise they're gonna be looking for their drives and I don't have them powered up right now. Okay. Oh, actually that doesn't look very good. DCI-E port errors. What? So let's make sure all of our drives are even showing up. Uh-oh. That is extremely worrying. Oh, that is extremely worrying. Oh, that's not good. That means potentially three of the drives in a RAID 5 are dead, which means this data is not coming back. None of it. Not this particular RAID? No, none of it. None of the... But I thought we have three RAID 5s. None of it will be coming back if we lost three drives from one of the RAID 5s. So let's try moving things around. I've never tested the SATA ports on this board. It's possible three of them are dead, however extraordinarily unlikely that would be. Five, six. Okay, we've got six now. Okay, that gives some hope. I'm just moving SATA ports around. I was just gonna say right now I'm even having difficulty in this system to post. So I'm at a whole new level of concern. Yeah, I'll let you know. I'll send an email. Okay. So I'm at the point now where I don't want to put the drives on this system. Let's go with this one. I didn't like all those PCI Express errors that the bootable USB was kicking out. I don't like that some of these drives aren't showing up. I'm starting to wonder if that's what corrupted the LSI card. This royal pain in the ass. A thousand things I can think of that I would rather be doing right now. Okay, I got seven right now. Really? Yeah. Wow, maybe it was bad PCIe. Linus speaking. Well, I can tell you what's happening so far is I put it on the system that I was running the RAID card and the drives in and that baby started spitting out PCIe errors all over the place. So I got everything pulled out of that system and I have plugged all the drives now into a different disk bench and I'm trying to see because I wasn't even getting all the drives showing up. Two, three, four, five, six, seven, eight. Okay. Hey, you guys are on the phone for the magic moment where all eight drives are detected by a new motherboard. So I'm beginning to think there may be a motherboard issue that caused the firmware on that LSI controller to corrupt in the first place. So I'm actually pressing enter and I am booting up to your USB and it is loading correctly on this. I think that board is dead. Okay. Well, that's I don't know if that makes it worse. Probably does, doesn't it? You can tell me. I can take it. It says waiting up to 60 more seconds for network configuration. Good timing with the follow-up call. If you called one minute earlier, I've been cursing you for dialing my number. So I have to answer my phone because I was in the middle of thinking only five of my drives were being detected. You guys know what you're doing, right? I had a local shop that I was going to physically take it to and they basically, they were like, they kind of freaked out when I said I was having someone use their remote tool. They were like, well, unless these guys really know what they're doing. That's pretty dangerous. And I was like, okay, well I'll, I'll check with them. I'll make sure they, they know what they're doing. So, so you guys know what you're doing. So basically they're explaining that they're going to virtually Mount the volumes. So we're not doing any writing to my discs. Um, so even in the event that we cannot recover it by this means, uh, that still leaves other options open to us. Okay. They're in. Okay. Well, that's, that's very wonderful to hear, but I'm trying not to get too excited right now. They sound pretty confident. This is like the ultimate roller coaster right now. So to put this in the appropriate context for the viewers right now, one X server has several in progress projects stored on it that do not have a backup anywhere else, including a fully shot channel, super fun that involved a thousand dollars in equipment rental, um, including, uh, lots of footage from Linus tech tips videos. Like it is going to be a huge, huge, devastating loss for us. To lose this. I mean, it's got a lot of our, our templates for editing, uh, editing videos for, for scripting templates for employee reviews, like all kinds of stuff. Like this is our main working server. No, the offsite backup server has not ever been built yet. The vault was the backup server. If I ever have to like act in a, in a video where I have to look for Lauren, I'm going to think back to this moment. Okay. So I do need to reboot it once in order to add that eight terabyte drive. So anyway, right. Okay. Thank you very much, gentlemen. I hope to speak to you, uh, with good news very soon, 12 SATA connectors. Ah, ah, ah. Okay. So let's make sure everything is showing up 1, 2, 3, 4, 5, 6, 7, 8 of those are boot device and our eight terabyte recovery media. We are ready to rock. Okay. So within about 10 minutes, they contacted me back. They said, okay, yeah, we're ready to have all the drives in there. So that means we are going to have to do some janky ass. Right now. So basically what you're looking at here is these drives are going to be powered off this power supply with a jumper. These cards and all these drives are going to be powered off of this rig. So now we can power all those drives without powering up this motherboard or any of that nonsense. Okay? So we have a grand total of 25 drives connected to this test bench. 16 of which are running in Bay's. of this enclosure and eight of which are spread out here. So there we go. There's one of my LSI controllers. There's its eight drives. There's my other one. Whoa, where are your virtual drives? I think I remember a troubleshooting step where I tried swapping all the drives to different bays where I was trying to see if it was the back plane or the cables. Wow, panic moment there, but it's okay. So now we should see two virtual drives here on the screen. That's one set. Show me the second set. Oh, I thought we lost another RAID 5 there. Okay, it's good. Feeling good. Reality TV ain't got nothing on IT work. Show me all the devices. Two RAID adapters, one eight terabyte drive, one, two, three, four, five, six, seven, eight Kingston drives. Boom. Okay. I will let... the data recovery specialists know. All right. So all that's left now then I think, I got in touch with them and I asked if we can put a 10 gigabit network card in so we can accelerate the offloading of all of our files. So all we're waiting for now is for them to reboot the machine and set me up with a network share that I can access and fingers crossed, just start pulling off all the footage and well, everything else. All right. So check this out here, around here. So I've got the instructions for how to access our data. So I go into the Tritium share and there it is. Bunch of EULAs, a bunch of stuff. We've got what looks like the file structure. Let's test it. How about one that I, that we haven't finished releasing yet? Like something from the, the melee battle here. Let's just, let's watch it. Hmm. Okay. XA footage for sure. Oh, okay. Here we can copy one of these. Let's copy one of these clips to my local machine and see if that helps. No. Oh boy. Let's have a look. I wonder if everything's even here. Like I don't even see the Linus tech tips folder. One and two are missing. Useful applications is empty. That should be full of things. Okay. Well, I guess we'd better get in touch with them. So thanks for watching. Subscribe, follow all that good stuff and see you next time. I just got an email back from them. Please stand by addressing your emails with the engineer. It's possible the RAID definitions were changed from one reboot to the next because of drive letter assignments in Linux. It's 26, so it's 26 drive limit. We have, I think we have more than 26 devices in there. I got another email back. Linus, we recreated the block devices. Please go ahead and check the videos in context. That you were looking at again. No way. Oh, the suspense. I'm not enjoying this. Is nothing there? I'll let them know the chair's empty again. This is just like the wildest roller coaster ever. I don't even, yeah. I don't even have anything else to say actually. It's almost funny. I hope to some of you it's funny. Here's the email I got 17 minutes ago. I was stuck on a call. We got everything remounted. So you can now go ahead and view the data. Once more. This is the moment. Do we not have a two? I don't think we ever had a two. Oh, okay. Okay, so let's go to that Channel Super Fun video that we were looking at. Yes, yes, Swagway jousting. Okay, and it was the delivery. I'm getting like tingling. Okay, okay, you gotta check the source files though because that was what we know was corrupted. Okay, okay, okay, okay. So untranscoded footage. XA, they did it. It's back. My God. It's back, here's another file. It's back. Do we have everything? In theory, yes. Really? Holy . Oh. I think it's finally over. Woo hoo. That is fantastic. Hey guys, guess what happened with the recovery? We got everything back. Oh, thank goodness. Woo hoo. Like it's just been such a stressful few weeks and that was just such a huge, huge thing. All of our pitches and proposals and stuff are on that server. Okay, Wanik's back. Bully? Really? Everything is back as far as I can tell. I have played back video files. They are not corrupted. Wanik is officially back. Swagway jousting footage for the melee battle should be fine. And we'll have a better backup scheme in the future, I promise. That is all. Yay. Yay. Yay. Video shot. We should physically print all of our videos, every frame of every video on that paper. I don't think so. Did you hear the good news? That Wanik's back? Wanik is back. Yay for hiring other people to do our jobs. No, I don't think it's back, but it is. It's back. What are you doing? I'm just letting everyone know that Taren, it's back. The server is back. It's back. The server is back. Okay? So it's not always that our sponsors tie so perfectly into our content, but today it happened. Today's sponsor is Rackspace, the top tier managed cloud computing company. They pride themselves on best in class service across all platforms. They've got over 300,000 customers in 120 countries with 10 worldwide data centers. They've got Red Hat, Cisco, Microsoft, VMware certifications, and all of that amounts to, whether you're running a small business or a billion dollar enterprise, whatever your needs are in terms of capacity storage or flash-based high performance storage, they can take care of it without you going gray or bald because you did something wrong and screwed up a backup and went and lost a RAID array or whatever else. They've got dedicated storage to meet your performance, security, network capacity, and compliance needs. Everything from direct attached storage, so that gives you the flexibility and scalability of just a simple, okay, let's attach some drives to this thing, it's redundant, cool, off you go to the races, to SAN, so that's high availability and reliability, fully redundant for business, to even NAS with support for demanding workloads like virtualization, file sharing, and rich media. In fact, their NAS stuff can scale up to 20 petabytes of capacity. They've got public cloud and private cloud options. As well, you can get your own server, a scalable private cloud in the data center, in your data center, in their data center, and it's all supported by Rackspace and VMware. They call their support fanatical because these people are available 24 hours a day, seven days a week, 365 days a year, and they've got industry leading service level agreements, both managed and intensive. So if you need dedicated storage, they got you covered. Go to rackspace.com. de to learn more, and not go through what I did. I just, yeah, I don't really have anything to say Thanks for watching guys. If you just like this video, then hit the dislike button, I guess. But come on. This is about as real as it gets around here. If you liked the video, hit that like button. Get subscribed, maybe even consider checking out our Amazon affiliate, instructions for how to use it. Whenever you buy stuff there, instructions are up there. Buy and share with worksheets. That's it. cool shirt like this one or even giving us a monthly contribution through our forum which gives you a little contributor badge now that you're done doing all that stuff hey maybe you want to check out that Channel Super Fun video that we just discovered is back and that'll definitely be worth your while we joust with each other on swagways it's pretty awesome so see you next time"}