{"video_id":"12l326jQML8","title":"I predicted the future! - GRAID Graphics Accelerated Storage","channel":"Linus Tech Tips","show":"Linus Tech Tips","published_at":"2022-05-05T14:53:29Z","duration_s":1594,"segments":[{"start_s":0.24,"end_s":3.28,"text":"i called it","speaker":null,"is_sponsor":0},{"start_s":3.36,"end_s":9.84,"text":"called what okay but i did though i mean sort of i'm","speaker":null,"is_sponsor":0},{"start_s":7.839,"end_s":13.679,"text":"pretty sure it was me and i'm also pretty sure it existed before that video","speaker":null,"is_sponsor":0},{"start_s":11.759,"end_s":21.6,"text":"even came out it doesn't matter the point is meet the supreme raid sr 1000","speaker":null,"is_sponsor":0},{"start_s":18.24,"end_s":23.76,"text":"it looks like an NVIDIA t1000","speaker":null,"is_sponsor":0},{"start_s":21.6,"end_s":29.519,"text":"workstation GPU in fact it even has the letters t1000 printed on it and the same","speaker":null,"is_sponsor":0},{"start_s":27.199,"end_s":32.719,"text":"mini displayport ports are in there but but they're blocked","speaker":null,"is_sponsor":0},{"start_s":30.96,"end_s":37.76,"text":"by solid metal","speaker":null,"is_sponsor":0},{"start_s":34.559,"end_s":39.76,"text":"that's because this GPU is not meant for","speaker":null,"is_sponsor":0},{"start_s":37.76,"end_s":43.68,"text":"graphics and before you say you know where this is going no it's not for","speaker":null,"is_sponsor":0},{"start_s":42.0,"end_s":50.48,"text":"cryptocurrency either so what the heck is it through some kind of","speaker":null,"is_sponsor":0},{"start_s":47.12,"end_s":53.76,"text":"software funkery g-raid is using this","speaker":null,"is_sponsor":0},{"start_s":50.48,"end_s":55.92,"text":"GPU to act as a freaking storage","speaker":null,"is_sponsor":0},{"start_s":53.76,"end_s":60.719,"text":"accelerator and if they're to be believed which i'm not sure if i do yet","speaker":null,"is_sponsor":0},{"start_s":58.559,"end_s":66.08,"text":"this thing with the right array of NVMe drives can supposedly sustain transfer","speaker":null,"is_sponsor":0},{"start_s":62.559,"end_s":68.0,"text":"speeds of over 100 gigabytes per second","speaker":null,"is_sponsor":0},{"start_s":66.08,"end_s":72.96,"text":"of sequential throughput holy sh","speaker":null,"is_sponsor":0},{"start_s":70.72,"end_s":79.92,"text":"is what you might say if i didn't segue to our sponsor keoxia their bg5 NVMe SSD","speaker":null,"is_sponsor":1},{"start_s":77.36,"end_s":85.36,"text":"brings PCIe gen 4 performance to an affordable price for systems and","speaker":null,"is_sponsor":1},{"start_s":82.24,"end_s":87.28,"text":"notebooks they even make a 2230 sized","speaker":null,"is_sponsor":1},{"start_s":85.36,"end_s":91.28,"text":"one so your pcs can be lighter and smaller than ever check them out at the","speaker":null,"is_sponsor":1},{"start_s":89.2,"end_s":96.479,"text":"link in the video description i have so many questions about this","speaker":null,"is_sponsor":1},{"start_s":94.32,"end_s":100.72,"text":"but before we can even start to answer them we need a little bit of background","speaker":null,"is_sponsor":0},{"start_s":98.479,"end_s":104.64,"text":"combining multiple storage devices has been a staple of computing for decades","speaker":null,"is_sponsor":0},{"start_s":102.88,"end_s":110.56,"text":"and generally falls under the umbrella of technologies that we call raid or","speaker":null,"is_sponsor":0},{"start_s":107.68,"end_s":114.64,"text":"redundant array of independent disks raid can serve a variety of purposes","speaker":null,"is_sponsor":0},{"start_s":113.2,"end_s":120.159,"text":"improving speed data protection capacity or usually some","speaker":null,"is_sponsor":0},{"start_s":117.84,"end_s":124.399,"text":"combination of all three of those compared to a single drive","speaker":null,"is_sponsor":0},{"start_s":122.0,"end_s":129.679,"text":"now traditionally high performance raid required dedicated co-processors","speaker":null,"is_sponsor":0},{"start_s":126.88,"end_s":133.44,"text":"typically found on hardware raid cards you would slot one of those bad boys","speaker":null,"is_sponsor":0},{"start_s":131.599,"end_s":138.16,"text":"into your motherboard connect all of your drives to it and it would handle","speaker":null,"is_sponsor":0},{"start_s":135.28,"end_s":142.64,"text":"both the high throughput of these many disk arrays as well as the parity","speaker":null,"is_sponsor":0},{"start_s":140.959,"end_s":147.599,"text":"calculations that are required by popular raid configurations like raid 5","speaker":null,"is_sponsor":0},{"start_s":145.36,"end_s":151.28,"text":"and raid 6. if you want to learn more we actually have a Techquickie on this","speaker":null,"is_sponsor":0},{"start_s":148.8,"end_s":154.72,"text":"subject from almost 10 years ago but","speaker":null,"is_sponsor":1},{"start_s":152.4,"end_s":160.08,"text":"raid cards have a problem as we've transitioned from mechanical","speaker":null,"is_sponsor":0},{"start_s":156.8,"end_s":162.8,"text":"drives to solid state and then to NVMe","speaker":null,"is_sponsor":0},{"start_s":160.08,"end_s":166.319,"text":"storage devices have gotten so fast that raid cards haven't been able to keep","speaker":null,"is_sponsor":0},{"start_s":164.48,"end_s":170.48,"text":"pace turning them into a performance bottleneck so the current meta is to","speaker":null,"is_sponsor":0},{"start_s":168.239,"end_s":177.28,"text":"connect your storage devices directly to your CPU via pci express this improves","speaker":null,"is_sponsor":0},{"start_s":174.08,"end_s":180.08,"text":"both the throughput and latency but","speaker":null,"is_sponsor":0},{"start_s":177.28,"end_s":185.92,"text":"requires the CPU to handle those parity calculations and any other overhead this","speaker":null,"is_sponsor":0},{"start_s":182.8,"end_s":188.0,"text":"is called software raid and","speaker":null,"is_sponsor":0},{"start_s":185.92,"end_s":192.959,"text":"in some ways it's actually kind of a big step backward first of all cpus are","speaker":null,"is_sponsor":0},{"start_s":190.879,"end_s":197.28,"text":"freaking expensive and in more ways than you might think a lot of enterprise","speaker":null,"is_sponsor":0},{"start_s":195.44,"end_s":202.48,"text":"software is licensed according to how many cpus or how many cores are present","speaker":null,"is_sponsor":0},{"start_s":200.239,"end_s":207.28,"text":"in your server so you better believe that big businesses are all about","speaker":null,"is_sponsor":0},{"start_s":204.56,"end_s":211.28,"text":"squeezing the absolute most out of every box","speaker":null,"is_sponsor":0},{"start_s":208.159,"end_s":213.28,"text":"also cpus are generalized processors i","speaker":null,"is_sponsor":0},{"start_s":211.28,"end_s":217.68,"text":"mean you can brute force it here's us hitting 20 gigabytes a second in","speaker":null,"is_sponsor":0},{"start_s":215.28,"end_s":223.519,"text":"software raid but the issue is that even with a 32 core epic processor we're","speaker":null,"is_sponsor":0},{"start_s":220.0,"end_s":226.08,"text":"looking at a lot of utilization here","speaker":null,"is_sponsor":0},{"start_s":223.519,"end_s":231.12,"text":"just to manage storage and if you compare that to the theoretical combined","speaker":null,"is_sponsor":0},{"start_s":228.159,"end_s":236.319,"text":"read speed of around 75 gigabytes a second for our 12 kiosks cd6 drives you","speaker":null,"is_sponsor":0},{"start_s":234.4,"end_s":240.799,"text":"can see that we were leaving a lot of performance on that table","speaker":null,"is_sponsor":0},{"start_s":238.239,"end_s":246.08,"text":"if i'm a server vendor that's too many wasted CPU cycles that my customers now","speaker":null,"is_sponsor":0},{"start_s":243.519,"end_s":249.84,"text":"can't allocate to something useful or can't rent out to their customers","speaker":null,"is_sponsor":0},{"start_s":248.879,"end_s":254.239,"text":"that is where g-raid comes into play","speaker":null,"is_sponsor":0},{"start_s":252.239,"end_s":261.44,"text":"the most obvious difference here right out of the gate is that there is no port","speaker":null,"is_sponsor":0},{"start_s":257.04,"end_s":262.479,"text":"to plug a drive into never mind 8 or 16","speaker":null,"is_sponsor":0},{"start_s":261.44,"end_s":267.6,"text":"drives instead the drives connect directly to","speaker":null,"is_sponsor":0},{"start_s":265.12,"end_s":271.759,"text":"the CPU's PCIe lanes just like they would with software raid so this server","speaker":null,"is_sponsor":0},{"start_s":270.16,"end_s":276.16,"text":"from Gigabyte handles all of that through a backplane here in the front of","speaker":null,"is_sponsor":0},{"start_s":273.44,"end_s":282.4,"text":"the chassis then the g-raid card just plugs into any available PCIe gen4 slot","speaker":null,"is_sponsor":0},{"start_s":279.84,"end_s":286.16,"text":"we'll do that a little bit later and all the storage communication happens over","speaker":null,"is_sponsor":0},{"start_s":284.24,"end_s":291.199,"text":"the PCIe bus no direct connection between our","speaker":null,"is_sponsor":0},{"start_s":288.479,"end_s":296.479,"text":"raid card and our drives weird i guess we won't need any of our","speaker":null,"is_sponsor":0},{"start_s":293.28,"end_s":298.4,"text":"new cable ties from lttstore.com","speaker":null,"is_sponsor":1},{"start_s":296.479,"end_s":304.88,"text":"are available in so many colors and you might be thinking gee Linus even at gen","speaker":null,"is_sponsor":1},{"start_s":301.36,"end_s":307.039,"text":"4 speeds a 16x PCIe slot can only push","speaker":null,"is_sponsor":0},{"start_s":304.88,"end_s":311.12,"text":"around 32 gigabytes a second in either direction how could this thing possibly","speaker":null,"is_sponsor":0},{"start_s":309.36,"end_s":315.6,"text":"do over 100 that's the special sauce","speaker":null,"is_sponsor":0},{"start_s":313.12,"end_s":320.24,"text":"none of the storage data actually goes through the card","speaker":null,"is_sponsor":0},{"start_s":317.28,"end_s":325.199,"text":"that's the old way of doing raid cards this card only handles the raid","speaker":null,"is_sponsor":0},{"start_s":322.4,"end_s":329.199,"text":"calculations and directing the system where to read and write from","speaker":null,"is_sponsor":0},{"start_s":327.12,"end_s":333.039,"text":"all of the actual data flow just goes directly between the drives and the","speaker":null,"is_sponsor":0},{"start_s":331.68,"end_s":338.24,"text":"system memory no man in the middle and it does all of","speaker":null,"is_sponsor":0},{"start_s":335.68,"end_s":342.16,"text":"this while using barely any CPU resources","speaker":null,"is_sponsor":0},{"start_s":339.919,"end_s":346.479,"text":"or so they claim what i think this means is that we could","speaker":null,"is_sponsor":0},{"start_s":344.0,"end_s":350.4,"text":"plot this g-raid card into any system and it would just work we are definitely","speaker":null,"is_sponsor":0},{"start_s":348.88,"end_s":353.759,"text":"going to try that later but for now we're going to stick to the validated","speaker":null,"is_sponsor":0},{"start_s":351.919,"end_s":361.919,"text":"server that Gigabyte sent over for performance testing our cpus are a pair","speaker":null,"is_sponsor":0},{"start_s":357.039,"end_s":364.96,"text":"of epic 75 f3 monsters they're only 32","speaker":null,"is_sponsor":0},{"start_s":361.919,"end_s":366.72,"text":"core but they'll hit 280 watts of max","speaker":null,"is_sponsor":0},{"start_s":364.96,"end_s":371.6,"text":"power consumption and will boost to 4 gigahertz and then we paired these with","speaker":null,"is_sponsor":0},{"start_s":369.199,"end_s":377.199,"text":"some equally monstrous memory micron sent over a metric whack ton of 32 mega","speaker":null,"is_sponsor":0},{"start_s":374.96,"end_s":381.68,"text":"transfer per second ecc RAM giving us a total of one terabyte of system memory","speaker":null,"is_sponsor":0},{"start_s":380.24,"end_s":385.759,"text":"shouldn't be a bottleneck right i don't think so it should be fine","speaker":null,"is_sponsor":0},{"start_s":383.36,"end_s":389.759,"text":"then for our drives we're using kyocia cd6rs","speaker":null,"is_sponsor":0},{"start_s":387.36,"end_s":394.0,"text":"they're a well-balanced enterprise gen 4nbma drive and with 12 of them in here","speaker":null,"is_sponsor":0},{"start_s":392.319,"end_s":398.24,"text":"we should be looking at raw sequential performance of around 75 gigabytes a","speaker":null,"is_sponsor":0},{"start_s":396.639,"end_s":401.84,"text":"second before we can set up an array though we need to install the g-raid","speaker":null,"is_sponsor":0},{"start_s":399.919,"end_s":405.44,"text":"supreme raid software and also we need to like finish putting all the RAM in","speaker":null,"is_sponsor":0},{"start_s":403.199,"end_s":410.319,"text":"this system it still blows me away how low profile of a cooler they can use for","speaker":null,"is_sponsor":0},{"start_s":407.68,"end_s":414.96,"text":"these 280 watt cpus but that's the thing is under this giant heat spreader all","speaker":null,"is_sponsor":0},{"start_s":412.56,"end_s":418.56,"text":"the dyes right it's a chiplet design so they're actually like freaking spread","speaker":null,"is_sponsor":0},{"start_s":416.8,"end_s":421.12,"text":"out like it's it's huge it's a lot of surface area to transfer that heat","speaker":null,"is_sponsor":0},{"start_s":420.16,"end_s":425.44,"text":"brother these are these are 80 watt fans consume","speaker":null,"is_sponsor":0},{"start_s":424.16,"end_s":430.24,"text":"80 watts that's uh each 12 volts 7 amp","speaker":null,"is_sponsor":0},{"start_s":429.12,"end_s":432.8,"text":"i think that might be part of the equation","speaker":null,"is_sponsor":0},{"start_s":432.88,"end_s":441.199,"text":"wow that is a heavy vapor chamber i love it","speaker":null,"is_sponsor":0},{"start_s":437.68,"end_s":443.84,"text":"for how small this thing is like","speaker":null,"is_sponsor":0},{"start_s":441.199,"end_s":446.88,"text":"it's a heavy boy oh my god this is a lot of memory","speaker":null,"is_sponsor":0},{"start_s":444.88,"end_s":450.96,"text":"it's gorgeous a freaking terabyte","speaker":null,"is_sponsor":0},{"start_s":449.28,"end_s":453.84,"text":"why are you whispering because it's a terrible memory i don't want to wake it","speaker":null,"is_sponsor":0},{"start_s":452.319,"end_s":458.319,"text":"up as of recording the video this only runs","speaker":null,"is_sponsor":0},{"start_s":456.08,"end_s":461.36,"text":"on Linux server operating systems so we're going to be firing it up with do","speaker":null,"is_sponsor":0},{"start_s":459.84,"end_s":463.52,"text":"we have an SSD in here or something back here","speaker":null,"is_sponsor":0},{"start_s":464.56,"end_s":472.56,"text":"yeah just a little SATA SSD there cool so we're gonna be running ubuntu server","speaker":null,"is_sponsor":0},{"start_s":469.0,"end_s":474.08,"text":"20.04 lts uh Jake has already gone ahead","speaker":null,"is_sponsor":0},{"start_s":472.56,"end_s":478.479,"text":"and installed that onto our boot drive as well as the required NVIDIA drivers","speaker":null,"is_sponsor":0},{"start_s":476.639,"end_s":484.0,"text":"and supreme raid itself this is cool i am liking this like super","speaker":null,"is_sponsor":0},{"start_s":481.199,"end_s":487.599,"text":"over engineered airflow director here pretty sure they just ripped it off of","speaker":null,"is_sponsor":0},{"start_s":485.28,"end_s":491.199,"text":"the dell but oh all right","speaker":null,"is_sponsor":0},{"start_s":489.12,"end_s":494.4,"text":"that goes hey yeah that's not even close to full send yet","speaker":null,"is_sponsor":0},{"start_s":493.28,"end_s":499.039,"text":"either the whole process was surprisingly easy i literally just had to copy paste some commands from their","speaker":null,"is_sponsor":0},{"start_s":497.199,"end_s":502.96,"text":"user guide and it looks to be working so i think we can make it a right now","speaker":null,"is_sponsor":0},{"start_s":501.12,"end_s":507.84,"text":"should we start with raid zero i'd be obviously have to start with rate zero","speaker":null,"is_sponsor":0},{"start_s":504.96,"end_s":513.839,"text":"raid zero is not really a great use case for this because raid zero has no parity","speaker":null,"is_sponsor":0},{"start_s":510.8,"end_s":516.64,"text":"data to calculate it's just taking each","speaker":null,"is_sponsor":0},{"start_s":513.839,"end_s":522.08,"text":"bit writing it to the next drive in the sequence and attempting to multiply your","speaker":null,"is_sponsor":0},{"start_s":519.2,"end_s":526.48,"text":"capacity and your speed you get no extra resiliency whatsoever in fact it's worse","speaker":null,"is_sponsor":0},{"start_s":524.32,"end_s":530.24,"text":"because if any one drive fails all the data is","speaker":null,"is_sponsor":0},{"start_s":527.68,"end_s":534.0,"text":"gone oh hey look at that you can totally use this with satan sas drives too i","speaker":null,"is_sponsor":0},{"start_s":532.64,"end_s":538.56,"text":"don't know if you'd want to because i think the limit is 32 drives per","speaker":null,"is_sponsor":0},{"start_s":536.64,"end_s":542.959,"text":"controller thing i don't know what if you could put multiple i","speaker":null,"is_sponsor":0},{"start_s":540.56,"end_s":546.88,"text":"guess why why yeah good point why why why it's for NVMe yes okay so here let's","speaker":null,"is_sponsor":0},{"start_s":545.12,"end_s":551.92,"text":"see list NVMe drive let's see if they all show up so we got 7.7","speaker":null,"is_sponsor":0},{"start_s":549.76,"end_s":556.72,"text":"it's probably tibi bites or what capacity are those stripes 7.68 oh so","speaker":null,"is_sponsor":0},{"start_s":554.72,"end_s":559.279,"text":"then that is terabytes cool there's a bit of an","speaker":null,"is_sponsor":0},{"start_s":557.839,"end_s":562.399,"text":"it's not interesting because it's very similar to like zfs but there's kind of","speaker":null,"is_sponsor":0},{"start_s":561.519,"end_s":567.68,"text":"a structure so you start with your physical drives you know you got your NVMe drives you","speaker":null,"is_sponsor":0},{"start_s":565.6,"end_s":570.88,"text":"can also connect NVMe over fiber drives which is pretty cool and have the","speaker":null,"is_sponsor":0},{"start_s":569.12,"end_s":574.88,"text":"controller in this and your your drives and some other jbod pretty sick some are","speaker":null,"is_sponsor":0},{"start_s":573.2,"end_s":577.6,"text":"across we're not going to do that yet there is still a limit of 32 drives so","speaker":null,"is_sponsor":0},{"start_s":576.48,"end_s":582.32,"text":"it's not like you're going to connect 200 right but 32 with that done you can","speaker":null,"is_sponsor":0},{"start_s":580.88,"end_s":587.2,"text":"go ahead and make your drive group which is kind of like a zfs z pool or just","speaker":null,"is_sponsor":0},{"start_s":585.2,"end_s":591.44,"text":"right it's like your array you can pick your raid level at this stage uh you can","speaker":null,"is_sponsor":0},{"start_s":590.0,"end_s":594.72,"text":"have multiple drive groups i think you can have four um so you could have like","speaker":null,"is_sponsor":0},{"start_s":593.44,"end_s":600.48,"text":"say you had 16 drives you could have like four groups of four sure those would all be discrete","speaker":null,"is_sponsor":0},{"start_s":598.32,"end_s":605.12,"text":"unlike zfs they're not it's not like having four v devs that you then combine","speaker":null,"is_sponsor":0},{"start_s":602.88,"end_s":610.0,"text":"into a pool yeah it's like having four pools yeah let's go back a bit and","speaker":null,"is_sponsor":0},{"start_s":606.959,"end_s":611.839,"text":"actually create our physical drives","speaker":null,"is_sponsor":0},{"start_s":610.0,"end_s":616.32,"text":"it says create that's the command but really it's like take me take me over","speaker":null,"is_sponsor":0},{"start_s":614.399,"end_s":619.839,"text":"we're unbinding it from the operating system and giving it to the g-rate","speaker":null,"is_sponsor":0},{"start_s":618.64,"end_s":626.0,"text":"controller there's a cool little feature here you can go like dev slash NVMe 0-11 oh","speaker":null,"is_sponsor":0},{"start_s":624.24,"end_s":630.24,"text":"that's cool and it makes them all you don't have to do it one by one there you","speaker":null,"is_sponsor":0},{"start_s":627.839,"end_s":635.04,"text":"go made them all successfully and then we can just check the list to see if","speaker":null,"is_sponsor":0},{"start_s":631.839,"end_s":636.399,"text":"they're all there ah cool yes this is","speaker":null,"is_sponsor":0},{"start_s":635.04,"end_s":641.279,"text":"pretty solid documentation from what i've seen so far compared to the documentation for microsoft storage","speaker":null,"is_sponsor":0},{"start_s":639.44,"end_s":645.76,"text":"spaces oh god compared to the documentation for anything microsoft all","speaker":null,"is_sponsor":0},{"start_s":643.839,"end_s":650.079,"text":"right create drive group uh we're gonna do raid and then","speaker":null,"is_sponsor":0},{"start_s":648.24,"end_s":654.8,"text":"pd id so that's a physical disk so we'll go zero to eleven","speaker":null,"is_sponsor":0},{"start_s":652.48,"end_s":661.32,"text":"it's doing stuff let's see if we can see it now g-raid","speaker":null,"is_sponsor":0},{"start_s":656.8,"end_s":661.32,"text":"list drive underscore group","speaker":null,"is_sponsor":0},{"start_s":662.32,"end_s":668.48,"text":"92 terabytes that's not bad just like that hey yeah that's fast a little bit","speaker":null,"is_sponsor":0},{"start_s":666.48,"end_s":671.68,"text":"of an interesting tidbit here we don't have a usable file system yet this is","speaker":null,"is_sponsor":0},{"start_s":670.72,"end_s":676.72,"text":"just the array it's not like zetta fest where there's a file system built in instead","speaker":null,"is_sponsor":0},{"start_s":675.279,"end_s":680.64,"text":"we actually have to make a virtual disk so you can make","speaker":null,"is_sponsor":0},{"start_s":679.12,"end_s":684.0,"text":"a number of them you could have like a 10 terabyte one you could have like a 50","speaker":null,"is_sponsor":0},{"start_s":682.48,"end_s":690.0,"text":"terabyte one sure we're just gonna make one that's the whole thing this is just block level storage yeah so we'll make","speaker":null,"is_sponsor":0},{"start_s":688.16,"end_s":692.64,"text":"big virtual disks that's the full size and then we'll have to put a file system","speaker":null,"is_sponsor":0},{"start_s":691.44,"end_s":698.16,"text":"on it as well so let's do that create virtual drive","speaker":null,"is_sponsor":0},{"start_s":696.24,"end_s":702.48,"text":"okay so our drive group is zero so we'll say zero and then i'm not gonna specify","speaker":null,"is_sponsor":0},{"start_s":700.56,"end_s":705.76,"text":"a size and i think yeah that'll make a full-size one","speaker":null,"is_sponsor":0},{"start_s":703.76,"end_s":710.959,"text":"and there we go we have our virtual drive it says it's optimal and it's 92","speaker":null,"is_sponsor":0},{"start_s":708.56,"end_s":714.16,"text":"terabytes okay now i got to make a file system on","speaker":null,"is_sponsor":0},{"start_s":712.48,"end_s":717.839,"text":"it i already like like made these commands so i can just copy past them","speaker":null,"is_sponsor":0},{"start_s":716.16,"end_s":721.839,"text":"got the file system working i think it was a little bit angry about how i had","speaker":null,"is_sponsor":0},{"start_s":720.079,"end_s":727.44,"text":"previously had a file system on these disks and just deleted it made a new one","speaker":null,"is_sponsor":0},{"start_s":724.32,"end_s":729.92,"text":"anyways i deleted everything rebooted it","speaker":null,"is_sponsor":0},{"start_s":727.44,"end_s":733.68,"text":"created it again now it's happy i also realized that because we now have two","speaker":null,"is_sponsor":0},{"start_s":732.0,"end_s":737.92,"text":"dimms per channel previously when i was just kind of tinkering with this i just","speaker":null,"is_sponsor":0},{"start_s":735.36,"end_s":741.76,"text":"had 16 sticks in which is eight per one per channel now we have twice that","speaker":null,"is_sponsor":0},{"start_s":740.079,"end_s":747.04,"text":"usually that means your memory speed is going to go down fortunately on the Gigabyte servers you can force it to be","speaker":null,"is_sponsor":0},{"start_s":745.2,"end_s":751.92,"text":"full speed captain we're doing 32q depth one meg sequential","speaker":null,"is_sponsor":0},{"start_s":749.6,"end_s":755.519,"text":"read and that's with 24 threads so two per NVMe drive that's usually pretty","speaker":null,"is_sponsor":0},{"start_s":756.839,"end_s":760.56,"text":"standard oh i hear it","speaker":null,"is_sponsor":0},{"start_s":761.6,"end_s":767.92,"text":"holy it is straight up just immediately okay","speaker":null,"is_sponsor":0},{"start_s":766.24,"end_s":772.0,"text":"it did go down a bit it's it's still twice as fast as what","speaker":null,"is_sponsor":0},{"start_s":770.079,"end_s":776.72,"text":"we've seen with CPU raid though yeah look at why ice is fast the CPU usage is","speaker":null,"is_sponsor":0},{"start_s":774.32,"end_s":780.32,"text":"only like and it's barely touching the CPU three four percent that might even","speaker":null,"is_sponsor":0},{"start_s":778.56,"end_s":785.04,"text":"just be like fio but actually it says it's system stuff so it probably isn't","speaker":null,"is_sponsor":0},{"start_s":782.399,"end_s":788.8,"text":"oh that's crazy it looks like we've looked you can hear","speaker":null,"is_sponsor":0},{"start_s":786.72,"end_s":792.32,"text":"the fans going though yeah it knows it's doing something it looks like we've","speaker":null,"is_sponsor":0},{"start_s":790.48,"end_s":796.16,"text":"leveled off around 40 gibby bites a second which is yeah it's basically twice what","speaker":null,"is_sponsor":0},{"start_s":794.959,"end_s":801.76,"text":"you could get with zetta fest pretty much that's insane what's to be more","speaker":null,"is_sponsor":0},{"start_s":799.6,"end_s":807.36,"text":"interesting is the rights because in like traditional raid 5 it's really CPU","speaker":null,"is_sponsor":0},{"start_s":804.72,"end_s":811.2,"text":"intensive to write you'll get like a 10th the performance of your read","speaker":null,"is_sponsor":0},{"start_s":808.88,"end_s":815.04,"text":"speed so if that's still good that will be this is grade 0 anyway","speaker":null,"is_sponsor":0},{"start_s":813.519,"end_s":820.399,"text":"though yeah so do we even care should we just switch to right five switch to raid five okay okay","speaker":null,"is_sponsor":0},{"start_s":819.04,"end_s":823.92,"text":"to be clear there are potential disadvantages of","speaker":null,"is_sponsor":0},{"start_s":822.24,"end_s":828.24,"text":"going with the solution like this one of the great things about zfs is its","speaker":null,"is_sponsor":0},{"start_s":826.079,"end_s":832.72,"text":"resiliency we have actually made significant progress big shout out","speaker":null,"is_sponsor":0},{"start_s":830.639,"end_s":837.519,"text":"wendell from level one techs by the way on the data restoration project from our","speaker":null,"is_sponsor":1},{"start_s":835.44,"end_s":841.519,"text":"failed zfs pools and uh we're gonna have an update for","speaker":null,"is_sponsor":0},{"start_s":838.959,"end_s":845.76,"text":"you guys we're down from like 169 million file errors to like six thousand","speaker":null,"is_sponsor":0},{"start_s":844.32,"end_s":850.48,"text":"so make sure you're subscribed so you don't miss that and i cannot necessarily","speaker":null,"is_sponsor":1},{"start_s":848.24,"end_s":853.92,"text":"say the same thing about whatever this is doing well the other thing is we're","speaker":null,"is_sponsor":1},{"start_s":851.76,"end_s":858.16,"text":"also locked into like their ecosystem now this is very like proprietary","speaker":null,"is_sponsor":0},{"start_s":855.6,"end_s":862.959,"text":"software we were able to take these zfs vdevs and pools and just like","speaker":null,"is_sponsor":0},{"start_s":860.8,"end_s":867.36,"text":"import them into trunas yeah even the ones i just did delta one and two those","speaker":null,"is_sponsor":0},{"start_s":865.199,"end_s":873.04,"text":"pools are from like 2015 new hardware new software new version of zfs","speaker":null,"is_sponsor":0},{"start_s":870.72,"end_s":876.88,"text":"i imported those 2016 pools it took like literally 30 minutes for it to import","speaker":null,"is_sponsor":0},{"start_s":874.639,"end_s":881.76,"text":"which is a scary 30 minutes but it just it did it do you want to do raid 5 or","speaker":null,"is_sponsor":0},{"start_s":878.32,"end_s":883.68,"text":"raid 10. raid 5 raid 10 is lame i mean","speaker":null,"is_sponsor":0},{"start_s":881.76,"end_s":887.36,"text":"it might be lame but it's fast well no it's not okay i shouldn't say it's lame","speaker":null,"is_sponsor":0},{"start_s":885.199,"end_s":892.8,"text":"there's a time and a place for raid 10. let me walk you through with raid 10 i","speaker":null,"is_sponsor":0},{"start_s":889.839,"end_s":896.56,"text":"get only 6 drives worth of capacity that's it the rest is all redundant","speaker":null,"is_sponsor":0},{"start_s":895.12,"end_s":901.04,"text":"which is great because all these could fail and i'd still have all my data but","speaker":null,"is_sponsor":0},{"start_s":898.88,"end_s":907.279,"text":"it's bad because that's expensive with raid 5 i get 11 drives worth of data","speaker":null,"is_sponsor":0},{"start_s":904.8,"end_s":911.199,"text":"but i can only sustain one failure on these kinds of solid state enterprise","speaker":null,"is_sponsor":0},{"start_s":909.44,"end_s":915.839,"text":"class devices though they usually all fail at the same time","speaker":null,"is_sponsor":0},{"start_s":913.76,"end_s":919.199,"text":"should be okay yeah you're going to want to have a backup you got a backup that's","speaker":null,"is_sponsor":0},{"start_s":917.92,"end_s":924.48,"text":"for i mean what's our backup within about 30 seconds or two minutes or something like that anyway yeah yeah","speaker":null,"is_sponsor":0},{"start_s":922.72,"end_s":931.6,"text":"we're on our raid 5 array now i'm going to be doing basically the same test","speaker":null,"is_sponsor":0},{"start_s":927.04,"end_s":934.0,"text":"wow me 32 32 qs one meg read sequential","speaker":null,"is_sponsor":0},{"start_s":931.6,"end_s":936.959,"text":"world of warcraft my whole face you see that's it's only 18 gigabytes a second","speaker":null,"is_sponsor":0},{"start_s":935.68,"end_s":940.8,"text":"that's not that bad wait what really 18 gigs a second what's","speaker":null,"is_sponsor":0},{"start_s":939.279,"end_s":947.519,"text":"our CPU usage 1.7 1.9 okay wow but wait now it's","speaker":null,"is_sponsor":0},{"start_s":945.199,"end_s":951.279,"text":"getting faster wait what happened did you know that was gonna happen yeah","speaker":null,"is_sponsor":0},{"start_s":949.199,"end_s":956.959,"text":"oh well i watched it happen earlier it does like two steps okay so it's written","speaker":null,"is_sponsor":0},{"start_s":954.8,"end_s":960.24,"text":"you know like 30 gigs and then it starts going","speaker":null,"is_sponsor":0},{"start_s":959.04,"end_s":967.04,"text":"and then there'll be kind of one more bump where it'll go like above 30 gigabytes a second wow","speaker":null,"is_sponsor":0},{"start_s":964.0,"end_s":969.6,"text":"that's freaking crazy with grade 5.","speaker":null,"is_sponsor":0},{"start_s":967.04,"end_s":973.44,"text":"still at 2.5 in CPU usage there we go 35 gigs a","speaker":null,"is_sponsor":0},{"start_s":972.639,"end_s":978.639,"text":"second we're at almost three percent CPU now remember this is a read test so it'll be","speaker":null,"is_sponsor":0},{"start_s":977.36,"end_s":982.32,"text":"interesting to see what the right is like yeah that's that's pretty quick 35.","speaker":null,"is_sponsor":0},{"start_s":980.72,"end_s":986.0,"text":"that's really fast 35 gigabytes of snapchat switching to this we might","speaker":null,"is_sponsor":0},{"start_s":984.16,"end_s":990.72,"text":"switch to this i want to try to put it in the wanix server because then we can","speaker":null,"is_sponsor":0},{"start_s":987.839,"end_s":994.32,"text":"use the same ssds right yeah that would be even faster but i'd have","speaker":null,"is_sponsor":0},{"start_s":992.32,"end_s":1000.079,"text":"to do that on a weekend that's Jake asking for overtime on camera","speaker":null,"is_sponsor":0},{"start_s":997.36,"end_s":1004.48,"text":"yeah no i don't want to work i have zero desire to do that okay let's","speaker":null,"is_sponsor":0},{"start_s":1003.04,"end_s":1008.56,"text":"try right now because that's really where we're gonna see the difference wow","speaker":null,"is_sponsor":0},{"start_s":1006.32,"end_s":1013.44,"text":"CPU usage is like 20 that's pretty hefty yeah","speaker":null,"is_sponsor":0},{"start_s":1011.12,"end_s":1017.44,"text":"and we're only doing five gigs a second that's not actually that great less","speaker":null,"is_sponsor":0},{"start_s":1015.92,"end_s":1023.04,"text":"impressive so CPU usage is actually going down","speaker":null,"is_sponsor":0},{"start_s":1020.8,"end_s":1028.559,"text":"while performance goes up yeah it's more like 12 CPU right now 11","speaker":null,"is_sponsor":0},{"start_s":1027.28,"end_s":1033.36,"text":"10. it's like it takes a second to like","speaker":null,"is_sponsor":0},{"start_s":1031.679,"end_s":1039.679,"text":"what am i doing where am i where am i putting stuff yeah like it needs a ramp","speaker":null,"is_sponsor":0},{"start_s":1035.36,"end_s":1040.559,"text":"up he's over there i'm coming back","speaker":null,"is_sponsor":0},{"start_s":1039.679,"end_s":1046.959,"text":"okay seems like it's leveled off around nine gigabytes a second with around nine to","speaker":null,"is_sponsor":0},{"start_s":1044.16,"end_s":1050.0,"text":"ten percent CPU usage so pretty good CPU usage","speaker":null,"is_sponsor":0},{"start_s":1047.919,"end_s":1054.0,"text":"still very strange i mean it's very acceptable","speaker":null,"is_sponsor":0},{"start_s":1052.4,"end_s":1059.2,"text":"in terms of performance i mean that's a gibby gibby or maybe bytes","speaker":null,"is_sponsor":0},{"start_s":1057.52,"end_s":1062.16,"text":"so it's probably closer to about 10 gigabytes a second","speaker":null,"is_sponsor":0},{"start_s":1060.64,"end_s":1066.4,"text":"should we try like a random test it'd be interesting to see how many iops because that's another thing software raid will","speaker":null,"is_sponsor":0},{"start_s":1065.039,"end_s":1071.44,"text":"struggle with yeah let's do that look at this guys like those wide stance here man","speaker":null,"is_sponsor":0},{"start_s":1070.16,"end_s":1077.039,"text":"spreading just trying to be a little more ergonomic here this is gonna be 4k","speaker":null,"is_sponsor":0},{"start_s":1074.48,"end_s":1080.48,"text":"random read we're doing 48 threads a little bit more and a q depth of 64 this","speaker":null,"is_sponsor":0},{"start_s":1079.2,"end_s":1084.96,"text":"time okay let's see oh that's that's","speaker":null,"is_sponsor":0},{"start_s":1083.2,"end_s":1090.32,"text":"usage though this is like the absolute","speaker":null,"is_sponsor":0},{"start_s":1087.44,"end_s":1095.2,"text":"most punishing test you can do and we're pulling off","speaker":null,"is_sponsor":0},{"start_s":1091.6,"end_s":1098.4,"text":"six and a half million iops on a raid 5","speaker":null,"is_sponsor":0},{"start_s":1095.2,"end_s":1100.799,"text":"and actually 25 gigs a second at 4k","speaker":null,"is_sponsor":0},{"start_s":1098.4,"end_s":1105.44,"text":"block size holy that's insane","speaker":null,"is_sponsor":0},{"start_s":1102.88,"end_s":1110.32,"text":"oh my god so the theoretical performance of these drives would put us at around","speaker":null,"is_sponsor":0},{"start_s":1107.44,"end_s":1114.0,"text":"12 million iops like raw to each of them that's","speaker":null,"is_sponsor":0},{"start_s":1111.32,"end_s":1116.88,"text":"insane pretty good if we were on an Intel based system we might actually be","speaker":null,"is_sponsor":0},{"start_s":1115.36,"end_s":1122.799,"text":"able to get a little bit more uh or with Intel drives but yeah","speaker":null,"is_sponsor":0},{"start_s":1119.76,"end_s":1123.679,"text":"dang that CPU usage is staying","speaker":null,"is_sponsor":0},{"start_s":1122.799,"end_s":1129.2,"text":"high i can tell you just from the temperature of the backplate though that GPU is at","speaker":null,"is_sponsor":0},{"start_s":1127.28,"end_s":1133.919,"text":"work we can look at it actually so it's at 70 degrees","speaker":null,"is_sponsor":0},{"start_s":1132.24,"end_s":1139.44,"text":"and considering the kind of airflow going over it right now the interesting","speaker":null,"is_sponsor":0},{"start_s":1136.96,"end_s":1143.12,"text":"thing is the GPU usage just it stays at 100 even if you're not using the array","speaker":null,"is_sponsor":0},{"start_s":1141.36,"end_s":1148.32,"text":"it's kind of weird i wonder if that's like uh","speaker":null,"is_sponsor":0},{"start_s":1144.48,"end_s":1150.72,"text":"look the fan is spinning it's 55","speaker":null,"is_sponsor":0},{"start_s":1148.32,"end_s":1154.559,"text":"it just has like ah it's like you you have like a little","speaker":null,"is_sponsor":0},{"start_s":1152.16,"end_s":1159.28,"text":"desk fan inside of like a like a hurricane yeah a wind tunnel just going","speaker":null,"is_sponsor":0},{"start_s":1156.96,"end_s":1163.76,"text":"past it oh thank you for the cooling yeah","speaker":null,"is_sponsor":0},{"start_s":1162.08,"end_s":1167.36,"text":"let's try rights same specs everything else let's just","speaker":null,"is_sponsor":0},{"start_s":1165.28,"end_s":1171.6,"text":"give it a sec to no no sec","speaker":null,"is_sponsor":0},{"start_s":1168.799,"end_s":1176.64,"text":"no sex no i mean i mean that's a camera","speaker":null,"is_sponsor":0},{"start_s":1175.12,"end_s":1180.96,"text":"and that's the kind of operation we're in that's one","speaker":null,"is_sponsor":0},{"start_s":1178.32,"end_s":1186.88,"text":"yeah it's writing though that's way harder so doing one million iops writing","speaker":null,"is_sponsor":0},{"start_s":1184.0,"end_s":1192.08,"text":"so these drives are only rated for 85k random right well that's really wow","speaker":null,"is_sponsor":0},{"start_s":1190.0,"end_s":1198.48,"text":"so that's that's actually almost perfect stealing yeah that's freaking incredible","speaker":null,"is_sponsor":0},{"start_s":1195.36,"end_s":1201.36,"text":"so if we do 85 times 12","speaker":null,"is_sponsor":0},{"start_s":1198.48,"end_s":1205.52,"text":"it's almost perfect scaling this that's probably the most impressive","speaker":null,"is_sponsor":0},{"start_s":1203.039,"end_s":1211.52,"text":"test we've seen so far then yeah that's crazy","speaker":null,"is_sponsor":0},{"start_s":1208.4,"end_s":1213.679,"text":"what's cool about the wrights still","speaker":null,"is_sponsor":0},{"start_s":1211.52,"end_s":1217.28,"text":"maxing out these drives though is that because people are actively editing off","speaker":null,"is_sponsor":0},{"start_s":1215.84,"end_s":1221.679,"text":"of these drives while you are dumping copious amounts of","speaker":null,"is_sponsor":0},{"start_s":1219.52,"end_s":1226.08,"text":"data onto them this could make a huge difference to","speaker":null,"is_sponsor":0},{"start_s":1223.52,"end_s":1230.08,"text":"footage ingest this is freaking crazy we're still gonna","speaker":null,"is_sponsor":0},{"start_s":1227.84,"end_s":1235.76,"text":"run into a huge bottleneck that is samba but maybe once they have smb direct on","speaker":null,"is_sponsor":0},{"start_s":1232.08,"end_s":1237.6,"text":"Linux like rdma support","speaker":null,"is_sponsor":0},{"start_s":1235.76,"end_s":1241.12,"text":"we kind of have to deploy this yeah i think so","speaker":null,"is_sponsor":0},{"start_s":1239.28,"end_s":1244.72,"text":"let's find out if we can oh yeah okay so that's why the bench is","speaker":null,"is_sponsor":0},{"start_s":1243.44,"end_s":1249.679,"text":"here i got a threadripper bench here we're gonna just put that card in it i got us a little NVMe","speaker":null,"is_sponsor":0},{"start_s":1248.799,"end_s":1255.679,"text":"drive we're kind of clashing brands here this is like wearing adidas and nike in the","speaker":null,"is_sponsor":0},{"start_s":1253.039,"end_s":1259.76,"text":"same outfit we got our our liquid drive this has got four NVMe drives on it and","speaker":null,"is_sponsor":0},{"start_s":1257.6,"end_s":1263.84,"text":"then just like a little plx switch so we'll put that in there and then we can","speaker":null,"is_sponsor":0},{"start_s":1261.36,"end_s":1268.559,"text":"raid those four drives okay it's even less sophisticated than i thought yeah","speaker":null,"is_sponsor":0},{"start_s":1266.08,"end_s":1271.12,"text":"it's just that's not on a normal card there's no like it's just this piece of","speaker":null,"is_sponsor":0},{"start_s":1270.159,"end_s":1276.72,"text":"metal so then they just","speaker":null,"is_sponsor":0},{"start_s":1273.28,"end_s":1278.799,"text":"probably sourced a random pci it looks","speaker":null,"is_sponsor":0},{"start_s":1276.72,"end_s":1283.36,"text":"the same as the regular NVIDIA one just because like these slots are the same","speaker":null,"is_sponsor":0},{"start_s":1280.559,"end_s":1287.039,"text":"but there's no cutouts yeah okay sure i'm gonna go get a cooling fan because","speaker":null,"is_sponsor":0},{"start_s":1284.96,"end_s":1291.919,"text":"that looks awful which one the one in there oh he's fine","speaker":null,"is_sponsor":0},{"start_s":1289.44,"end_s":1296.159,"text":"no he's not fine Jake he'll be all right","speaker":null,"is_sponsor":0},{"start_s":1293.28,"end_s":1300.24,"text":"he's a good guy so let's see describe license and see if our license is still","speaker":null,"is_sponsor":0},{"start_s":1297.84,"end_s":1305.679,"text":"valid license is still valid well that's awesome interesting","speaker":null,"is_sponsor":0},{"start_s":1302.72,"end_s":1308.48,"text":"okay yeah it's these are nvmes interesting","speaker":null,"is_sponsor":0},{"start_s":1307.36,"end_s":1312.799,"text":"oh these don't support 4k block size","speaker":null,"is_sponsor":0},{"start_s":1311.919,"end_s":1318.96,"text":"oh do we do we think this is all we really needed to know it boots it runs","speaker":null,"is_sponsor":0},{"start_s":1317.12,"end_s":1324.08,"text":"it probably works let's just boot into Windows and we'll get the last the last","speaker":null,"is_sponsor":0},{"start_s":1321.6,"end_s":1329.52,"text":"answer holy sh it's their Linus what NVIDIA t1000 is device manager it's just","speaker":null,"is_sponsor":0},{"start_s":1326.96,"end_s":1333.2,"text":"a freaking GPU i just want to see if this BIOS version matches any of the","speaker":null,"is_sponsor":0},{"start_s":1331.6,"end_s":1338.559,"text":"BIOS versions in the tech powerup database because if it does chances are oh","speaker":null,"is_sponsor":0},{"start_s":1337.679,"end_s":1346.72,"text":"hello uh yeah i'm pretty sure that just worked","speaker":null,"is_sponsor":0},{"start_s":1342.48,"end_s":1348.799,"text":"gc display connected no way oh my god it","speaker":null,"is_sponsor":0},{"start_s":1346.72,"end_s":1352.4,"text":"just works it's a graphics card well i mean we knew it was a graphics","speaker":null,"is_sponsor":0},{"start_s":1350.559,"end_s":1357.039,"text":"card it's a functional display outputting graphics card","speaker":null,"is_sponsor":0},{"start_s":1355.44,"end_s":1360.4,"text":"okay i got to see this BIOS thing now it's a pny t-1000 so they must be who","speaker":null,"is_sponsor":0},{"start_s":1359.28,"end_s":1367.76,"text":"the uh pny is the exclusive manufacturer of","speaker":null,"is_sponsor":0},{"start_s":1362.64,"end_s":1369.44,"text":"quadro or excuse me NVIDIA rtx","speaker":null,"is_sponsor":0},{"start_s":1367.76,"end_s":1372.96,"text":"works it's exactly the same in every other metric","speaker":null,"is_sponsor":0},{"start_s":1370.799,"end_s":1377.679,"text":"GPU accelerated raid what world are we even living in","speaker":null,"is_sponsor":0},{"start_s":1375.039,"end_s":1382.64,"text":"it's just a regular ass GPU and it's not even a crazy powerful one like this is a","speaker":null,"is_sponsor":0},{"start_s":1380.08,"end_s":1386.96,"text":"basically what like a 1650 yeah something like that literally a 1650","speaker":null,"is_sponsor":0},{"start_s":1384.64,"end_s":1391.36,"text":"same same silicon basically so what can we just run can we just run g-rate on","speaker":null,"is_sponsor":0},{"start_s":1389.6,"end_s":1395.559,"text":"like freaking like a6000","speaker":null,"is_sponsor":0},{"start_s":1395.76,"end_s":1402.159,"text":"so maybe it just completely doesn't care what GPU should we put a different GPU","speaker":null,"is_sponsor":0},{"start_s":1399.919,"end_s":1406.24,"text":"in that should i go get a GPU there's got to be some ballad i'm going to get a","speaker":null,"is_sponsor":0},{"start_s":1404.08,"end_s":1409.84,"text":"gp okay well what i'll be back do we have an a anything i'll just get like a","speaker":null,"is_sponsor":0},{"start_s":1408.08,"end_s":1415.2,"text":"quadro or something or a t would be better touring excuse me jesus","speaker":null,"is_sponsor":0},{"start_s":1413.919,"end_s":1419.36,"text":"it took a little while to generate the rendering kernels but uh","speaker":null,"is_sponsor":0},{"start_s":1416.96,"end_s":1423.76,"text":"it blenders oh i think um i found something it's pretty fast too oh my god","speaker":null,"is_sponsor":0},{"start_s":1422.159,"end_s":1426.159,"text":"are we going to have g-raid check that license","speaker":null,"is_sponsor":0},{"start_s":1426.559,"end_s":1431.44,"text":"so let's see let's first see if","speaker":null,"is_sponsor":0},{"start_s":1429.6,"end_s":1434.799,"text":"the server is okay it's not running on the GPU you see","speaker":null,"is_sponsor":0},{"start_s":1433.52,"end_s":1439.6,"text":"it would it would show the running process so let's see it might not work","speaker":null,"is_sponsor":0},{"start_s":1437.679,"end_s":1445.36,"text":"i don't think it's going to bomber that would be so funny","speaker":null,"is_sponsor":0},{"start_s":1443.039,"end_s":1451.36,"text":"i think the license is done per GPU oh i wonder if it","speaker":null,"is_sponsor":0},{"start_s":1449.2,"end_s":1455.12,"text":"as part of applying the license that binds it to it","speaker":null,"is_sponsor":0},{"start_s":1452.96,"end_s":1458.559,"text":"it's relatively unsophisticated it makes you kind of wonder why they would even","speaker":null,"is_sponsor":0},{"start_s":1456.64,"end_s":1462.88,"text":"bother at that point but money no no i don't mean that like","speaker":null,"is_sponsor":0},{"start_s":1460.72,"end_s":1466.96,"text":"licensing is a waste of time i just mean like it's relatively unsophisticated if","speaker":null,"is_sponsor":0},{"start_s":1464.799,"end_s":1471.84,"text":"you wanted to spoof that yeah you can do it yeah yeah","speaker":null,"is_sponsor":0},{"start_s":1468.24,"end_s":1473.919,"text":"huh oh i'm disappointed i wanted to like","speaker":null,"is_sponsor":0},{"start_s":1471.84,"end_s":1477.679,"text":"throw eight times the compute at it and see what it would do","speaker":null,"is_sponsor":0},{"start_s":1476.0,"end_s":1481.279,"text":"oh well to be fair the game is not launching watch g-raid reach out to us","speaker":null,"is_sponsor":0},{"start_s":1480.0,"end_s":1486.32,"text":"after this and be like yeah we can hook you up with that and there we go continue campaign sure","speaker":null,"is_sponsor":0},{"start_s":1484.72,"end_s":1491.44,"text":"five percent how do i play this uh with a controller this is a good game","speaker":null,"is_sponsor":0},{"start_s":1489.52,"end_s":1495.279,"text":"you should totally play it though it's really there's actually no way to","speaker":null,"is_sponsor":0},{"start_s":1492.799,"end_s":1498.32,"text":"play with the keyboard i don't know it kind of looks like i like wasd or","speaker":null,"is_sponsor":0},{"start_s":1497.279,"end_s":1504.559,"text":"like shift god damn it all right i'm glad i told rocket league to keep downloading","speaker":null,"is_sponsor":0},{"start_s":1502.0,"end_s":1507.279,"text":"this doesn't launch either what the hell yeah the t-1000 is behaving kind of","speaker":null,"is_sponsor":0},{"start_s":1506.0,"end_s":1512.799,"text":"weird but it can game we launched broforce it's fine and you know what else is fine our sponsor","speaker":null,"is_sponsor":0},{"start_s":1511.2,"end_s":1517.039,"text":"thanks to telus for sponsoring today's video we've done a ton of upgrades to my","speaker":null,"is_sponsor":1},{"start_s":1515.039,"end_s":1522.32,"text":"house over the past few months and probably the most important one to me is","speaker":null,"is_sponsor":1},{"start_s":1519.279,"end_s":1524.559,"text":"our new telus pure fiber x connection","speaker":null,"is_sponsor":1},{"start_s":1522.32,"end_s":1528.799,"text":"telus pure fiber x comes with telus's wi-fi 6 access point and can get you","speaker":null,"is_sponsor":1},{"start_s":1526.72,"end_s":1533.679,"text":"download and upload speeds of up to two and a half gigabit per second that is","speaker":null,"is_sponsor":1},{"start_s":1530.96,"end_s":1537.6,"text":"the fastest residential freaking internet speeds that you can get in","speaker":null,"is_sponsor":1},{"start_s":1535.279,"end_s":1541.919,"text":"western canada and is perfect for multiple devices and simultaneous","speaker":null,"is_sponsor":1},{"start_s":1539.76,"end_s":1547.52,"text":"streaming with the new consoles out it means that you can download a new 50","speaker":null,"is_sponsor":1},{"start_s":1544.48,"end_s":1551.6,"text":"Gigabyte game in less than three minutes","speaker":null,"is_sponsor":1},{"start_s":1547.52,"end_s":1553.84,"text":"or a 5 Gigabyte 1080p movie in just 16","speaker":null,"is_sponsor":1},{"start_s":1551.6,"end_s":1557.76,"text":"seconds assuming that the connection on the other side can keep up with you","speaker":null,"is_sponsor":1},{"start_s":1555.76,"end_s":1561.76,"text":"you'll also get to enjoy an upload speed that's up to 25 times faster than","speaker":null,"is_sponsor":1},{"start_s":1560.24,"end_s":1565.679,"text":"competitors which means that your streams will be crystal clear i mean you","speaker":null,"is_sponsor":1},{"start_s":1563.84,"end_s":1570.96,"text":"could be streaming in like freaking 8k at that point so get unparalleled speed","speaker":null,"is_sponsor":1},{"start_s":1568.159,"end_s":1576.24,"text":"on canada's fastest internet technology with telus purefibre x by going to","speaker":null,"is_sponsor":1},{"start_s":1572.96,"end_s":1578.159,"text":"telus.com purefibrex","speaker":null,"is_sponsor":1},{"start_s":1576.24,"end_s":1582.48,"text":"if you guys enjoyed this video this little SSD from liquid here is hardly","speaker":null,"is_sponsor":1},{"start_s":1579.919,"end_s":1588.24,"text":"the most potent we built a server using five i think of their like way bigger","speaker":null,"is_sponsor":0},{"start_s":1585.6,"end_s":1594.799,"text":"eight SSD honey badgers it's called the badger den and it's freaking amazing","speaker":null,"is_sponsor":0},{"start_s":1591.6,"end_s":1594.799,"text":"that's crazy","speaker":null,"is_sponsor":0}],"full_text":"i called it called what okay but i did though i mean sort of i'm pretty sure it was me and i'm also pretty sure it existed before that video even came out it doesn't matter the point is meet the supreme raid sr 1000 it looks like an NVIDIA t1000 workstation GPU in fact it even has the letters t1000 printed on it and the same mini displayport ports are in there but but they're blocked by solid metal that's because this GPU is not meant for graphics and before you say you know where this is going no it's not for cryptocurrency either so what the heck is it through some kind of software funkery g-raid is using this GPU to act as a freaking storage accelerator and if they're to be believed which i'm not sure if i do yet this thing with the right array of NVMe drives can supposedly sustain transfer speeds of over 100 gigabytes per second of sequential throughput holy sh is what you might say if i didn't segue to our sponsor keoxia their bg5 NVMe SSD brings PCIe gen 4 performance to an affordable price for systems and notebooks they even make a 2230 sized one so your pcs can be lighter and smaller than ever check them out at the link in the video description i have so many questions about this but before we can even start to answer them we need a little bit of background combining multiple storage devices has been a staple of computing for decades and generally falls under the umbrella of technologies that we call raid or redundant array of independent disks raid can serve a variety of purposes improving speed data protection capacity or usually some combination of all three of those compared to a single drive now traditionally high performance raid required dedicated co-processors typically found on hardware raid cards you would slot one of those bad boys into your motherboard connect all of your drives to it and it would handle both the high throughput of these many disk arrays as well as the parity calculations that are required by popular raid configurations like raid 5 and raid 6. if you want to learn more we actually have a Techquickie on this subject from almost 10 years ago but raid cards have a problem as we've transitioned from mechanical drives to solid state and then to NVMe storage devices have gotten so fast that raid cards haven't been able to keep pace turning them into a performance bottleneck so the current meta is to connect your storage devices directly to your CPU via pci express this improves both the throughput and latency but requires the CPU to handle those parity calculations and any other overhead this is called software raid and in some ways it's actually kind of a big step backward first of all cpus are freaking expensive and in more ways than you might think a lot of enterprise software is licensed according to how many cpus or how many cores are present in your server so you better believe that big businesses are all about squeezing the absolute most out of every box also cpus are generalized processors i mean you can brute force it here's us hitting 20 gigabytes a second in software raid but the issue is that even with a 32 core epic processor we're looking at a lot of utilization here just to manage storage and if you compare that to the theoretical combined read speed of around 75 gigabytes a second for our 12 kiosks cd6 drives you can see that we were leaving a lot of performance on that table if i'm a server vendor that's too many wasted CPU cycles that my customers now can't allocate to something useful or can't rent out to their customers that is where g-raid comes into play the most obvious difference here right out of the gate is that there is no port to plug a drive into never mind 8 or 16 drives instead the drives connect directly to the CPU's PCIe lanes just like they would with software raid so this server from Gigabyte handles all of that through a backplane here in the front of the chassis then the g-raid card just plugs into any available PCIe gen4 slot we'll do that a little bit later and all the storage communication happens over the PCIe bus no direct connection between our raid card and our drives weird i guess we won't need any of our new cable ties from lttstore.com are available in so many colors and you might be thinking gee Linus even at gen 4 speeds a 16x PCIe slot can only push around 32 gigabytes a second in either direction how could this thing possibly do over 100 that's the special sauce none of the storage data actually goes through the card that's the old way of doing raid cards this card only handles the raid calculations and directing the system where to read and write from all of the actual data flow just goes directly between the drives and the system memory no man in the middle and it does all of this while using barely any CPU resources or so they claim what i think this means is that we could plot this g-raid card into any system and it would just work we are definitely going to try that later but for now we're going to stick to the validated server that Gigabyte sent over for performance testing our cpus are a pair of epic 75 f3 monsters they're only 32 core but they'll hit 280 watts of max power consumption and will boost to 4 gigahertz and then we paired these with some equally monstrous memory micron sent over a metric whack ton of 32 mega transfer per second ecc RAM giving us a total of one terabyte of system memory shouldn't be a bottleneck right i don't think so it should be fine then for our drives we're using kyocia cd6rs they're a well-balanced enterprise gen 4nbma drive and with 12 of them in here we should be looking at raw sequential performance of around 75 gigabytes a second before we can set up an array though we need to install the g-raid supreme raid software and also we need to like finish putting all the RAM in this system it still blows me away how low profile of a cooler they can use for these 280 watt cpus but that's the thing is under this giant heat spreader all the dyes right it's a chiplet design so they're actually like freaking spread out like it's it's huge it's a lot of surface area to transfer that heat brother these are these are 80 watt fans consume 80 watts that's uh each 12 volts 7 amp i think that might be part of the equation wow that is a heavy vapor chamber i love it for how small this thing is like it's a heavy boy oh my god this is a lot of memory it's gorgeous a freaking terabyte why are you whispering because it's a terrible memory i don't want to wake it up as of recording the video this only runs on Linux server operating systems so we're going to be firing it up with do we have an SSD in here or something back here yeah just a little SATA SSD there cool so we're gonna be running ubuntu server 20.04 lts uh Jake has already gone ahead and installed that onto our boot drive as well as the required NVIDIA drivers and supreme raid itself this is cool i am liking this like super over engineered airflow director here pretty sure they just ripped it off of the dell but oh all right that goes hey yeah that's not even close to full send yet either the whole process was surprisingly easy i literally just had to copy paste some commands from their user guide and it looks to be working so i think we can make it a right now should we start with raid zero i'd be obviously have to start with rate zero raid zero is not really a great use case for this because raid zero has no parity data to calculate it's just taking each bit writing it to the next drive in the sequence and attempting to multiply your capacity and your speed you get no extra resiliency whatsoever in fact it's worse because if any one drive fails all the data is gone oh hey look at that you can totally use this with satan sas drives too i don't know if you'd want to because i think the limit is 32 drives per controller thing i don't know what if you could put multiple i guess why why yeah good point why why why it's for NVMe yes okay so here let's see list NVMe drive let's see if they all show up so we got 7.7 it's probably tibi bites or what capacity are those stripes 7.68 oh so then that is terabytes cool there's a bit of an it's not interesting because it's very similar to like zfs but there's kind of a structure so you start with your physical drives you know you got your NVMe drives you can also connect NVMe over fiber drives which is pretty cool and have the controller in this and your your drives and some other jbod pretty sick some are across we're not going to do that yet there is still a limit of 32 drives so it's not like you're going to connect 200 right but 32 with that done you can go ahead and make your drive group which is kind of like a zfs z pool or just right it's like your array you can pick your raid level at this stage uh you can have multiple drive groups i think you can have four um so you could have like say you had 16 drives you could have like four groups of four sure those would all be discrete unlike zfs they're not it's not like having four v devs that you then combine into a pool yeah it's like having four pools yeah let's go back a bit and actually create our physical drives it says create that's the command but really it's like take me take me over we're unbinding it from the operating system and giving it to the g-rate controller there's a cool little feature here you can go like dev slash NVMe 0-11 oh that's cool and it makes them all you don't have to do it one by one there you go made them all successfully and then we can just check the list to see if they're all there ah cool yes this is pretty solid documentation from what i've seen so far compared to the documentation for microsoft storage spaces oh god compared to the documentation for anything microsoft all right create drive group uh we're gonna do raid and then pd id so that's a physical disk so we'll go zero to eleven it's doing stuff let's see if we can see it now g-raid list drive underscore group 92 terabytes that's not bad just like that hey yeah that's fast a little bit of an interesting tidbit here we don't have a usable file system yet this is just the array it's not like zetta fest where there's a file system built in instead we actually have to make a virtual disk so you can make a number of them you could have like a 10 terabyte one you could have like a 50 terabyte one sure we're just gonna make one that's the whole thing this is just block level storage yeah so we'll make big virtual disks that's the full size and then we'll have to put a file system on it as well so let's do that create virtual drive okay so our drive group is zero so we'll say zero and then i'm not gonna specify a size and i think yeah that'll make a full-size one and there we go we have our virtual drive it says it's optimal and it's 92 terabytes okay now i got to make a file system on it i already like like made these commands so i can just copy past them got the file system working i think it was a little bit angry about how i had previously had a file system on these disks and just deleted it made a new one anyways i deleted everything rebooted it created it again now it's happy i also realized that because we now have two dimms per channel previously when i was just kind of tinkering with this i just had 16 sticks in which is eight per one per channel now we have twice that usually that means your memory speed is going to go down fortunately on the Gigabyte servers you can force it to be full speed captain we're doing 32q depth one meg sequential read and that's with 24 threads so two per NVMe drive that's usually pretty standard oh i hear it holy it is straight up just immediately okay it did go down a bit it's it's still twice as fast as what we've seen with CPU raid though yeah look at why ice is fast the CPU usage is only like and it's barely touching the CPU three four percent that might even just be like fio but actually it says it's system stuff so it probably isn't oh that's crazy it looks like we've looked you can hear the fans going though yeah it knows it's doing something it looks like we've leveled off around 40 gibby bites a second which is yeah it's basically twice what you could get with zetta fest pretty much that's insane what's to be more interesting is the rights because in like traditional raid 5 it's really CPU intensive to write you'll get like a 10th the performance of your read speed so if that's still good that will be this is grade 0 anyway though yeah so do we even care should we just switch to right five switch to raid five okay okay to be clear there are potential disadvantages of going with the solution like this one of the great things about zfs is its resiliency we have actually made significant progress big shout out wendell from level one techs by the way on the data restoration project from our failed zfs pools and uh we're gonna have an update for you guys we're down from like 169 million file errors to like six thousand so make sure you're subscribed so you don't miss that and i cannot necessarily say the same thing about whatever this is doing well the other thing is we're also locked into like their ecosystem now this is very like proprietary software we were able to take these zfs vdevs and pools and just like import them into trunas yeah even the ones i just did delta one and two those pools are from like 2015 new hardware new software new version of zfs i imported those 2016 pools it took like literally 30 minutes for it to import which is a scary 30 minutes but it just it did it do you want to do raid 5 or raid 10. raid 5 raid 10 is lame i mean it might be lame but it's fast well no it's not okay i shouldn't say it's lame there's a time and a place for raid 10. let me walk you through with raid 10 i get only 6 drives worth of capacity that's it the rest is all redundant which is great because all these could fail and i'd still have all my data but it's bad because that's expensive with raid 5 i get 11 drives worth of data but i can only sustain one failure on these kinds of solid state enterprise class devices though they usually all fail at the same time should be okay yeah you're going to want to have a backup you got a backup that's for i mean what's our backup within about 30 seconds or two minutes or something like that anyway yeah yeah we're on our raid 5 array now i'm going to be doing basically the same test wow me 32 32 qs one meg read sequential world of warcraft my whole face you see that's it's only 18 gigabytes a second that's not that bad wait what really 18 gigs a second what's our CPU usage 1.7 1.9 okay wow but wait now it's getting faster wait what happened did you know that was gonna happen yeah oh well i watched it happen earlier it does like two steps okay so it's written you know like 30 gigs and then it starts going and then there'll be kind of one more bump where it'll go like above 30 gigabytes a second wow that's freaking crazy with grade 5. still at 2.5 in CPU usage there we go 35 gigs a second we're at almost three percent CPU now remember this is a read test so it'll be interesting to see what the right is like yeah that's that's pretty quick 35. that's really fast 35 gigabytes of snapchat switching to this we might switch to this i want to try to put it in the wanix server because then we can use the same ssds right yeah that would be even faster but i'd have to do that on a weekend that's Jake asking for overtime on camera yeah no i don't want to work i have zero desire to do that okay let's try right now because that's really where we're gonna see the difference wow CPU usage is like 20 that's pretty hefty yeah and we're only doing five gigs a second that's not actually that great less impressive so CPU usage is actually going down while performance goes up yeah it's more like 12 CPU right now 11 10. it's like it takes a second to like what am i doing where am i where am i putting stuff yeah like it needs a ramp up he's over there i'm coming back okay seems like it's leveled off around nine gigabytes a second with around nine to ten percent CPU usage so pretty good CPU usage still very strange i mean it's very acceptable in terms of performance i mean that's a gibby gibby or maybe bytes so it's probably closer to about 10 gigabytes a second should we try like a random test it'd be interesting to see how many iops because that's another thing software raid will struggle with yeah let's do that look at this guys like those wide stance here man spreading just trying to be a little more ergonomic here this is gonna be 4k random read we're doing 48 threads a little bit more and a q depth of 64 this time okay let's see oh that's that's usage though this is like the absolute most punishing test you can do and we're pulling off six and a half million iops on a raid 5 and actually 25 gigs a second at 4k block size holy that's insane oh my god so the theoretical performance of these drives would put us at around 12 million iops like raw to each of them that's insane pretty good if we were on an Intel based system we might actually be able to get a little bit more uh or with Intel drives but yeah dang that CPU usage is staying high i can tell you just from the temperature of the backplate though that GPU is at work we can look at it actually so it's at 70 degrees and considering the kind of airflow going over it right now the interesting thing is the GPU usage just it stays at 100 even if you're not using the array it's kind of weird i wonder if that's like uh look the fan is spinning it's 55 it just has like ah it's like you you have like a little desk fan inside of like a like a hurricane yeah a wind tunnel just going past it oh thank you for the cooling yeah let's try rights same specs everything else let's just give it a sec to no no sec no sex no i mean i mean that's a camera and that's the kind of operation we're in that's one yeah it's writing though that's way harder so doing one million iops writing so these drives are only rated for 85k random right well that's really wow so that's that's actually almost perfect stealing yeah that's freaking incredible so if we do 85 times 12 it's almost perfect scaling this that's probably the most impressive test we've seen so far then yeah that's crazy what's cool about the wrights still maxing out these drives though is that because people are actively editing off of these drives while you are dumping copious amounts of data onto them this could make a huge difference to footage ingest this is freaking crazy we're still gonna run into a huge bottleneck that is samba but maybe once they have smb direct on Linux like rdma support we kind of have to deploy this yeah i think so let's find out if we can oh yeah okay so that's why the bench is here i got a threadripper bench here we're gonna just put that card in it i got us a little NVMe drive we're kind of clashing brands here this is like wearing adidas and nike in the same outfit we got our our liquid drive this has got four NVMe drives on it and then just like a little plx switch so we'll put that in there and then we can raid those four drives okay it's even less sophisticated than i thought yeah it's just that's not on a normal card there's no like it's just this piece of metal so then they just probably sourced a random pci it looks the same as the regular NVIDIA one just because like these slots are the same but there's no cutouts yeah okay sure i'm gonna go get a cooling fan because that looks awful which one the one in there oh he's fine no he's not fine Jake he'll be all right he's a good guy so let's see describe license and see if our license is still valid license is still valid well that's awesome interesting okay yeah it's these are nvmes interesting oh these don't support 4k block size oh do we do we think this is all we really needed to know it boots it runs it probably works let's just boot into Windows and we'll get the last the last answer holy sh it's their Linus what NVIDIA t1000 is device manager it's just a freaking GPU i just want to see if this BIOS version matches any of the BIOS versions in the tech powerup database because if it does chances are oh hello uh yeah i'm pretty sure that just worked gc display connected no way oh my god it just works it's a graphics card well i mean we knew it was a graphics card it's a functional display outputting graphics card okay i got to see this BIOS thing now it's a pny t-1000 so they must be who the uh pny is the exclusive manufacturer of quadro or excuse me NVIDIA rtx works it's exactly the same in every other metric GPU accelerated raid what world are we even living in it's just a regular ass GPU and it's not even a crazy powerful one like this is a basically what like a 1650 yeah something like that literally a 1650 same same silicon basically so what can we just run can we just run g-rate on like freaking like a6000 so maybe it just completely doesn't care what GPU should we put a different GPU in that should i go get a GPU there's got to be some ballad i'm going to get a gp okay well what i'll be back do we have an a anything i'll just get like a quadro or something or a t would be better touring excuse me jesus it took a little while to generate the rendering kernels but uh it blenders oh i think um i found something it's pretty fast too oh my god are we going to have g-raid check that license so let's see let's first see if the server is okay it's not running on the GPU you see it would it would show the running process so let's see it might not work i don't think it's going to bomber that would be so funny i think the license is done per GPU oh i wonder if it as part of applying the license that binds it to it it's relatively unsophisticated it makes you kind of wonder why they would even bother at that point but money no no i don't mean that like licensing is a waste of time i just mean like it's relatively unsophisticated if you wanted to spoof that yeah you can do it yeah yeah huh oh i'm disappointed i wanted to like throw eight times the compute at it and see what it would do oh well to be fair the game is not launching watch g-raid reach out to us after this and be like yeah we can hook you up with that and there we go continue campaign sure five percent how do i play this uh with a controller this is a good game you should totally play it though it's really there's actually no way to play with the keyboard i don't know it kind of looks like i like wasd or like shift god damn it all right i'm glad i told rocket league to keep downloading this doesn't launch either what the hell yeah the t-1000 is behaving kind of weird but it can game we launched broforce it's fine and you know what else is fine our sponsor thanks to telus for sponsoring today's video we've done a ton of upgrades to my house over the past few months and probably the most important one to me is our new telus pure fiber x connection telus pure fiber x comes with telus's wi-fi 6 access point and can get you download and upload speeds of up to two and a half gigabit per second that is the fastest residential freaking internet speeds that you can get in western canada and is perfect for multiple devices and simultaneous streaming with the new consoles out it means that you can download a new 50 Gigabyte game in less than three minutes or a 5 Gigabyte 1080p movie in just 16 seconds assuming that the connection on the other side can keep up with you you'll also get to enjoy an upload speed that's up to 25 times faster than competitors which means that your streams will be crystal clear i mean you could be streaming in like freaking 8k at that point so get unparalleled speed on canada's fastest internet technology with telus purefibre x by going to telus.com purefibrex if you guys enjoyed this video this little SSD from liquid here is hardly the most potent we built a server using five i think of their like way bigger eight SSD honey badgers it's called the badger den and it's freaking amazing that's crazy"}