now we've yet to actually boot this motherboard
with six graphics cards this could be the thing that kills the
entire project
the whole system restarted when i bumped that external pci express cable my level
of pissed off right now is pretty high that's not what i was expecting
story blocks video offers you studio quality clips at a fraction of the cost
check them out today at the link in the video description
so we found a pattern vm2 and vm6
are not working and honestly i'm at the point now where all i can really do is
give up on two and six in their current configurations and we're gonna try
swapping out the graphics cards because we finally got an error and it was
something to do with the onboard audio controller for the graphics cards makes
no sense never seen this before on any pascal card especially a founder's
edition so here's a titan v which is a good
choice because this is what we'd like to use for the final build anyway because
it has single slot i o although we have to convince NVIDIA to send us six of
them and they're like this is not a supported configuration for the titan v
i'm like yeah yeah i know people are gonna know that
anyway our quadro systems are working fine too so we're going to swap out our
1070 and our titan x which conveniently are right on the top of the pile here
with these two puppies
i could do that a thousand times and i'd never stop dropping things and i'd never
stop wearing sandals okay i think it's booted
so it picked that up so that's fine and six
needs a GPU as well and it picked up that one which might be
fine oh sound card d9
okay new update we swapped in two new graphics cards and then we did end up
having to disable their audio controllers which is weird but then one
of them worked so we've actually got five lit up right now so it's just the
last one that's not firing up so i'm wondering if maybe it's just like my
DisplayPort to HDMI cable or something no i don't think so because num lock's
not working on here so we're gonna throw an rtx 2080 in here
because nothing's worked yet
i really doubt it's gonna fire up though no
yeah it lit up all right let's try the other five no
i'm not actually doing it i'm not actually doing it i want to see all of them up
i know it's normal when you're installing display drivers for the screen to go blank but i'm just
very on edge about this
num lock's not working you have got to be kidding me
yeah five didn't fire up why
what is the symbolism okay well six fired up but it's in
Windows recovery nonsense mode
all the usb devices are there and
and now it's off bastard it's trying so hard
you're just capturing my sadness over here yeah thanks
so after sleeping on it i thought maybe we need to get a little bit back to
basics here i pulled out all the graphics cards and
re-ran the utility called hw lock
so something's going on with pci express that's causing some kind of nonsense in
the system it was causing hw lock to crash and it was causing our vms not to load
now with our new BIOS which took forever to even start posting now what we can do
is start to slowly re-add hardware
until we reach a problem
everything's in it should be fine
NVIDIA confirmed the shipment of our six titan v's for this project
there seems to have been a mistake because i never had the briefing call
where i you know came up with some real workloads to use them with
you know like realistically they were going to support this cool project they
just jumped the gun a little that's all
now i do want to get in here and turn on above 4g decoding and it's weird because
i've had a really hard time getting into the BIOS ever since i updated it like i
just go straight from this pre-BIOS loading thing to
boom unraid's firing up
oh there's a ps2 port maybe i should plug into that
the saga continues
it's erroring out on probably pci resources allocation
but instead of letting me go into the BIOS and reconfigure that it's just hard
resetting you
bastard it's funny i remember sending ASUS a
memo about this board saying this is the best behaved dual socket board i've yet
seen to issue a retraction
i mean it's not garbage it's like really cool fancy stuff but like
i need it to work come on
just output
new plan we pull out all the gpus again
and we reconfigure the pci 4g decoding
thing with only the onboard connected
go just go
instead of showing me this whole bar and showing me all the postcodes it just
shows me the first one it's not working correctly
turned off
come on
yes okay
so we are on the latest BIOS
let's just go through here and see what all we need to change
disable the serial port CPU storage nope
on board lan disabled
ASUS engine booster above 4g decoding on
okay see i don't know what aspm does we don't need any of these operons
never extract csm i mean maybe if we just start
disabling devices that'll help
this is it everything's reconfigured which means we
need to plug these SATA devices into the other controller
now we are going to add graphics cards
let's just throw them all in i mean why not yellow it again that worked out so
well for us last time guys how many times do you live
i don't make everything hard i mean just ask the audience i'm sure tons of them
are flaccid right now
okay so we're back to square i don't know what square this is but
we're back to actually being able to boot on raid see
a display off of the onboard vga and we have all six graphics cards installed in
the system what square is that
hip square you know like it's hip to be i'm gonna go get my laptop
so here's what we're gonna do we're gonna edit
all of our vms to
not have oh well they're gone so that's convenient okay they don't have their
usb controllers because those aren't plugged in
so can we start them which one's this four let's start number
four i expect this to turn on
i'm using the power of positive speech oh
crap domain id3 is tainted this is not a good
indicator right now because if it's the graphics cards that are our problem we
are a lot further away from this project working than i hoped
okay so it fired up but i don't see anything here yet which is also
not a good indicator because remember four worked before
maybe turning my mom's around again yeah we can try that
i don't think so though oh boy
okay good work david uh okay
starting numero 6 internal error yeah once again it is
flipping out about the audio crap that's
bad what the hell
okay i'm really screwing this up right now
okay let's try two just prefer the lulls
device eight f zero zero whatever the crap what do you
mean af whatever what what is this okay did it change the is that the GPU
it changed my GPU ids oh crap so ah man which one it okay
forget it we're just going to create a new one however much memory it doesn't
matter right now let's just get it working primary disk location manual
graphics card titan v create
all right let's see if that worked oh maybe actually yes
so there it is keyboard and mouse obviously aren't working right now
because i don't have my usb controller plugged in but
okay well let's keep going then i guess
uh oh dang it well
two's not up unless it is when i turn the monitor
back on new plan
we're gonna try two and six as the only things plugged into the system
okay i got
ls toppo to run so the difference between our new ls toppo run and the old
one is now i have all six gpus installed
in the system so what that allows us to do
if this image viewing program wasn't completely worthless
sure good enough what this allows us to do is figure out
which gpus are the problem so we can see here
six is our titan x and two is our titan
v so then if we go into system devices
this tells us that 1d81
right here and then our titan x
those are our problem gpus okay
so this confirms a few things number one is it confirms
that my map of the pci express slots is probably right because slot two and slot
three which is where those are one of them should be CPU two i believe that's
slot two and one of them should be CPU one i believe that's slot three and
actually i checked that that's right so it confirms that
and it confirms why i think
four was fine and six is bad both of them are the only subordinate
devices on a plx bridge
and those are the ones i'm having issues with so
it is my belief that either the solution is UEFI tuning so i've drafted an email
to ASUS explaining that these are my problems
or a new UEFI i remember how i started that sentence
my brain is very tired um we're i'm
gonna send them an email hopefully it'll be coherent in the meantime though
now what i haven't been having problems with
is my u.2 to pci express
x4 adapters
like this really is just for testing i intend to populate these
two now graphics card can draw up to 75
watts through the pci express slot itself so obviously
this SATA to power adapter thing like look at these pin or wires on this thing
but we are not going to be loading this thing anyway i just want to see them
work i don't need to run doom for 24
hours straight or anything like that all right so
here goes one and here goes
two
so bad so i'm going to put this
here so we're only getting four lanes but
that's four lanes PCIe gen three like that's not bad
does anyone else have really mixed feelings when they
you know finally reached the end of a problem like where part of them is relieved and
happy and the other part of them is just like pissed off that it took so long
if the cringing was real last time
now it's reached a level of hyper realism that we thought only possible
with rtx okay so i need this
to reach in through here
ah yeah
this seems fine so step number one is finding out if my
gpus are even working in those slots i mean
theoretically there's no real difference between this
and this they're just pci express over a cable but you never know
titan v boom there it is and one b zero zero
this one's the same so that's our titan x so they're there
so let's try starting it started right away
i don't know if we have a display output though
well would you look at that not only is it working but what's really
odd is it's running in the correct resolution let's see if it survives a reboot here
need i remind you guys that this is the GPU that is plugged into an extension
plugged into a daughter board is vertical in the case and completely
unsupported by anything like if this works it's a small christmas miracle
hey are you working or what
same behavior
let's try the titan v let's see what happens oh oh oh oh oh
okay yay obtain it's going fast at least wow
it's going really fast now it slowed down and of course it's 100 complete and
yet here we are progress bars man how are they this broken
i'm gonna see if i can reboot that other one in the meantime
so i ran ls topo again and let's have a look at what our topology is now so
here's those two devices we're still having issues why
why those two specifically we're actually moving backwards we're losing
vms now four no longer boots i'm gonna try uh
three and one as well one's up five's up
three's up i don't know if they're gonna stay up
five three and one let's try stopping them
and then we'll try rebooting them that's the real test
and none of them have a signal
okay then okay
i had really hoped these adapters were going to be the solution
we are going to put our gpus
in this thing like there's nothing
that would lead me to believe that this will be better than using those NVMe
adapters blah all right maybe we can shove this
over to get that HDMI cable in there yep there we go okay so that's in let's go
ahead and throw our expansion card doodad
back in here oh ballsack which slot was i think it's the bottom one that i use
okay ah
why is this keyboard powered up oh these cards have power even though
they don't have a data signal did that just shut down and reboot
sounds good okay
that's the pain train it's leaving the station
so the configuration's a complete cluster at this point the two that i'm
really worried about are the titan x in vm number two and the titan v and
number six so why don't we do those first all right so they're both up but
let's find out if they're going to stay up this has a blinking cursor so that'll
tell us if this freezes mind you if it behaves like before it'll probably just
go away did that just go away
oh whoa whoa whoa whoa whoa whoa whoa whoa
this one's up i'm gonna shut it down and reboot it
is it doing a graceful shutdown okay so this is still working i can tell
because the system clock is still right this one finished its update while i
wasn't looking we're gonna rerun our command here
did we just crash oh yeah that bm's gone
we're not going hail mary anymore we're not going for glory so
we're gonna go back to basics and we're just gonna start with one then we're
gonna put in two and three and four we're gonna see where we crap
out and see if we can find a pattern here so after sleeping on it i thought
maybe we need to get a little bit back to basics here now what we can do is
start to slowly re-add hardware
until we reach a problem the is that
this is honestly starting to feel a little bit like a journal that i'm
keeping on a on a desert island where i'm the only survivor or some garbage
like that like so while the camera was off last night we
went back to unraid 6.5.3
just in case the newer Linux kernel with all the spectre and meltdown nonsense
that's going on was was causing some problems that was a suggestion from the
unraid guys and i went down to one card and it was working
then i added a second card and boom already flaky behavior
i don't even i don't even know if that was one of the slots i was having trouble with before so this is this is
like a big problem and now that i had all the mapping information i was
definitely assigning gpus to the correct
CPU course so we weren't having to cross between the cpus in order to get to the
pci express lanes so that got me thinking
what if the problem is hardware now ivan
is a competent pc technician
but he doesn't have experience with lga 3647
and it occurred to me that he was the one who installed my
cpus so that got me thinking
what if we just have a bent pin in the socket
so
it was a little piece of debris in the socket but
it seems to be clear now all right so there's one
now let's try the other one no obvious issues
i should put them in the opposite sockets just to see if like
maybe i start having problems with like different slots or something
you know
well so much for that theory i mean you never
know when you know reseeding can just
magically solve something though so still
i'm not very hopeful but you never know
so this is interesting we just got a kernel panic again
this time it's CPU one
processor id one it is still possible
that one of my you know ten thousand dollar cpus is
defective but i'm really leaning towards like motherboard
BIOS optimizations that are
wonky at this point we're not getting away from these hard resets
so my idea just now was do a fresh install to that vm and see if we can
avoid whatever devices it's trying to install drivers for
so that all is behaving completely normally
wouldn't that be a weird thing
if it spazzed out over the copied vdisks but then
getting ready i don't know about you computer but i've
been ready for a while here
hey uh
what what's going on
hi
we have a Windows desktop
on machine number two
how did this happen so this is how it's going to be for now
we finally got machine 2
working we think but Windows updates
are not having any of it storyblocks video gets you studio
quality video clips at a fraction of the cost you can download all the stock
video that your heart desires from their member library including hd and
4k footage after effects templates motion backgrounds and more we actually
use it quite a bit here on our tech wiki channel plus you can get exclusive
discounts on millions of additional marketplace clips you'll save 40 on the
purchase compared to non-members and what's really cool is the original artist will take a commission on each
sale all the content is royalty free so you can use it even for commercial use
to your heart's content and for personal projects like youtube videos for example
and new clips are added regularly so there's always something new to check
out so check it out today at the link below so thanks for watching guys
if you disliked this video then quite frankly you and i are in the same boat
because i did not have a good time over the last few days working on this and
i'm not looking forward to continuing to work on it but if you liked it hit like
get subscribed and maybe consider checking out where to buy the stuff we featured at the link in the video
description but i wouldn't recommend buying it yet because we haven't quite determined if any of this works also
down there is our merch store which has cool shirts like this one and our community forum which you should
definitely join