this might look like a GPU smell like a
GPU stick in a PCI Express 16x slot like
a GPU and even talk like a GPU look at
me I'm a little GPU but I assure you its intended purpose is very different and
to some potentially terrifying it's
dubbed the ASUS iot AI accelerator PCIe
card or crl dg16 new- P3 DF for short
and it's designed for artificial intelligence comput ation
ooh historically we've mostly shied away
from covering AI stuff and there's a couple of good reasons for it one is
we're not AI or machine learning developers so finding practical use
cases that we are able to set up and demo for you guys can be a little tricky
and second a lot of those use cases are
extremely technical behind the scenes kinds of things at least for now so
Hardware like this isn't too applicable to end users like me and you that is
until today while doing research for and testing the home automation setup at my
new house get subscribed by the way you don't want to miss any future content
around revamping that place we stumbled into a bit of a roadblock regarding
presence detection or the ability for the systems in the house to be aware of
whether or not anyone is actually home presence detection provides us with a
number of benefits for example we can turn off lights and set the HVAC to be
more energy efficient when there's nobody at home and this is relatively
simple to implement on a whole house level but therein lies the challenge we
put a lot of effort into making my house a lot more granularly controllable
practically every room has its own independent HVAC and lighting which is
awesome for personal preferences like the kids might want their rooms cooler
than my wife and I do for example or vice versa but it can also be used to
improve efficiency and save cost if no one's in the giant wreck room downstairs
what's the point of heating and cooling it I mean surely that can't be that hard
to automate well as we found out it can
be really freaking hard one idea we had was to install Bluetooth low energy
beacons in every room and read people's presence through their phones the only
problem with that is that people have this tendency when they're at home to
put their phone on a charger and oh I don't know walk around their house so
that won't work another idea was to use motion sensors but those kind of crap
the bed as soon as you decide to take a nap on the couch and then you wake up in
a sauna of a room in the summer which sounds like a pretty bad time to me so
this is where our AI card comes into
play this is an ASUS designed product but all of the important AI bits are
actually made by Google of all people under the brand Coral we've got both of
them linked down in the description below let's take this thing apart this
specific card has 16 onboard Edge tpus
as Google calls them and the TPU stands for tensor Processing Unit in the most
simple terms these tpus are Hardware processors that are physically designed
and optimized to run a specific application you may have heard of
something called an Asic as something that's really good at mining as specific
cryptocurrency well these are the same idea except that in instead of mining
Bitcoin they're for running machine learning inference calculations so what
we're looking at here is essentially just a PCI Express 16x interface here
going you can actually see the traces going right into this PCI Express switch
which appears to be just splitting out
these Lanes into an m.2 2x interface for each one of these
little dual tpus here huh I I was
expecting something less kind of modular
and science fair projecty now each of the EDG tpus of which there are two on
these m.2 cards can do around 4 trillion
operations per second with a power draw of only two Watts so 2 Watts * 16 tpus
brings us to 32 watts for the card you add in your PCIe switch and fans and
that brings you to 52 Watts which is
well within the 75 Watts that this PCI Express slot could power but for some
reason it still requires a six pin power connector and it's got this gigantic
copper fin cooler on it now you're probably thinking wow that's really cool
Linus but most of this sounds like the boring technical stuff that you were
talking about earlier but if you were following closely while we were
researching better ways to detect presence in my new house we stumbled
upon some software called frig that uses
cameras plus this Hardware to really
efficiently detect presence it's really freaking cool let's take it for a test
drive I got to put this back together and we're up this test bench is running
the latest version of UNR and it's going to stand in for my home NZ now you might
notice there is also a GPU installed in
it so we need that as a video output because this is a non Apu ryzen
processor but it's also useful for offloading the camera video decode
process from the CPU now in my home deployment I'll probably just use the
CPU because I'm going to have 24 epic cores that are otherwise going to go
unused but for now this is the better solution
now once un raid's booted up you can see that we have just a single array disc
with no redundancy that's not what I'd recommend as a production deployment but
it's good enough for this video now we just pop over to the community apps Tab
and installed apps so we've got our Coral accelerator module drivers we
don't need this if we're using a USB version of Google's AI processor we only
need it for the PCI Express version and then we've also got our NVIDIA drivers
which are for our GPU now it should be noted that you can use your CPU for frig
but performance is not good next we
install frig now we can't just launch it we've got to actually configure frig
otherwise it's going to have no idea what devices it's supposed to be using
so step one we're going to copy the GPU uu ID from our NVIDIA driver and paste
that there then for our TPU mapping we've mapped it to our Apex or0 device
and that is only one of the 16 tpus that
are on that card because we're only going to have a couple of cameras we're
not going to bother mapping all of them but we did map five of them so that that
should be enough for our demo for now wow this is a great password Jake I like
it I see I don't get overwhelmed most of
this is included in the template but there are a couple things that we had to
tweak so one is we had to set up our camera so you can see we've pulled our
camera from The Writer's den and we've got our network path for it right here
we're just using the one camera for now we actually didn't take all five of our
npus we've just got four of them configured here again this is total
overkill for what we're doing and then we've also got this right here so we are
limiting our frames per second for the camera feed that we're using to 5 FPS
that is all we're going to need in 99% of situations to have functional
detection and adding more processing to this would serve no real purpose one
other critical thing back up at the top here is that you can Plum this into an
mqtt server which is basically its way of communicating back to your other home
automation devices So This mqtt Server 10.2.0 71 is running on our office home
assistant instance hey 5 FPS of green I can see why you said we didn't need more
FPS there's not a whole lot of movement there it's peanut butter demo time
peanut butter demo time peanut butter demo time where you at where you at so
that one camera is set up we have another one too but let's just okay hi
there oh wait hold on I screwed up oh all right do we not have the detection
on uh we but I I didn't turn on like the the bounding boxes bounding boxes are
just little boxes that show us what the
software is detecting and the degree of certainty that it has for what that
object is humans aren't the only thing that it can detect but they're kind of
the only thing that we care about for home automation presence detection
you're telling me we don't have to detect how many apples you have in your fruit bowl oh yeah oh yeah oh you were
there buddy oh it's kind of detecting your feet does it know who I am that
part we have not done yet what if it only sees like this of me yeah it still
thinks you're there what if my legs show
oh it lost you it lost me what about a hand oh yeah it sees you again no really
okay what if it just saw an elbow oh yeah it's detecting you what really yeah
okay can it can it see me it's confused right now it's a little confused what
about if there's just like a casual like hand there it's kind of freaking out so
sometimes it actually sees two people um when he's kind of obscured but that's
totally fine oh like it thinks I might be like two people having a sex orgy
behind the curtain uh yeah sure but it's
fine because all we need to know is if there's anyone right right yeah and sex
orgy aside you don't want the heat to cut out during your sex orgy every get
cold man you can do so much cool stuff with this but wait there's more what if
you have a common area like a kitchen or a recck room and you want to be able to
apply personalized preferences when you're present like say have your
Spotify music with your specific playlist play when you're in the room
and not when ivonne's in the room right oh I never even thought of that you
could walk around the house and have Spotify playing here oh I'm over here
now and now there's my Spotify over here oh my God that would be so cool turns
out someone had that exact thought and created Double Take which is a piece of
software that allows you to determine the face this is more
creepy yeah but it's all local I mean we
could just configure this to not log any of this right yeah did a Linus appear oh
wow yeah nice now Double Take isn't available on the unraid community app
store at least not yet so we need to manually set up the docker container
ourselves then once it's set up we need to configure it to listen to frig it
over mqtt it should be noted that double take itself doesn't do facial
recognition it's just an API and interface to make hooking into and
training facial recognition software easier so the first thing we need to do
is grab some pictures of me which are going to be training data for our power
facial recognition so I guess I need to go over there right uh we could go over
there or we can just upload some photos we already have Double T currently supports deep stack compre face and face
box they're all machine learning based facial recognition models deep stack is
on the community App Store so that's the easiest to set up but I tried it out and
it wasn't really that great with the limited training data I had compri face
on the other hand it requires Docker compos have to run in a VM but it's
scary good I put a couple photos of lonus and secretly kind of had him walk
by without telling him and it was like lus and then I walked by it was like
Jake with like 10 photos of each of us all right let's try it why do you have
so many pictures of me on your hard drive well you make my my drive hard why
CPU when I can see you pe why talk to you when I can leave can I zoom how do I
use a Mac do you know no these Stills are horrible it's scary good Lord to see
you without a beard oh wow yeah yeah yeah oh my God 95
5.8% how can it be that sure on unbearded Linus training there was only
one that it is actually confident about I mean 94% seems pretty damn confident
to me we can do is take all of these yeah and say train off of those data now
you want to walk over there again yeah for sure okay that's plenty there's so
many luses now Linus Linus Linus Linus there's a couple times it thinks it's
James I think like once you trained a good model with compr face I'd probably
say that like it would only accept it as as that person person unless the
confidence was like 99% some of these other models Madison I also made with
people's photos with them wearing a mask so it like literally has the eyes okay
go back run over there all right I'm going I'm going I'm going see if it makes more mistakes I'm going to pull up
a chair lonus style it's very confident
about lonus now what who else would
behave like this I don't I don't think that's what it's looking for but man so
you could just kind of wander around your house for 10 minutes find any
anything that's wrong boom train it part of it is also going to be specific
cameras because you know each camera looks a little different one thing I
asked Jake about and he said it doesn't have this functionality for now is can
it morph over time and can it continue to self-train the thing with facial
recognition is like the more data you feed it the more uncertain it becomes
all of this leaves us with some questions do you even need this thing
could a bog standard CPU do the same calculations the answer is yes it could
but way slower to put it in perspective
your average quad core CPU would probably only be able to handle a few
frames per second on a quad core but
because these Edge tpus are so optimized
for running these specific calculations they are stupid fast at them a single
Coral Edge TPU can handle around 100 frames per second of people detection in
frig and we've got 16 of them so if my
rough math is correct that that's around 1,600 frames per second we might
actually be pushing that once we've got all the cameras in the new place we're only taking 5 FPS from each still it's a
lot of cameras No No 5 FPS yeah five okay yeah man that's crazy so a single
one of these tpus could probably handle 15 to 20 cameras at 5 FPS another
question this raises is are we worried about the security implications
fortunately since we have all of our own hardware and all of this is open source
and locally hosted none the data goes
outside of my house so I'm actually pretty happy with the level of security
and safety as for being worried about AI
taking over no one's asking that question it's in here anyway the answer
is no leaving only one question then who's our sponsor freshbooks freshbooks
is the simple accounting solution that is designed specifically with you in
mind the small business owner it features built-in automation that allows
you to spend less time tracking projects and more time growing your business so
whether you're a trades person creative agency or a YouTuber you can choose the
plan that's right for you with freshbooks they have an award-winning
Toronto based support team who are always happy to help you if you need it
and you don't have to take my word for it you can try fresh books for free for
30 days today with no credit card required at freshbooks.com
Linus if you enjoyed this video maybe go check out the video where we built the
server that is going to be running all this stuff we're going to have to get a little bit creative with PCI Express
expansion on it um but that's the machine that's going to somehow run this
full height card yeah yeah so good luck everybody are we going to do an external
PCI box we should totally do it yeah we should totally do it yeah that'll be a
video get subscribed we can water cool it and everything