I Hope Google Doesn’t Ban Us... - Abusing Unlimited Google Drive
Linus Tech Tips
·Linus Tech Tips
·2019-05-06
·
2,171 words · ~10 min read
0:00
Just over a year ago,
0:01
we deployed an insane solution to our storage problem.
0:06
The petabyte project.
0:08
A petabyte of raw archival storage.
0:11
I mean, that thing is freaking awesome.
0:13
We have yet to have any problems with it,
0:16
but there's still a major issue with this deployment.
0:20
It's not geo-redundant at all,
0:23
which means that if there were a fire
0:25
in the Linus Media Group warehouse,
0:27
or I mean really any kind of natural disaster,
0:30
we could potentially lose our entire back catalog.
0:34
That is anything that we haven't already uploaded to YouTube.
0:38
So we endeavored to find a way to,
0:41
as inexpensively as possible,
0:44
back up the entire server.
0:46
That's all 370 terabytes of currently stored data,
0:52
plus anything we would wanna add in the future.
0:55
So that's easy enough, right?
0:58
You know what else is easy?
1:02
Your move over to a better chair,
1:06
like the ones from Noble Chairs.
1:08
Their icon series real leather edition gaming chair
1:10
was designed in Germany
1:11
and inspired by luxury car interiors.
1:13
Learn more at the link below.
1:15
Just hope they weren't inspired by like the trunk
1:17
of a luxury car, because that'd be very uncomfortable.
1:29
Now there are a lot of different ways
1:30
that we could go about backing up our server.
1:32
Some of them easier and some of them harder than others,
1:35
but almost all of them are gonna be limited
1:38
by water.
1:39
One thing, our internet bandwidth.
1:42
I mean, don't get me wrong.
1:42
We have a symmetric gigabit connection.
1:44
I'm not complaining,
1:45
but when you're talking about hundreds of terabytes
1:49
that are already in the archive,
1:50
and then up to a couple of terabytes a day being added,
1:55
things can start to take a while, like months a while.
1:59
Now, our first idea was to build something locally,
2:03
fill it up with data here using our 10 gigabit network,
2:07
and then ship it off site.
2:09
A sort of like, ah, thank you, Jake.
2:11
Oh, wow.
2:12
Petabyte project part two.
2:15
Now, this solution wouldn't actually
2:18
expand our storage capacity, because it would just
2:21
be an identical server, or that is to say two servers,
2:25
with the same data on it.
2:27
But since our internet service provider offers us
2:30
inexpensive co-location, it would actually
2:33
be very cost effective from an ongoing standpoint.
2:37
Except for one small problem.
2:40
If we had to buy two more decked out 60 drive store-inators,
2:45
we'd be looking at, with the drives included,
2:48
upwards of 70,000 US dollars.
2:53
So, converted to Canadian rubles,
2:55
that is getting dangerously close to six figure territory.
3:00
You can take that away.
3:01
So, that eliminates hard drives.
3:04
You good?
3:05
Yeah.
3:06
Okay, just remember, lift with your back,
3:07
the twisting, jerking motion.
3:08
Yeah, I was working on that.
3:10
So, that eliminates hard drives.
3:12
What about tape storage?
3:14
A rack mount tape library, like this one from HP Enterprise,
3:18
only costs about five to $7,000.
3:21
And high capacity 12 terabyte LTO8 tapes
3:24
are about 160 bucks a piece.
3:27
So, for over 500 terabytes of uncompressed backup capacity,
3:33
that's actually looking pretty good by comparison.
3:36
But,
3:37
there's the fact that we'd need two of them,
3:39
and it looks like we're still spending
3:42
well over 20,000 US dollars up front.
3:47
Lump sum, throw that on the black card.
3:49
You know how it is.
3:51
So, hardware-based solutions are not looking too peachy.
3:56
What about, shoot, what's it called?
3:59
It's kind of like buzzword-y, disruptive, blockchain.
4:04
No.
4:05
Oh, oh, oh, cloud!
4:06
Cloud storage.
4:08
You guys were all shrieking at us
4:11
back when we originally built Petabyte Project
4:13
for not using cloud storage for our archive
4:17
in the first place.
4:18
So, I think this is where you come in.
4:21
We're gonna check out some AWS here, there, buddy.
4:24
This is Glacier, so this is like slow.
4:28
Start the download.
4:30
Okay, so if we just wanted to back up everything
4:31
we've got right now.
4:32
How many-ish terabytes?
4:34
$1,500 a month.
4:36
Wow.
4:38
Plus support.
4:38
Oh, yeah.
4:39
You know, you need support.
4:40
Yeah.
4:41
1,667.
4:42
So, if we wanted to store the full, what is it?
4:46
About 760 terabytes that we have?
4:48
Something like that, that we could have.
4:50
Holy!
4:51
$3,400 a month!
4:56
I mean, I get it.
4:57
I guess these guys have to cover the upfront costs
4:59
that they're paying for that same hardware.
5:02
And they've got, you know, geo-replication
5:05
and all that kind of cool stuff as well.
5:07
But like, holy!
5:09
Holy balls!
5:10
Wait, hold on a second.
5:12
Go back to that.
5:16
No, no, no.
5:17
This is before you even actually pull any data out of it.
5:23
Okay.
5:23
You have to pay to pull data out.
5:25
So, let's go the cheapest one.
5:27
Yeah, sure.
5:27
Let's say we want to pull the whole thing.
5:30
Another two grand?
5:33
So, this is great.
5:34
So, let's say your office burns down
5:37
and you've got to pull all your data out of the cloud.
5:39
You get a nice handy dandy bill for $2,000 to add in something.
5:42
Any consul to injury?
5:44
That's the slowest.
5:45
That's the slowest speed.
5:46
So, let's say we want to-
5:47
Let's say you need all of it right away.
5:49
Real fast.
5:50
Oh, gigabytes.
5:51
Oh.
5:52
Whoa!
5:54
12 grand.
5:55
No big deal.
5:56
Whatever.
5:57
It's fine.
5:58
Don't worry about it.
5:58
Casual, right?
5:59
Casual.
6:00
And the crazy part of this is that other enterprise solutions
6:02
from the likes of Google, Microsoft, and Backblaze
6:06
aren't priced that differently.
6:10
So, what are we gonna do?
6:13
I'm not paying three to five grand a month
6:16
to store data on tape.
6:18
Wait, does Google still have unlimited storage?
6:22
Well, yeah, but there's no way that's actually unlimited.
6:25
Well, I mean, it's gotta be,
6:26
that would be false advertising, wouldn't it be?
6:28
Unlimited cloud storage.
6:29
With the $10 per user tier, unlimited cloud storage,
6:33
or one terabyte per user, if fewer than five users, so.
6:37
We just get five users.
6:39
That's like $50 a month, that's nothing.
6:40
It can't be unlimited.
6:42
I guess we gotta try.
6:44
To find out if this loophole's actually gonna work,
6:47
we fired up a Windows 10 VM on our Utility Unraid server
6:50
and installed a tool called Rclone.
6:53
So Jake's got the unlimited Google Drive account
6:57
set up and working, so it should just be a matter
7:00
of running the command and seeing what happens.
7:04
So maybe start with Channel Super Fun?
7:08
Channel Super Fun.
7:08
How big's that?
7:10
Oh, let's see.
7:12
That might.
7:13
That might take a while.
7:14
But remember, guys, this is over 100 drives
7:19
spread across two completely separate boxes
7:21
that are networked to each other.
7:25
Wow, this is taking forever.
7:26
I think we had, I indexed the entire Petabyte project,
7:29
and I think we had almost a million files on it.
7:32
It took like literally, I think, a couple hours to do,
7:37
but I did index the whole thing.
7:38
WinDirStat doesn't work, it dies.
7:40
It like, just can't handle that much.
7:43
Okay.
7:45
Should we see how it's going here?
7:48
Oh, it's not going at all.
7:49
What the heck?
7:52
What?
7:53
Oh.
7:54
There's a limit.
7:57
So you were right.
7:58
Yeah.
7:58
It is limited.
8:00
It's limited.
8:00
But it's limited in a way that the advertising's not false.
8:04
So it's unlimited data.
8:06
Yeah.
8:07
As long as it's 750 gigs per day.
8:10
Uploaded, yeah.
8:11
As it turns out, Google saw Bastards Like Us
8:14
coming a mile away, and it looks like, if you're going to do it,
8:14
you're going to have to do it.
8:15
You're going to have to do it.
8:15
You're going to have to do it.
8:16
You're going to have to do it.
8:16
You're going to have to do it.
8:17
You're going to have to do it.
8:17
Like, if you don't want to have your upload cut off,
8:21
you're only able to do 750 gigs a day.
8:24
Otherwise, you have to wait an additional 24 hours
8:28
for an upload ban to pass.
8:31
So what we would have to do then is slow our uploads to just
8:36
around 8 megabytes a second, which
8:39
would mean that our current 370 terabytes would, uh,
8:43
you got that calculator handy?
8:45
370 terabytes.
8:47
Oh.
8:48
Would take about two years.
8:51
So while Google hasn't limited our uploads,
8:55
they have effectively made it completely impractical for us
8:59
to abuse their system this way.
9:01
Guess that makes sense.
9:04
Oh, wait.
9:04
Nope.
9:05
We're not defeated yet.
9:07
New plan.
9:08
So you're actually required to pay
9:11
for a minimum of five G Suite accounts
9:14
in order to unlock that unlimited storage tier.
9:18
But.
9:19
That got us thinking.
9:21
And actually I'm gonna give Jake full credit for this idea,
9:26
what if we used that limitation to our advantage?
9:31
Check this out.
9:32
So it turns out that the 750 Gig per day upload cap
9:36
is per each one of our users here.
9:41
So by just adding a couple more accounts,
9:44
so user 6 here and uh, Mr. Jake Tyvie,
9:47
we can actually end up even if we lose, uh, our users, folks,
9:47
we can actually end up even if we lose, uh, a few more accounts,
9:48
we can actually end up even if we lose, uh, a few more accounts,
9:48
we can actually end up even if we lose, uh, a few more accounts,
9:50
if we limit each account to not go over 750 gigs a day, we can still end up maxing out our upload
9:57
speed to drive at about 40 to 45 megabytes a second. Now, it's not the prettiest solution,
10:04
but this has been going for a week and each one of our instances here has now done about four
10:10
terabytes of data. Yeah. It's a lot of data. It's a lot of data. So conclusion time then,
10:20
I guess. I mean, for now, there's no guarantee whatsoever that Google
10:25
isn't going to shut down this workaround. And there's a fair number of Googlers out there,
10:30
I think, that watch our videos. So us making this video sort of increases our chances. But
10:35
for now, at zero upfront cost and $70 a month for our seven G Suite accounts,
10:43
that's less than $1,000 a year, we have got what is quite literally the least expensive
10:49
multi-tasker, multi-tasker, multi-tasker, multi-tasker, multi-tasker, multi-tasker,
10:50
multi-tasker, multi-tasker, multi-tasker, multi-tasker, multi-tasker, multi-tasker.
10:50
terabyte backup solution that you could have. We're basically terrible people. I'm okay with
10:57
it though. I can live with myself. On the plus side though, this supports encryption on upload
11:02
and download. It does. And there are no fees for pulling the data down in the event of some kind
11:06
of a disaster. I mean, who knows how fast it's going to be. That helps me sleep better at night.
11:11
So there you go. We're pretty happy, even if anyone watching at Google isn't. Hi,
11:17
our YouTube rep. I don't know if I'm supposed to tell people your name. Oh, yeah. Probably
11:22
don't. Ting is the mobile carrier that's focused on customer service and customer satisfaction.
11:29
When you call Ting, you don't speak to a robot. You get put through directly to a person who can
11:35
help you. And you don't pay extra for the privilege. The average Ting bill is just 23 bucks a month per
11:41
device. And they've got cheaper data rates than ever. It's now just $10 a gig beyond the first
11:47
gig. So go try out their savings calculator over at linus2018.ting.com. If it turns out you'll save
11:53
on Ting, they'll actually cover 25% of your cancellation fee for your existing contract
11:58
up to $75. And using our link, that's linus2018.ting.com, you can get 25 bucks in service
12:07
credit or towards a new device. Go check it out at the link in the video description.
12:11
So thanks for watching, guys. If you work for Google, well, you know what to do. But if the
12:15
video was awesome, get subscribed, hit that like button, or check out the link to where to buy the
12:20
stuff we featured in the video description. Also linked in the description, oh, I guess if you're
12:23
a Google shopper, you can check out the link in the description. And if you're a Google shopper,
12:23
shareholder, you're probably not stoked on this either. Also linked in the description is our
12:27
merch store, which has cool shirts like neither of the ones I'm wearing. Do you have an LTT shirt?
12:32
Like this one, cool shirts like this one, and our community forum, which you should totally join.