10 Gigabit Networking - I wanna go fast. Really, really fast.
Linus Tech Tips
·Linus Tech Tips
·2016-05-06
·
1,843 words · ~9 min read
0:00
10 gigabit networking i mean think about
0:04
it transferring files around to remote pcs at speeds upwards of one Gigabyte
0:09
per second that's a thousand megabytes per second well today we're not only
0:14
going to do it we're going to give you some tips and tricks that will hopefully
0:18
help you out when the time comes to move to 10 gigabit yourself whether it's at
0:22
work or at home and on the subject of work i thought to myself hey how about
0:27
clearing some space on my desk at work by building a pc
0:31
under the desk drop a like on the video if you're amped
0:35
to see my upcoming desk pc build blog
0:46
the master case 5 by cooler master gives you the freedom to truly make your
0:50
mid-tower pc case your own with a variety of modular parts and accessories
0:54
click on the link in the video description to learn more
0:58
you know i still remember how mind-blowing it was like the first time
1:02
that i saw something transfer over my network at 60 to 70 megabytes per second
1:08
with the onboard gigabit lan port on some enforce board that i had then
1:12
shortly afterward my mind was blown again when the pci bus bottleneck was
1:17
removed and thanks to onboard pci express gigabit networking i could
1:22
transfer files thanks to my trusty diamond max 9 120 gig raid 0 boot drive
1:28
at over a hundred megabytes per second
1:32
wow then for the last 10 years well it's
1:36
as though time has pretty much been standing still i mean sure the enterprise space is getting 40 gigabit
1:41
and 100 gigabit links but it really feels like about a decade ago the entire
1:46
industry had a meeting and decided that yeah gigabit is good enough for
1:50
consumers and small businesses and let's just leave it at that well i say no more
1:54
we are using and i'm using this term
1:57
fairly loosely here affordable hardware to go 10 gig at the office today
2:03
it begins with the nix that is to say network interface cards
2:08
we'll be using Intel x540 t2s but the
2:12
only difference between it and the t1 being that it has two ports instead of
2:15
well one and while any 10 gigabit card will knock the socks off of a gigabit
2:20
one we chose these for two reasons one their Intel which is basically the
2:25
industry standard for prosumer network cards not to mention our testing with
2:29
them has gone very well and two because they're both somewhat affordable at 300
2:34
bucks a pop on ebay although i got some of mine for cheaper and available in an
2:38
rj45 equipped variant this is important
2:42
because we wanted to use Ethernet cables
2:45
rather than the expensive sfp plus connectors and direct attach copper
2:50
cables that we were using in our previous 10 gigabit setup the reason for
2:55
that is that while that configuration worked well for us before starting out
2:59
with pcs attached daisy chaining to each other actually then moving up to using
3:05
the backbone links on a 48 port netgear gs752 txs switch it doesn't play nicely
3:11
with running cabling through the walls uh definitely desirable for an office
3:15
space and it's not scalable and if we want to run more pcs on 10 gig at a time
3:21
we need a 10 gig switch with more ports
3:24
and pretty much the switch right now if you don't want to spend an absolute
3:28
fortune and you know then you might as well get sfp plus gear is the xs 712 t12
3:33
port or its eight port little brother from netgear and those use rj45
3:38
connectors all right so it's simple right plug in some Ethernet cables and
3:42
bippidybop boom you're transferring files at a thousand megabytes per second
3:46
right actually not quite first you'll need to make sure that
3:49
you're using cat6a cables if you want
3:52
reliable data transfer over a reasonable
3:55
distance and second and you may not have considered this but 10 gigabit is so
4:01
fast that it exceeds the six gigabit per
4:05
second limit of third generation SATA
4:08
so even if you have a wicked fast SSD drive you'll be limited to speeds in the
4:14
neighborhood of 500 to 550 megabytes per
4:17
second and while pci express ssds that
4:20
overcome this bottleneck do exist
4:23
i stole our Intel 750 series and put it in my personal rig so i ended up using
4:28
another solution i've got our 24 SSD storage server on one end the one that
4:33
we built in this video here and i've got a test bench with that 128
4:38
Gigabyte kit of dominator platinum ddr4 from Corsair on the other end with a
4:43
free and easy to use utility called soft perfect RAM disc used to treat that RAM
4:48
like a hard drive so with such fast storage on either end it's much easier
4:53
to evaluate the performance of the actual network link which frankly didn't
4:58
go so hard on the first kick at the cam i mean i don't know
5:01
who these people getting perfect 10 gig performance out of the box
5:06
are or how many horseshoes they had to put up their butts to make them so damn
5:10
lucky but i was seeing this weird cap at
5:13
around 300 to 350 megabytes per second
5:17
now to be clear that is still a significant improvement and already well
5:22
worth the upgrade for our purposes but i wanted more damn it i paid for more
5:27
and thanks to an amazing series of posts on the synovate blog which you should
5:32
definitely check out we can link them below the video i was able to do much
5:36
better than that mostly so the first tip from them was this
5:40
great tool called nttcp that has a command line interface so not the
5:44
friendliest thing but that they simplify by basically showing you where to plug
5:48
in the program install path and machine ip addresses and boom it's off to the
5:52
races so this tool revealed that my two machines were capable of more than one
5:58
Gigabyte per second performance between them so what gives
6:02
well time to dig into the advanced settings to see if there's anything that
6:05
we can tweak i started by enabling jumbo packets on all clients involved in the
6:09
transfer and on my network switch side
6:13
note here it doesn't seem to matter if the values don't match up exactly as
6:16
long as your switch is set to something higher than your nics so this means that
6:21
now effectively the data packets sent back and forth are bigger which is
6:25
better for a high speed transfer of large files next i tweaked receive side
6:29
scaling a feature that leverages more CPU cores for network transfers and i
6:34
set it to match the settings with my number of logical processors on my pc so
6:39
16 on my 5960x test bench uh 8 on my
6:43
6700k test bench and 16 on my xeon server and finally i increased the size
6:48
of both the send and receive buffers to
6:51
their maximum this increases memory usage but can yield extra performance so
6:56
then what did i get well performance wise over the original numbers in ntt
7:01
tcp not much but my real world transfers
7:05
were much higher in fact more higher than they should have been but i'm not
7:10
going to complain and i was seeing sustained speeds of
7:14
over double my original transfer rates
7:17
with that said not all is necessarily rosy and for whatever reason maybe some
7:22
of you sharp people out there can go ahead and contribute your comments below
7:26
some transfers are faster or slower than
7:30
others so i ended up actually putting together two test benches with brand spanking new
7:35
installations of Windows 10 and Windows server 2012 r2 just for the sake of
7:39
trying to eliminate as many variables as possible and i was still running into
7:43
weirdness where one machine with a particular os would saturate the
7:48
connection writing to the wanik SSD server but then be capable of
7:53
half or 60 percent of that speed when reading from it and then another would
7:57
saturate on reeds and then limp along on rights i mean
8:01
that's the one you're looking at here by the way but as snake bitten as i seem to
8:05
be when it comes to networking stuff limping along in this case is still 500
8:09
plus megabytes per second so i guess i'll just have to deal with my 34
8:14
Gigabyte file transferring a little slower than it otherwise would overall
8:20
very happy with the results and speaking of being happy with the
8:24
results squarespace if you were thinking to yourself gee i'd like to build myself
8:29
a website but i sure don't want to be unhappy with the
8:32
results bam pick out a template on squarespace customize that template to
8:37
build yourself whatever it is you're trying to build yourself you want to build a store a company website a you
8:42
know menu a blog a portfolio use their
8:46
simple templates and their powerful creation tools and their 24 7 tech
8:50
support and it starts at only eight dollars a month so it's affordable you
8:53
won't break the bank and i'm sure i have more talking points but i think you guys
8:57
pretty much get the point you want to build an easy to use reliable website
9:02
squarespace.com they've got a free trial which you can sign up for today and then
9:06
if you end up liking the site that you create you can look into their affordable plans pretty freaking awesome
9:13
there's no credit card required by the way to try out the trial so pretty cool
9:17
when and if you decide to sign up for squarespace make sure though to use offer code Linus and get 10 off your
9:22
first purchase
9:26
so that's it guys thanks for watching the video if you disliked it i think you know where that button is if you liked
9:30
it though hit that like button hit that subscribe button unless you're already
9:34
subscribed in which case please don't hit the subscribe button and maybe also
9:38
consider supporting us by shopping at amazon with our affiliate link instructions up there or by what are the
9:43
other things i'm sure there's other things right t-shirts and oh right monthly contributions to the forum you
9:47
get a cool little contributor badge good stuff like that now that you're done
9:51
doing all that stuff you're probably wondering what to watch next so go check out that little button in the top right
9:55
corner for video x where i something i don't know we'll put something in there
9:59
i forgot to fill it in on the script
10:04
you