Iperf & Seedbox speed testing

Many folks are unaware of the dramatic effect disk speeds have on there network speeds, since the bulk of your data comes from or goes to disk, and mass storage is the slowest link in the chain, it is the single most important factor in how fast you can download media home, download torrent payloads, or even upload to others.

The chain is:

When tuning a system these are the three components that garner the most attention: how can I get things on to and off of my disk quickly; how best to optimize my cache; and then finally how can I insure my network is running at its peak.

In a previous posting we looked at how best diagnose route and peering issues using the linux command mtr. But mtr looks at road conditions, what is likely slowing you down. It doesn't address how fast you can actually go. We can do that with Iperf, and significantly iperf removes your disk from the mix. MTR lays out the map, the roads you are taking, any potholes, traffic jams, bad interchanges, each turn and curve along the way. Iperf looks at how fast your car can go down that route, pure speed, in real numbers. MTR lays out the track, iperf runs the race.

This is a tutorial, it is recommended you follow along using your server and your home machine to run the commands in parallel.

Iperf, or its latest incarnation, iperf3, is a client/server tool. You run iperf on your server, then from a client connect and see how fast your server either receives or can send you packets. It is really quite simple.

First we need to install iperf3 on your server, as always if you have superuser this is simple, if not a bit harder, with sudo (everything here is debian or ubuntu):

If you don't have root you'll need to manually install it, here is what you need to do:

Running things on the client is now easier than tying your shoes:

10666 is the port that we will be using, the server will be listen for connections on. It is optional, but since the default port might be taken (5201), I've included how to set your own.

Now you need to install the client, a client for virtually any platform can be found over at iperf.fr, in particular https://iperf.fr/iperf-download.php

I'm going to assume you have windows, but it is largely the same for any non-Unix client, you download the zip file, unzip it into its own directory, then open a DOS or command window in that directory. With Windows, iperf3 uses the cygwin runtime in the form of DLL(s).

Ok, with our server set up, and the client installed, a quick aside for those unanointed in parlance of networking. The two most common internet protocols are TCP (Transmission Control Protocol) and UDP (User Datagram Protocol). TCP is like a phone call, a circuit is created between the caller and the person being called, they say something you'll hear it unless the whole line drops, it is reliable. UDP is more like sending a letter, you drop a packet on the wire with an address and best effort is made to get it delivered, it is inherently unreliable. We'll be using both in our tests. Thing to understand is UDP it is cheaper, and its unreliability can be useful.

Ok, lets start with the simplest case:

This will SEND packets to the server for 10 seconds, and tell you how fast the packets went, the output will look like:

The last set of numbers (sender / receiver) is our summary line, average speed. This is in megabits per second, so my upload speed is 2.23MB/s.

This though is just a test of how fast I can talk to my server, which isn't nearly as interesting as how fast my server can talk to me, to do that we need to tell iperf to receive. We add the -R parameter

Now this is interesting, my speed down is hitting Sunday night traffic, and is significantly slower. See the number of retries (Retr)? There is some problem on the down route, you can also see the jitter (variations in latency or speed).

Now you should compare this to the speed your are getting on a single FTP connection using like Filezilla. Checking my connection, I'm getting .5MB/s that means the disk is costing me 25% of my download speed, not unusual. If the disk is re

To see how fast I can get things to, like segmented FTP, I'm going to need to use multiple simultaneous connections, lucky iperf3  can do that too. Lets do 10 segments, with the -P 10 parameter.

Just the summary lines, but you can see much faster. Lets try 30.

That scales rather nicely, I am receiving 6.75MB/s, but look at my retransmits, that has scaled too. Server is sending at over 8MB/s.

At this point further escalation of connections, doesn't scale nearly as well.

OK, lets see what might be going on, we'll use UDP now (by adding the -u parameter), with 10 connections, from me to my server, :

No packet loss, of the 1998 packets sent 0% were lost, at a speed of 13.1Mbits/s - pretty good.

Now going the other way (-R):

Fewer packets sent and an overall loss of .56% of the packets wandered off. I'm willing to bet if I did an mtr, I'd see some bad jitter, and probably some packet loss.

Yep, checking mtr, we see where the problem is:

1.9% packet loss in Chicago, those ruffians.

So you can see, iperf can be an effective tool, primarily for determining the hit you are taking because of your hard disk, but also in analyzing route dynamics, what is costing you against your ideal. I've covered most of the interesting parameters, the only other flag I use is -t as in -t 30, so that iperf will run for longer (30 secs vs 10sec default), which can give you a better picture. But with all linux commands, you should take a look at the manpage yourself: https://www.mankier.com/1/iperf3

Disclaimer: I fiddled with the numbers to make this more interesting, I also ran a laptop against a Chmuranet 10G server, a DC based CPU-laden 10G server can send packets much faster then my poor laptop can handle, this is part of the packet loss you saw in my examples. Consider this when looking at your own server.