Contents |
Speed Test Tools
There are a number of tools for testing network speeds. The simplest is to just grab a file using wget, the output of which will display the transfer speed. You can just put a 5 or 10MB file on one of your servers in a web directory and then wget that file from another server.
To make an arbitrary size file you can use:
#dd if=/dev/zero of="10meg.test" bs=1024k count=10
Will create a 10MB file. Change count= to any size you like.
Now, from another server wget that file you just made:
archimedes@Anduril:~$ wget 66.118.10.130/10meg.test
--07:40:02-- http://66.118.10.130/10meg.test
=> `10meg.test'
Connecting to 66.118.10.130:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 10,485,760 (10M) [text/plain]
100%[====================================================>] 10,485,760 8.86M/s
07:40:03 (8.86 MB/s) - `10meg.test' saved [10485760/10485760]
This shows the file was downloaded at 8.8MB/sec
Try a test download from our network:
5MB test download file - wget http://66.111.32.81/5meg.file.txt 10MB test download file - wget http://66.111.32.81/10meg.file.txt 50MB test download file - wget http://66.111.32.81/50meg.file.txt
Extended Speed Testing
wget is certainly fine in many cases, however another tool to use is nttcp - a network speed diagnostic program. This is easy to obtain, you can apt-get install it for Debian and for CentOS / Fedora / Suse you an easily find an RPM for it at sites like http://rpm.pbone.net/
Download it once on to a server and place it where you can just wget it as needed onto other servers you want to test. It must be present on both sending and receiving ends.
Note: nttcp sends and receives udp on port 5037 by default. Ensure your firewall is configured correctly, or disable it, or specify open ports using the -p option
Using it is very simple. On one end simply start it with the -i switch.
#nttcp -i
On the other end you can run tests like this:
#nttcp -t -T destination-ip
(-t for transmit from local server)
#nttcp -r -T destination-ip
(-r for receive to local server)
It will output results like this:
[root@66-118-10-0 ~]# nttcp -t -T 66.118.10.5
Bytes Real s CPU s Real-MBit/s CPU-MBit/s Calls Real-C/s CPU-C/s
l 8388608 0.74 0.01 90.1597 13421.7728 2048 2751.46 409600.0
1 8388608 0.75 0.06 89.4336 1198.3084 5365 7149.75 95798.4
You can see this shows 90Mbit (or 9MB +/-) transfer speeds on this server.
The manual page covers other options - you can designate the port to use and whether to send TCP or UDP packets as well as the amount of data to send. Default is 8 MB.
Use the -n switch to specify the number of buffers (each of which has a default size of 4096 bytes) to send. If you send 2048 buffers (each 4096 bytes) you get 8MB. So for 16MB just send 4096 buffers, etc.
#nttcp -t -T -p 5037 -n 4096 66.118.149.130
This would -t transmit data to -p port 5037 (default) sending -n 4096 buffers (16MB) to destination 66.118.149.130. To reverse the direction of transfer (make it go from remote host to local host) simple change the -t to a -r for receive.
iperf tool
iperf can be downloaded as an RPM from http://rpm.pbone.net/
Sample of this test looks like so:
-c specifies to run the client side of the program and the server in this case is 66.118.10.20 -i = interval reporting 1 second in this case -t = 5 second test -r = do a bidirectional test one way at a time -w = specifying window size ( 83.3k is approx 80 Mbps ) Higher is more for example 100k would max out the NIC performance to 12.5 Mbps if the system can handle it and generate the traffic. bash-2.05b# iperf -c 66.118.10.20 -w 83.3k -i 1 -t 5 -r ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 83.3 KByte ------------------------------------------------------------ ------------------------------------------------------------ Client connecting to 66.118.10.20, TCP port 5001 TCP window size: 83.4 KByte (WARNING: requested 83.3 KByte) ------------------------------------------------------------ [ 5] local 207.150.10.20 port 55239 connected with 66.118.10.20 port 5001 [ 5] 0.0- 1.0 sec 9.98 MBytes 83.8 Mbits/sec [ 5] 1.0- 2.0 sec 10.4 MBytes 87.1 Mbits/sec [ 5] 2.0- 3.0 sec 10.4 MBytes 87.0 Mbits/sec [ 5] 3.0- 4.0 sec 10.3 MBytes 86.6 Mbits/sec [ 5] 4.0- 5.0 sec 10.4 MBytes 87.1 Mbits/sec [ 5] 0.0- 5.0 sec 51.5 MBytes 86.3 Mbits/sec [ 4] local 207.150.10.20 port 5001 connected with 66.118.10.20 port 60513 [ 4] 0.0- 1.0 sec 10.1 MBytes 84.6 Mbits/sec [ 4] 1.0- 2.0 sec 8.35 MBytes 70.1 Mbits/sec [ 4] 2.0- 3.0 sec 7.12 MBytes 59.7 Mbits/sec [ 4] 3.0- 4.0 sec 9.97 MBytes 83.6 Mbits/sec [ 4] 4.0- 5.0 sec 10.5 MBytes 87.8 Mbits/sec [ 4] 0.0- 5.0 sec 46.0 MBytes 77.2 Mbits/sec bash-2.05b#
Analysing Results
If you see speeds that are lower than you expect, here are some things to consider:
A 100Mbit link will show an effective transfer of approximately 80MB = ((100Mbit / 8 bits per byte) - TCP/UDP/Ethernet overhead)
If you are testing this between servers which are both on our network you should see excellent speeds approaching link capacity.
If you test from a remote server (or home / office) to your server in the DC your speeds will likely be significantly lower as you are going through many routers and networks to get to use. If your internal (server to server) results are great and your external (outside to sever) results are poor, it is almost certain there is an issue between you and us (en route). This can be with a provider having a saturated link, with a temporary outage or planned service, or with a number of other factors which are outside the bounds of our network.
So if that is the case how can you tell where the problem really is?
Trouble Shooting Slow Links
If you are having slow transfer speeds we will very likely ask you for trace route or mtr results.
The standard trace route is great for showing which step of the route from you to your server is the bottle neck. Simply use:
Callandor:~ # traceroute 66.118.10.5 traceroute to 66.118.149.130 (66.118.10.5), 30 hops max, 40 byte packets 1 192.168.2.1 (192.168.2.1) 2.250 ms 2.641 ms 1.633 ms 2 10.97.99.99 (10.97.99.99) 8.794 ms 14.778 ms 16.925 ms 3 gig10-0-0-2101.tampfledc-rtr2.tampflrdc.rr.com (65.32.29.114) 19.692 ms 11.051 ms 11.321 ms 4 ge1-2-0.tampfledc-rtr4.tampflrdc.rr.com (65.32.13.33) 15.365 ms 15.030 ms 15.316 ms 5 te-3-1.car2.Tampa1.Level3.net (4.79.146.1) 16.050 ms 19.491 ms 12.478 ms 6 ae-7-7.ebr1.Dallas1.Level3.net (4.69.133.42) 23.627 ms 24.856 ms 19.892 ms 7 ae-0-11.bar1.Tampa1.Level3.net (4.69.137.109) 16.285 ms 16.814 ms 12.529 ms 8 ae-6-6.ebr2.Atlanta2.Level3.net (4.69.137.114) 37.614 ms 30.896 ms 29.215 ms 9 ae-22-52.car2.Atlanta1.Level3.net (4.68.103.35) 29.659 ms ae-12-51.car2.Atlanta1.Level3.net (4.68.103.3) 29.983 ms ae-22-52.car2.Atlanta1.Level3.net (4.68.103.35) 36.124 ms 10 WBS-CONNECT.car2.Atlanta1.Level3.net (4.78.211.54) 37.043 ms 36.982 ms 36.270 ms 11 ve40.core01a.tpa.sagonet.net (63.246.159.41) 49.014 ms 48.494 ms 41.420 ms 12 GigE4.ds-b.03.tpa.sagonet.net (66.118.133.214) 42.089 ms GigE1.ds-b.03.tpa.sagonet.net (65.110.32.48) 46.069 ms GigE4.ds-b.03.tpa.sagonet.net (66.118.133.214) 46.713 ms 13 moria (66.118.10.5) 49.687 ms 46.721 ms 46.868 ms
Here you can see the entire route and the length of time from each hop (router) to the next. You can also see the provider's name (Roadrunner, Level3 and then Sago). All the times are from 20 - 50ms If you see any in the hundreds of higher, that is bad.
You can also use a great tool called mtr - this is easily installed for all distros from their repos or for Windows from http://winmtr.sourceforge.net/
This is like running a trace rotue over and over and accumulating statistics.
You can pause it with "p" or run it for a certain number of -c counts and -r report the results:
Callandor:~ # mtr -c 10 -r 66.118.10.20 HOST: Callandor Loss% Snt Last Avg Best Wrst StDev 1. 192.168.2.1 0.0% 10 1.5 1.5 1.5 1.7 0.1 2. 10.97.96.99 0.0% 10 8.7 8.6 7.8 11.5 1.1 3. gig10-0-0-2101.tampfledc-rtr 0.0% 10 11.8 10.7 8.9 12.0 1.0 4. ge1-2-0.tampfledc-rtr4.tampf 0.0% 10 11.7 11.3 9.2 14.4 1.6 5. te-3-1.car2.Tampa1.Level3.ne 0.0% 10 123.5 22.1 9.0 123.5 35.6 6. ae-7-7.ebr1.Dallas1.Level3.n 0.0% 10 10.7 13.1 10.3 32.6 6.9 7. ae-0-11.bar1.Tampa1.Level3.n 0.0% 10 13.2 11.3 10.1 13.2 0.8 8. ae-6-6.ebr2.Atlanta2.Level3. 0.0% 10 40.9 34.7 28.7 40.9 4.5 9. ae-22-52.car2.Atlanta1.Level 0.0% 10 29.0 79.6 28.2 199.4 64.2 10. WBS-CONNECT.car2.Atlanta1.Le 0.0% 10 33.6 30.3 28.5 33.6 1.4 11. ve40.core01a.tpa.sagonet.net 0.0% 10 41.2 41.9 39.8 44.7 1.3 12. GigE1.ds-b.03.tpa.sagonet.ne 0.0% 10 40.4 41.9 40.4 44.5 1.1 13. moria 0.0% 10 42.0 42.6 41.7 43.7 0.7
Speed Duplex Issues
Allright, you have run the above tests using wget and nttcp and are seeing that server to server transfer speeds are extremely slow, like 1 MB/sec or something really silly.
There is a 99% chance the issue is due to a speed / duplex setting mismatch between your server and the switch.
To determine what your card and link is currently running at use:
[root@tardis web]# mii-tool eth0: negotiated 100baseTx-FD, link ok eth1: no link
Some distros do not have that installed by default, but do have ethtool:
[root@tardis web]# ethtool eth0
Settings for eth0:
Supported ports: [ TP MII FIBRE ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Advertised auto-negotiation: Yes
Speed: 100Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbag
Wake-on: d
Current message level: 0x000040c5 (16581)
Link detected: yes
You can reconfigure the link with:
[root@tardis web]# mii-tool -F 100baseTx-FD eth0 [root@tardis web]# mii-tool -rR resetting the transceiver... restarting autonegotiation... resetting the transceiver... restarting autonegotiation... [root@tardis web]# [root@tardis web]# mii-tool eth0: negotiated 100baseTx-FD, link ok eth1: no link
NOTE: mii-tool -? and man mii-tool (or man ethtool) will give all the options. If you set this incorrectly you can lock yourself out.
If that happens you an attempt a remote reboot from the customer portal and try other settings.
If you still can't resolve it, open a ticket and please provide your root log in credentials so we can trouble shoot the link settings.
