Skip to content

Instantly share code, notes, and snippets.

@jdickey
Last active June 23, 2017 15:35
Show Gist options
  • Save jdickey/da8e093578918da71c59362e15236e1a to your computer and use it in GitHub Desktop.
Save jdickey/da8e093578918da71c59362e15236e1a to your computer and use it in GitHub Desktop.
Benchmarking Access to a Static Site While Using a VPN: It's Not That Straightforward

Contents

Introduction

The site is a demo/tutorial static site, hosted on a DigitalOcean droplet physically located in Singapore. The benchmarks were run from a Mid-2011 iMac running OSX 10.11.6 using Apache Benchmark 2.3. The Mac was connected to the Internet via an 802.11ac wireless network to a MyRepublic 1Gbps fibre connection. Speedtest.net reported download speeds varying between 112.43 Mbps and 222.58 Mbps download while not connected to the VPN. Even at the lowest reported speed, the benchmarking did not saturate the connection, with transfer speeds in the mere kilobytes per second.

Apache Benchmark was run against various nodes supported by NordVPN, which has thousands of nodes in dozens of countries all over the world, including 11 active nodes in Singapore as this is being written. Nodes outside Singapore were chosen by the NordVPN app for Mac OS X when directed to connect to a node of its choice in a specific country.

Command line:

$ ab -t 60 -c 200 http://seven-sigma.site/
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
... results ...
$

Summary of Transfer Results

In the following charts, the VPN Node value is the identifier of a particular VPN server in the NordVPN network; the Distance (km) is as reported by the NordVPN control app for Mac OS X.

Sorted by distance to VPN endpoint

VPN Node Distance (km) Requests completed Requests/second 95% service time (ms) 99% service time (ms)
NONE 0 2521 34.63 1359 2487
SG#2 1 920 15.33 19564 35987
SG#9 1 915 15.14 19725 35839
SG#7 1 908 15.08 19494 35844
MY#1 317 900 15.0 6097 20032
VN#2 1100 815 13.58 9652 20420
TH#1 1434 916 15.26 5345 11484
HK#7 2587 916 15.26 5383 11632
TW#5 3121 903 15.05 7653 11808
AU#35 3908 907 15.11 4810 12015
KR#5 4679 909 15.14 5598 20067
NL#22 10500 872 14.51 8003 20456
UK#72 10856 902 15.02 6039 20583
US#364 12991 905 15.08 4871 20333

Sorted by requests completed (descending)

Where multiple nodes have the same completed-request count (e.g., HK#7 and TH#1), they are presented with the farthest node first. Singapore #7 and Singapore #2 are reported as being the same distance away; they are presented in order of average requests per second.

VPN Node Distance (km) Requests completed Requests/second 95% service time (ms) 99% service time (ms)
NONE 0 2521 34.63 1359 2487
SG#2 1 920 15.33 19564 35987
SG#7 1 908 15.08 19494 35844
HK#7 2587 916 15.26 5383 11632
TH#1 1434 916 15.26 5345 11484
SG#9 1 915 15.14 19725 35839
KR#5 4679 909 15.14 5598 20067
AU#35 3908 907 15.11 4810 12015
US#364 12991 905 15.08 4871 20333
TW#5 3121 903 15.05 7653 11808
UK#72 10856 902 15.02 6039 20583
MY#1 317 900 15.0 6097 200032
NL#22 10500 872 14.51 8003 20456
VN#2 1100 815 13.58 9652 20420

Sorted by service time for 99th percentile of requests

VPN Node Distance (km) Requests completed Requests/second 95% service time (ms) 99% service time (ms)
NONE 0 2521 34.63 1359 2487
TH#1 1434 916 15.26 5345 11484
HK#7 2587 916 15.26 5383 11632
TW#5 3121 903 15.05 7653 11808
AU#35 3908 907 15.11 4810 12015
KR#5 4679 909 15.14 5598 20067
MY#1 317 900 15.0 6097 20032
US#364 12991 905 15.08 4871 20333
VN#2 1100 815 13.58 9652 20420
NL#22 10500 872 14.51 8003 20456
UK#72 10856 902 15.02 6039 20583
SG#9 1 915 15.14 19725 35839
SG#7 1 908 15.08 19494 35844
SG#2 1 920 15.33 19564 35987

Results

The Singapore nodes are surprisingly slow; while more requests passed through one of the three Singapore nodes were completed in 60 seconds than any other node tested, that was only 36.5% as many requests as were completed without any VPN connection at all. A VPN should provide anonymity and other security-related benefits; one might easily argue that slowing the transfer rate by some 2/3 is a hefty price to pay for that.

Number of requests completed were highly uniform regardless of distance to VPN endpoint; there was no visible correlation between the two.

Distance did not strictly correlate to service time, either; rather, the countries reputed to have better Internet access (Australia, Hong Kong, and the US as a special case) had service times significantly better than other, geographically closer countries did. It is again interesting that VPN endpoints in Singapore, where the tests were run from and where the site being accessed is hosted, posted the worst 95% and 99% service times by considerable margins. The consistency of this suggests that the problem may not be solely technical, but political as well. We note that NordVPN offer no endpoints in the People's Republic of China; we also note that even Vietnam's and Thailand's endpoints, where the governments also exercise enhanced control of residents' access to and behaviour on the Internet, put less of a drag on access times. For a country that brands itself as a Smart Nation leveraging the Internet, policy speaks louder than propaganda.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment