Created
May 11, 2012 03:53
-
-
Save courtneycouch/2657432 to your computer and use it in GitHub Desktop.
AB test results - nodejs vs vert.x
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Single core - ab test single concurrency 100k requests | |
| Server Software: | |
| Server Hostname: 127.0.0.1 | |
| Server Port: 8080 | |
| Document Path: / | |
| Document Length: 89 bytes | |
| Concurrency Level: 1 | |
| Time taken for tests: 26.165080 seconds | |
| Complete requests: 100000 | |
| Failed requests: 0 | |
| Write errors: 0 | |
| Total transferred: 17200000 bytes | |
| HTML transferred: 8900000 bytes | |
| Requests per second: 3821.89 [#/sec] (mean) | |
| Time per request: 0.262 [ms] (mean) | |
| Time per request: 0.262 [ms] (mean, across all concurrent requests) | |
| Transfer rate: 641.92 [Kbytes/sec] received | |
| Connection Times (ms) | |
| min mean[+/-sd] median max | |
| Connect: 0 0 0.0 0 0 | |
| Processing: 0 0 0.2 0 9 | |
| Waiting: 0 0 0.1 0 9 | |
| Total: 0 0 0.2 0 9 | |
| Percentage of the requests served within a certain time (ms) | |
| 50% 0 | |
| 66% 0 | |
| 75% 0 | |
| 80% 0 | |
| 90% 0 | |
| 95% 0 | |
| 98% 0 | |
| 99% 0 | |
| 100% 9 (longest request) | |
| Single core - ab test 10x concurrency 100k requests | |
| Server Software: | |
| Server Hostname: 127.0.0.1 | |
| Server Port: 8080 | |
| Document Path: / | |
| Document Length: 89 bytes | |
| Concurrency Level: 10 | |
| Time taken for tests: 18.556226 seconds | |
| Complete requests: 100000 | |
| Failed requests: 0 | |
| Write errors: 0 | |
| Total transferred: 17200000 bytes | |
| HTML transferred: 8900000 bytes | |
| Requests per second: 5389.03 [#/sec] (mean) | |
| Time per request: 1.856 [ms] (mean) | |
| Time per request: 0.186 [ms] (mean, across all concurrent requests) | |
| Transfer rate: 905.14 [Kbytes/sec] received | |
| Connection Times (ms) | |
| min mean[+/-sd] median max | |
| Connect: 0 0 0.0 0 0 | |
| Processing: 1 1 0.5 1 11 | |
| Waiting: 0 1 0.5 1 11 | |
| Total: 1 1 0.5 1 11 | |
| Percentage of the requests served within a certain time (ms) | |
| 50% 1 | |
| 66% 1 | |
| 75% 1 | |
| 80% 1 | |
| 90% 1 | |
| 95% 1 | |
| 98% 1 | |
| 99% 2 | |
| 100% 11 (longest request) | |
| Single core - ab test 100x concurrency 100k requests | |
| Server Software: | |
| Server Hostname: 127.0.0.1 | |
| Server Port: 8080 | |
| Document Path: / | |
| Document Length: 89 bytes | |
| Concurrency Level: 100 | |
| Time taken for tests: 18.239243 seconds | |
| Complete requests: 100000 | |
| Failed requests: 0 | |
| Write errors: 0 | |
| Total transferred: 17200000 bytes | |
| HTML transferred: 8900000 bytes | |
| Requests per second: 5482.68 [#/sec] (mean) | |
| Time per request: 18.239 [ms] (mean) | |
| Time per request: 0.182 [ms] (mean, across all concurrent requests) | |
| Transfer rate: 920.87 [Kbytes/sec] received | |
| Connection Times (ms) | |
| min mean[+/-sd] median max | |
| Connect: 0 0 0.1 0 4 | |
| Processing: 0 17 6.3 17 35 | |
| Waiting: 0 17 6.3 17 35 | |
| Total: 2 17 6.3 17 35 | |
| Percentage of the requests served within a certain time (ms) | |
| 50% 17 | |
| 66% 23 | |
| 75% 23 | |
| 80% 23 | |
| 90% 23 | |
| 95% 24 | |
| 98% 29 | |
| 99% 31 | |
| 100% 35 (longest request) | |
| Single core - ab test 1000x concurrency 100k requests | |
| Server Software: | |
| Server Hostname: 127.0.0.1 | |
| Server Port: 8080 | |
| Document Path: / | |
| Document Length: 89 bytes | |
| Concurrency Level: 1000 | |
| Time taken for tests: 17.803068 seconds | |
| Complete requests: 100000 | |
| Failed requests: 0 | |
| Write errors: 0 | |
| Total transferred: 17200000 bytes | |
| HTML transferred: 8900000 bytes | |
| Requests per second: 5617.01 [#/sec] (mean) | |
| Time per request: 178.031 [ms] (mean) | |
| Time per request: 0.178 [ms] (mean, across all concurrent requests) | |
| Transfer rate: 943.43 [Kbytes/sec] received | |
| Connection Times (ms) | |
| min mean[+/-sd] median max | |
| Connect: 0 79 598.3 0 9001 | |
| Processing: 3 43 24.8 44 2074 | |
| Waiting: 3 43 24.7 44 2074 | |
| Total: 3 123 602.4 44 9066 | |
| Percentage of the requests served within a certain time (ms) | |
| 50% 44 | |
| 66% 44 | |
| 75% 45 | |
| 80% 46 | |
| 90% 52 | |
| 95% 55 | |
| 98% 3030 | |
| 99% 3047 | |
| 100% 9066 (longest request) | |
| 8 core - ab test 1x concurrency 100k requests | |
| Server Software: | |
| Server Hostname: 127.0.0.1 | |
| Server Port: 8080 | |
| Document Path: / | |
| Document Length: 89 bytes | |
| Concurrency Level: 1 | |
| Time taken for tests: 27.673151 seconds | |
| Complete requests: 100000 | |
| Failed requests: 0 | |
| Write errors: 0 | |
| Total transferred: 17200000 bytes | |
| HTML transferred: 8900000 bytes | |
| Requests per second: 3613.61 [#/sec] (mean) | |
| Time per request: 0.277 [ms] (mean) | |
| Time per request: 0.277 [ms] (mean, across all concurrent requests) | |
| Transfer rate: 606.94 [Kbytes/sec] received | |
| Connection Times (ms) | |
| min mean[+/-sd] median max | |
| Connect: 0 0 0.0 0 4 | |
| Processing: 0 0 0.1 0 9 | |
| Waiting: 0 0 0.1 0 9 | |
| Total: 0 0 0.1 0 9 | |
| Percentage of the requests served within a certain time (ms) | |
| 50% 0 | |
| 66% 0 | |
| 75% 0 | |
| 80% 0 | |
| 90% 0 | |
| 95% 0 | |
| 98% 0 | |
| 99% 0 | |
| 100% 9 (longest request) | |
| 8 core - ab test 10x concurrency 100k requests | |
| Server Software: | |
| Server Hostname: 127.0.0.1 | |
| Server Port: 8080 | |
| Document Path: / | |
| Document Length: 89 bytes | |
| Concurrency Level: 10 | |
| Time taken for tests: 8.602445 seconds | |
| Complete requests: 100000 | |
| Failed requests: 0 | |
| Write errors: 0 | |
| Total transferred: 17200000 bytes | |
| HTML transferred: 8900000 bytes | |
| Requests per second: 11624.60 [#/sec] (mean) | |
| Time per request: 0.860 [ms] (mean) | |
| Time per request: 0.086 [ms] (mean, across all concurrent requests) | |
| Transfer rate: 1952.47 [Kbytes/sec] received | |
| Connection Times (ms) | |
| min mean[+/-sd] median max | |
| Connect: 0 0 0.0 0 4 | |
| Processing: 0 0 0.5 0 12 | |
| Waiting: 0 0 0.2 0 11 | |
| Total: 0 0 0.5 0 12 | |
| Percentage of the requests served within a certain time (ms) | |
| 50% 0 | |
| 66% 0 | |
| 75% 0 | |
| 80% 0 | |
| 90% 1 | |
| 95% 1 | |
| 98% 1 | |
| 99% 1 | |
| 100% 12 (longest request) | |
| 8 core - ab test 100x concurrency 100k requests | |
| Server Software: | |
| Server Hostname: 127.0.0.1 | |
| Server Port: 8080 | |
| Document Path: / | |
| Document Length: 89 bytes | |
| Concurrency Level: 100 | |
| Time taken for tests: 9.98349 seconds | |
| Complete requests: 100000 | |
| Failed requests: 0 | |
| Write errors: 0 | |
| Total transferred: 17200172 bytes | |
| HTML transferred: 8900089 bytes | |
| Requests per second: 10991.01 [#/sec] (mean) | |
| Time per request: 9.098 [ms] (mean) | |
| Time per request: 0.091 [ms] (mean, across all concurrent requests) | |
| Transfer rate: 1846.16 [Kbytes/sec] received | |
| Connection Times (ms) | |
| min mean[+/-sd] median max | |
| Connect: 0 0 1.3 1 9 | |
| Processing: 1 7 1.5 7 21 | |
| Waiting: 0 2 1.6 3 17 | |
| Total: 1 8 2.0 8 23 | |
| Percentage of the requests served within a certain time (ms) | |
| 50% 8 | |
| 66% 9 | |
| 75% 10 | |
| 80% 10 | |
| 90% 11 | |
| 95% 12 | |
| 98% 13 | |
| 99% 14 | |
| 100% 23 (longest request) | |
| 8 core - ab test 1000x concurrency 100k requests | |
| Server Software: | |
| Server Hostname: 127.0.0.1 | |
| Server Port: 8080 | |
| Document Path: / | |
| Document Length: 89 bytes | |
| Concurrency Level: 1000 | |
| Time taken for tests: 9.831740 seconds | |
| Complete requests: 100000 | |
| Failed requests: 0 | |
| Write errors: 0 | |
| Total transferred: 17201032 bytes | |
| HTML transferred: 8900534 bytes | |
| Requests per second: 10171.14 [#/sec] (mean) | |
| Time per request: 98.317 [ms] (mean) | |
| Time per request: 0.098 [ms] (mean, across all concurrent requests) | |
| Transfer rate: 1708.45 [Kbytes/sec] received | |
| Connection Times (ms) | |
| min mean[+/-sd] median max | |
| Connect: 0 15 9.9 15 83 | |
| Processing: 16 81 11.0 82 169 | |
| Waiting: 0 36 10.3 36 153 | |
| Total: 22 97 17.3 96 251 | |
| Percentage of the requests served within a certain time (ms) | |
| 50% 96 | |
| 66% 103 | |
| 75% 108 | |
| 80% 111 | |
| 90% 118 | |
| 95% 126 | |
| 98% 135 | |
| 99% 141 | |
| 100% 251 (longest request) | |
| ab -c 1 -n 100000 http://127.0.0.1:8080/ | |
| Vert.X | |
| 8 core - ab test 1x concurrency 100k requests | |
| Server Software: | |
| Server Hostname: 127.0.0.1 | |
| Server Port: 8080 | |
| Document Path: / | |
| Document Length: 89 bytes | |
| Concurrency Level: 1 | |
| Time taken for tests: 24.581249 seconds | |
| Complete requests: 100000 | |
| Failed requests: 0 | |
| Write errors: 0 | |
| Total transferred: 15300000 bytes | |
| HTML transferred: 8900000 bytes | |
| Requests per second: 4068.14 [#/sec] (mean) | |
| Time per request: 0.246 [ms] (mean) | |
| Time per request: 0.246 [ms] (mean, across all concurrent requests) | |
| Transfer rate: 607.82 [Kbytes/sec] received | |
| Connection Times (ms) | |
| min mean[+/-sd] median max | |
| Connect: 0 0 0.0 0 1 | |
| Processing: 0 0 0.2 0 58 | |
| Waiting: 0 0 0.2 0 58 | |
| Total: 0 0 0.2 0 58 | |
| Percentage of the requests served within a certain time (ms) | |
| 50% 0 | |
| 66% 0 | |
| 75% 0 | |
| 80% 0 | |
| 90% 0 | |
| 95% 0 | |
| 98% 0 | |
| 99% 0 | |
| 100% 58 (longest request) | |
| 8 core - ab test 10x concurrency 100k requests | |
| Server Software: | |
| Server Hostname: 127.0.0.1 | |
| Server Port: 8080 | |
| Document Path: / | |
| Document Length: 89 bytes | |
| Concurrency Level: 10 | |
| Time taken for tests: 11.864840 seconds | |
| Complete requests: 100000 | |
| Failed requests: 0 | |
| Write errors: 0 | |
| Total transferred: 15300000 bytes | |
| HTML transferred: 8900000 bytes | |
| Requests per second: 8428.26 [#/sec] (mean) | |
| Time per request: 1.186 [ms] (mean) | |
| Time per request: 0.119 [ms] (mean, across all concurrent requests) | |
| Transfer rate: 1259.27 [Kbytes/sec] received | |
| Connection Times (ms) | |
| min mean[+/-sd] median max | |
| Connect: 0 0 0.0 0 7 | |
| Processing: 0 0 0.9 1 8 | |
| Waiting: 0 0 0.1 0 7 | |
| Total: 0 0 0.9 1 8 | |
| WARNING: The median and mean for the processing time are not within a normal deviation | |
| These results are probably not that reliable. | |
| WARNING: The median and mean for the total time are not within a normal deviation | |
| These results are probably not that reliable. | |
| Percentage of the requests served within a certain time (ms) | |
| 50% 1 | |
| 66% 1 | |
| 75% 1 | |
| 80% 1 | |
| 90% 1 | |
| 95% 1 | |
| 98% 1 | |
| 99% 1 | |
| 100% 8 (longest request) | |
| 8 core - ab test 100x concurrency | |
| Server Software: | |
| Server Hostname: 127.0.0.1 | |
| Server Port: 8080 | |
| Document Path: / | |
| Document Length: 89 bytes | |
| Concurrency Level: 100 | |
| Time taken for tests: 11.915829 seconds | |
| Complete requests: 100000 | |
| Failed requests: 0 | |
| Write errors: 0 | |
| Total transferred: 15306273 bytes | |
| HTML transferred: 8903649 bytes | |
| Requests per second: 8392.20 [#/sec] (mean) | |
| Time per request: 11.916 [ms] (mean) | |
| Time per request: 0.119 [ms] (mean, across all concurrent requests) | |
| Transfer rate: 1254.38 [Kbytes/sec] received | |
| Connection Times (ms) | |
| min mean[+/-sd] median max | |
| Connect: 0 4 95.0 1 3014 | |
| Processing: 1 7 3.2 7 226 | |
| Waiting: 0 2 2.4 3 224 | |
| Total: 1 11 95.1 8 3025 | |
| Percentage of the requests served within a certain time (ms) | |
| 50% 8 | |
| 66% 10 | |
| 75% 11 | |
| 80% 12 | |
| 90% 13 | |
| 95% 15 | |
| 98% 16 | |
| 99% 17 | |
| 100% 3025 (longest request) | |
| 8 core - ab test 1000x concurrency | |
| Vert.X crashes. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| NodeJS 1 core 100k requests | |
| Concurrency 1x - 3821.89 [requests/sec] Test Time: 26.165080 seconds | |
| Concurrency 10x - 5389.03 [requests/sec] Test Time: 18.556226 seconds | |
| Concurrency 100x - 5482.68 [requests/sec] Test Time: 18.239243 seconds | |
| Concurrency 1000x- 5617.01 [requests/sec] Test Time: 17.803068 seconds | |
| NodeJS 8 core 100k requests | |
| Concurrency 1x - 3613.61 [requests/sec] Test Time: 27.673151 seconds | |
| Concurrency 10x - 11624.60 [requests/sec] Test Time: 8.602445 seconds | |
| Concurrency 100x - 10991.01 [requests/sec] Test Time: 9.98349 seconds | |
| Concurrency 1000x- 10171.14 [requests/sec] Test Time: 9.831740 seconds | |
| Vert.x 8 core 100k requests | |
| Concurrency 1x - 4068.14 [requests/sec] Test Time: 24.581249 seconds | |
| Concurrency 10x - 8428.26 [requests/sec] Test Time: 11.864840 seconds | |
| Concurrency 100x - 8392.20 [requests/sec] Test Time: 11.915829 seconds | |
| Concurrency 1000x- VERT.X CRASHES SO THIS TEST WON'T RUN |
Oops wrong instance type listed. I meant I ran on this instance:
High-Memory Quadruple Extra Large Instance
68.4 GB of memory
26 EC2 Compute Units (8 virtual cores with 3.25 EC2 Compute Units each)
1690 GB of instance storage
64-bit platform
I/O Performance: High
API name: m2.4xlarge
I don't believe in this ... i run vert.x test on my dual-core machine with 2gb ram and with loaded gnome interface and some programs with more than 10k reqs/sec!
ab -c 100 -n 100000 http://localhost:8082/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests
Server Software:
Server Hostname: localhost
Server Port: 8082
Document Path: /
Document Length: 76 bytes
Concurrency Level: 100
Time taken for tests: 9.541 seconds
Complete requests: 100000
Failed requests: 0
Write errors: 0
Total transferred: 15500000 bytes
HTML transferred: 7600000 bytes
Requests per second: 10480.67 #/sec
Time per request: 9.541 ms
Time per request: 0.095 ms
Transfer rate: 1586.43 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 4 60.7 0 1002
Processing: 0 5 7.9 4 659
Waiting: 0 5 7.9 4 658
Total: 0 9 62.3 4 1225
Percentage of the requests served within a certain time (ms)
50% 4
66% 5
75% 6
80% 7
90% 8
95% 10
98% 13
99% 15
100% 1225 (longest request)
It depends what those requests are doing. The test above involves file IO. It is an extension of this: https://gist.github.com/2652991
Simply posting that you can get much larger numbers of requests through vert.x is sort of a nonsense statement. Of course you can. I'm sure someone can push 30k requests through it. The point of this isn't to say "how many requests can you get through" but to compare the same activity on both platforms to see the performance comparison.
I can easily push 30k requests through nodejs on less hardware, far more than this test is showing. Again this is more of a file io test to compare to the bs vert.x microbenchmarks that came out a while back.
Your results are meaningless without an explanation of what you are testing exactly and some base comparison with something else doing the same workload.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The tests above were conducted on the following instance type:
High-CPU Extra Large Instance
7 GB of memory
20 EC2 Compute Units (8 virtual cores with 2.5 EC2 Compute Units each)
1690 GB of instance storage
64-bit platform
I/O Performance: High
API name: c1.xlarge