docs.rs benchmarks

bench.txt

# summary

docs.rs is _much_ faster than my hard disk.
Adding concurrency reduces overall time but increases individual time
Most of the delay on current prod is due to network latency, not CPU time
I cannot reproduce the CPU usage locally - requests are just as fast or faster on the version that overloaded prod as on the latest version.

## extended timing info

### Current prod (I _believe_ c3d3ed3d2e0c4274c0c12a7b1f5e56e68eb8b8bf, remote)

#### With concurrency

$ ab -c 100 -t 6 docs.rs/cranelift/0.44.0/cranelift/
Document Path:          /cranelift/0.44.0/cranelift/
Document Length:        183 bytes

Concurrency Level:      100
Time taken for tests:   6.000 seconds
Complete requests:      19246
Failed requests:        0
Non-2xx responses:      19248
Total transferred:      11452560 bytes
HTML transferred:       3522384 bytes
Requests per second:    3207.55 [#/sec] (mean)
Time per request:       31.176 [ms] (mean)
Time per request:       0.312 [ms] (mean, across all concurrent requests)
Transfer rate:          1863.96 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        8   15   2.2     15      44
Processing:     9   16   5.5     16     256
Waiting:        9   16   5.2     15     219
Total:         21   31   5.7     30     273

Percentage of the requests served within a certain time (ms)
  50%     30
  66%     31
  75%     32
  80%     33
  90%     35
  95%     37
  98%     41
  99%     43
 100%    273 (longest request)

#### Without concurrency

$ ab -t 6 docs.rs/cranelift/0.44.0/cranelift/
Document Path:          /cranelift/0.44.0/cranelift/
Document Length:        183 bytes

Concurrency Level:      1
Time taken for tests:   6.009 seconds
Complete requests:      341
Failed requests:        0
Non-2xx responses:      341
Total transferred:      202895 bytes
HTML transferred:       62403 bytes
Requests per second:    56.75 [#/sec] (mean)
Time per request:       17.620 [ms] (mean)
Time per request:       17.620 [ms] (mean, across all concurrent requests)
Transfer rate:          32.98 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        7    8   1.4      8      18
Processing:     7    9   4.7      8      86
Waiting:        7    9   4.7      8      86
Total:         14   17   4.9     16      94

Percentage of the requests served within a certain time (ms)
  50%     16
  66%     17
  75%     18
  80%     19
  90%     20
  95%     22
  98%     26
  99%     29
 100%     94 (longest request)

### Bad version (62b8f70ecc09143e563f9ce530822930247e0306, local)

#### With concurrency

$ ab -c 100 -n 1000 localhost:3000/cranelift/0.44.0/cranelift/
Document Path:          /cranelift/0.44.0/cranelift/
Document Length:        14163 bytes

Concurrency Level:      100
Time taken for tests:   1.767 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      14282000 bytes
HTML transferred:       14163000 bytes
Requests per second:    565.78 [#/sec] (mean)
Time per request:       176.748 [ms] (mean)
Time per request:       1.767 [ms] (mean, across all concurrent requests)
Transfer rate:          7891.04 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   2.4      0      14
Processing:    36  168  35.0    168     288
Waiting:       35  165  34.5    164     288
Total:         47  169  33.7    168     288

Percentage of the requests served within a certain time (ms)
  50%    168
  66%    179
  75%    188
  80%    193
  90%    210
  95%    225
  98%    244
  99%    255
 100%    288 (longest request)

#### Without concurrency

$ ab -n 1000 localhost:3000/cranelift/0.44.0/cranelift/
Document Path:          /cranelift/0.44.0/cranelift/
Document Length:        14163 bytes

Concurrency Level:      1
Time taken for tests:   4.376 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      14282000 bytes
HTML transferred:       14163000 bytes
Requests per second:    228.50 [#/sec] (mean)
Time per request:       4.376 [ms] (mean)
Time per request:       4.376 [ms] (mean, across all concurrent requests)
Transfer rate:          3186.88 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     3    4   0.7      4      19
Waiting:        3    4   0.7      4      18
Total:          4    4   0.7      4      20

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      4
  80%      5
  90%      5
  95%      5
  98%      5
  99%      6
 100%     20 (longest request)

### Known good version (799f5d214405ed3a32fb5616791d83382c4becef, local)

#### With concurrency

$ ab -c 100 -n 1000 http://localhost:3000/cranelift/0.44.0/cranelift/
Document Path:          /cranelift/0.44.0/cranelift/
Document Length:        14163 bytes

Concurrency Level:      100
Time taken for tests:   2.743 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      14282000 bytes
HTML transferred:       14163000 bytes
Requests per second:    364.62 [#/sec] (mean)
Time per request:       274.257 [ms] (mean)
Time per request:       2.743 [ms] (mean, across all concurrent requests)
Transfer rate:          5085.48 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.9      0       5
Processing:    43  262  42.3    265     407
Waiting:       43  254  41.6    256     393
Total:         48  263  41.7    266     410

Percentage of the requests served within a certain time (ms)
  50%    266
  66%    277
  75%    284
  80%    290
  90%    308
  95%    318
  98%    343
  99%    359
 100%    410 (longest request)

#### Without concurrency

$ ab -n 1000 http://localhost:3000/cranelift/0.44.0/cranelift/
Document Path:          /cranelift/0.44.0/cranelift/
Document Length:        14163 bytes

Concurrency Level:      1
Time taken for tests:   7.276 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      14282000 bytes
HTML transferred:       14163000 bytes
Requests per second:    137.43 [#/sec] (mean)
Time per request:       7.276 [ms] (mean)
Time per request:       7.276 [ms] (mean, across all concurrent requests)
Transfer rate:          1916.79 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     6    7   1.1      7      24
Waiting:        6    7   1.1      7      23
Total:          6    7   1.1      7      24

Percentage of the requests served within a certain time (ms)
  50%      7
  66%      7
  75%      7
  80%      8
  90%      8
  95%      8
  98%      9
  99%     10
 100%     24 (longest request)