Keystone performance benchmarking, using devstack stable/kilo

_Fernet_results.md

Fernet token creation

Time per request: 62.748 [ms] (mean)

Requests per second: 74.76 [#/sec] (mean)

Fernet token validation

Time per request: 93.782 [ms] (mean)

Requests per second: 48.26 [#/sec] (mean)

_summary.md

Response time

Fernet token creation response time is 13% faster than UUID.

Fernet token validation response time is 400% slower than UUID.

Requests per second

(default devstack configuration)

Fernet token creation requests per second is 30% faster than UUID.

Fernet token validation requests per second is 81% slower than UUID.

_UUID_results.md

UUID token creation

Time per request: 72.454 [ms] (mean)

Requests per second: 57.60 [#/sec] (mean)

UUID token validation

Time per request: 18.774 [ms] (mean)

Requests per second: 256.69 [#/sec] (mean)

auth_request_admin.json

{"auth": {"scope": {"project": {"name": "admin", "domain": {"id": "default"}}}, "identity": {"methods": ["password"], "password": {"user": {"domain": {"id": "default"}, "name": "admin", "password": "nomoresecrete"}}}}}

auth_request_demo.json

{"auth": {"scope": {"project": {"name": "demo", "domain": {"id": "default"}}}, "identity": {"methods": ["password"], "password": {"user": {"domain": {"id": "default"}, "name": "demo", "password": "nomoresecrete"}}}}}

benchmark.sh

#!/bin/bash
set -e

HOST=${HOST:-127.0.0.1}
SECONDS_PER_BENCHMARK=${T:-60}
CONCURRENCY=${C:-20}

function authenticate() {
    BODY=`cat $1`
    X_SUBJECT_TOKEN=`http --verbose post $ENDPOINT/v3/auth/tokens "$BODY" --content-type="application/json" | grep "X-Subject-Token: "`
    TOKEN=${X_SUBJECT_TOKEN##*:}
}

ENDPOINT="http://$HOST:35357"

# generate an admin token for later
authenticate auth_request_admin.json
ADMIN_TOKEN=$TOKEN
echo "Admin token: $ADMIN_TOKEN"

authenticate auth_request_demo.json
DEMO_TOKEN=$TOKEN
echo "Demo token: $DEMO_TOKEN"

# I'm sure you can do this in bash, but this seemed easier.
TOKEN_FORMAT=`python -c "print('UUID' if len('$ADMIN_TOKEN'.strip()) == 32 else 'Fernet')"`

# reset results file
: > _${TOKEN_FORMAT}_results.md

echo "Benchmarking token creation (1/2)..."
echo "## $TOKEN_FORMAT token creation" >> _${TOKEN_FORMAT}_results.md
echo >> _${TOKEN_FORMAT}_results.md
ab -t $SECONDS_PER_BENCHMARK -c 1 -p auth_request_demo.json -T "application/json" $ENDPOINT/v3/auth/tokens | grep "(mean)" | grep "Time per request" >> _${TOKEN_FORMAT}_results.md
echo "Benchmarking token creation (2/2)..."
echo >> _${TOKEN_FORMAT}_results.md
ab -t $SECONDS_PER_BENCHMARK -c $CONCURRENCY -p auth_request_demo.json -T "application/json" $ENDPOINT/v3/auth/tokens | grep "Requests per second:" >> _${TOKEN_FORMAT}_results.md

echo "Benchmarking token validation (1/2)..."
echo >> _${TOKEN_FORMAT}_results.md
echo "## $TOKEN_FORMAT token validation" >> _${TOKEN_FORMAT}_results.md
echo >> _${TOKEN_FORMAT}_results.md
ab -t $SECONDS_PER_BENCHMARK -c 1 -T "application/json" -H "X-Auth-Token: $ADMIN_TOKEN" -H "X-Subject-Token: $TOKEN" $ENDPOINT/v3/auth/tokens | grep "(mean)" | grep "Time per request" >> _${TOKEN_FORMAT}_results.md
echo "Benchmarking token validation (2/2)..."
echo >> _${TOKEN_FORMAT}_results.md
ab -t $SECONDS_PER_BENCHMARK -c $CONCURRENCY -T "application/json" -H "X-Auth-Token: $ADMIN_TOKEN" -H "X-Subject-Token: $TOKEN" $ENDPOINT/v3/auth/tokens | grep "Requests per second:" >> _${TOKEN_FORMAT}_results.md

# let's see the results!
cat _${TOKEN_FORMAT}_results.md

pypi-requirements.txt

httpcli

ab - Apache HTTP server benchmarking tool

The program 'ab' is currently not installed. You can install it by typing:
sudo apt install apache2-utils