Skip to content

Instantly share code, notes, and snippets.

@hhcordero

hhcordero/1_Dockerised_JMeter.md

Last active March 30, 2022 13:44
Show Gist options
  • Save hhcordero/abd1dcaf6654cfe51d0b to your computer and use it in GitHub Desktop.
Save hhcordero/abd1dcaf6654cfe51d0b to your computer and use it in GitHub Desktop.
Download ZIP
Dockerized JMeter - A Distributed Load Testing Workflow
Raw
1_Dockerised_JMeter.md

Dockerized JMeter - A Distributed Load Testing Workflow

Supported Cloud Providers:

Images used:

Pre-requisites:

  1. Docker and Docker-Machine cli installed on your host (https://docs.docker.com/installation/)

  2. You can choose either Amazon or DigitalOcean

    Amazon:

    • Access Key
    • Secret Key
    • Region
    • VPC ID
    • Subnet ID
    • Zone
    • AMI

    DigitalOcean:

    • Access Token

  3. JMeter test plan on your host (http://jmeter.apache.org/usermanual/build-web-test-plan.html)

Steps:

  1. Provision machine for JMeter Master (Non-gui mode)

    For AWS:

    $ ./launch_jmeter_master_aws

    For DigitalOcean:

    $ ./launch_jmeter_master_do

    For local machine:

    $ ./launch_jmeter_master_local

  2. Provision machine for JMeter Slave (Server mode)

    Optional parameter - indicate number of servers to provision, value between 1 to 20. Default to 1 if no parameter is given.

    For AWS:

    $ ./launch_jmeter_slave_aws 2

    For DigitalOcean:

    $ ./launch_jmeter_slave_do 2

  3. Copy test plan (jmx file) to jmeter-master machine inside /load_tests

    Connect to jmeter-master machine and create /load_tests directory

    $ eval "$(docker-machine env --swarm jmeter-master)"
$ docker-machine ssh jmeter-master

    For AWS:

    $ sudo mkdir /load_tests && sudo chown ubuntu:ubuntu /load_tests

    For DigitalOcean:

    $ mkdir /load_tests

    For local machine:

    $ sudo mkdir /load_tests && sudo chown docker:docker /load_tests

    Then go back to host

    $ exit

    Copy test plan from host to jmeter-master machine

    Syntax:

    docker-machine scp [path to test directory] [machine name]:/load_tests

    Example:

    $ docker-machine scp -r /home/user/docker-jmeter-master/load_tests/my_test jmeter-master:/load_tests

    Then go back to host

    $ exit

  4. Run JMeter load test - execute the following commands, replace values if necessary before executing

    Get JMeter master machine ip

    $ IP=$(docker-machine ip jmeter-master)

    Replace the value with the output from ./launch_jmeter_slave - must be comma separated jmeter slave ip addresses

    $ REMOTE_HOSTS=""

    Parent directory for the test plan

    $ TEST_DIR="my_test"

    Test plan without file extension

    $ TEST_PLAN="test-plan"

    Run the JMeter master non-gui and perform load test

    $ docker run \
    --detach \
    --publish 1099:1099 \
    --volume /load_tests/$TEST_DIR:/load_tests/$TEST_DIR \
    --env TEST_DIR=$TEST_DIR \
    --env TEST_PLAN=$TEST_PLAN \
    --env IP=$IP \
    --env REMOTE_HOSTS=$REMOTE_HOSTS \
    --env constraint:type==master \
    hhcordero/docker-jmeter-client

    To monitor output, follow the logs:

    Syntax:

    docker logs -f [container name]

    Example:

    $ docker logs -f jmeter-master/tender_feynman

  5. Save the result, look for the .jtl file inside jmeter-master /load_tests/$TEST_DIR

    Syntax:

    docker-machine scp [machine name]:/load_tests/[test dir]/[test plan result] [path to test directory]

    Example:

    $ docker-machine scp jmeter-master:/load_tests/${TEST_DIR}/${TEST_PLAN}.jtl /home/user/docker-jmeter-master/load_tests/my_test/.
Raw
1_launch_jmeter_master_local
#!/bin/bash
# replace env file to use, default to digitalocean
app_env='app_do.env'
# Generate token - use swarm for service discovery
TOKEN=$(docker run --rm swarm create)
echo "export TOKEN=$TOKEN" >> ./$app_env
# Provision new machine
echo -e "
Provision machine for JMeter (Master):
docker-machine create
--driver virtualbox
--engine-label type=master
--swarm
--swarm-master
--swarm-discovery token://$TOKEN
jmeter-master"
docker-machine create \
--driver virtualbox \
--engine-label type=master \
--swarm \
--swarm-master \
--swarm-discovery token://$TOKEN \
jmeter-master
echo "To manage swarm, make sure to run: eval \"\$(docker-machine env --swarm jmeter-master)\""
exit
Raw
2_app_aws.env
#!/bin/bash
export TOKEN=[auto-populate by running launch_jmeter_master_aws]
export AWS_ACCESS_KEY_ID=[access key here]
export AWS_SECRET_ACCESS_KEY=[secret key here]
export AWS_DEFAULT_REGION=[region here ie. ap-southeast-1]
export AWS_VPC_ID=[vpc id here ie. vpc-xxxxxxxx]
export AWS_SUBNET_ID=[subnet id here ie. subnet-xxxxxxxx]
export AWS_ZONE=[zone here ie. a]
export AWS_AMI=[ami id here ie. ami-b899a2ea]
Raw
2_launch_jmeter_master_aws
#!/bin/bash
# Generate token - use swarm for service discovery
TOKEN=$(docker run --rm swarm create)
sed -i "/export TOKEN=/c\export TOKEN=$TOKEN" ./app_aws.env
# Initialise env variables
source ./app_aws.env
# Provision new machine
echo -e "
Provision machine for JMeter (Master):
docker-machine --debug create
--driver amazonec2
--amazonec2-access-key $AWS_ACCESS_KEY_ID
--amazonec2-secret-key $AWS_SECRET_ACCESS_KEY
--amazonec2-region $AWS_DEFAULT_REGION
--amazonec2-vpc-id $AWS_VPC_ID
--amazonec2-subnet-id $AWS_SUBNET_ID
--amazonec2-zone $AWS_ZONE
--amazonec2-ami $AWS_AMI
--engine-label type=master
--swarm
--swarm-master
--swarm-discovery token://$TOKEN
jmeter-master"
docker-machine create \
--driver amazonec2 \
--amazonec2-access-key $AWS_ACCESS_KEY_ID \
--amazonec2-secret-key $AWS_SECRET_ACCESS_KEY \
--amazonec2-region $AWS_DEFAULT_REGION \
--amazonec2-vpc-id $AWS_VPC_ID \
--amazonec2-subnet-id $AWS_SUBNET_ID \
--amazonec2-zone $AWS_ZONE \
--amazonec2-ami $AWS_AMI \
--engine-label type=master \
--swarm \
--swarm-master \
--swarm-discovery token://$TOKEN \
jmeter-master
echo "To manage swarm, make sure to run: eval \"\$(docker-machine env --swarm jmeter-master)\""
exit
Raw
2_launch_jmeter_slave_aws
#!/bin/bash
# Default to 1 if no parameter is given
count=${1-1}
# Max of 20 machines only, change accordingly
max=20
if ! [[ $count =~ ^[0-9]+$ ]] || ! [[ "$count" -gt 0 && "$count" -le "$max" ]]; then
echo "[ERROR] Parameter is not a positive integer (must be 1 to $max)." >&2; exit 1
fi
if [ "$count" -gt "$max" ]; then
count=$max
fi
# Initialise env variables
source ./app_aws.env
iterator=1
while [ "$iterator" -le "$count" ]; do
# Provision new machine
echo -e "
Provision machine for JMeter (Slave):
docker-machine create
--driver amazonec2
--amazonec2-access-key $AWS_ACCESS_KEY_ID
--amazonec2-secret-key $AWS_SECRET_ACCESS_KEY
--amazonec2-region $AWS_DEFAULT_REGION
--amazonec2-vpc-id $AWS_VPC_ID
--amazonec2-subnet-id $AWS_SUBNET_ID
--amazonec2-zone $AWS_ZONE
--amazonec2-ami $AWS_AMI
--engine-label type=slave
--swarm
--swarm-discovery token://$TOKEN
jmeter-slave-$iterator"
docker-machine create \
--driver amazonec2 \
--amazonec2-access-key $AWS_ACCESS_KEY_ID \
--amazonec2-secret-key $AWS_SECRET_ACCESS_KEY \
--amazonec2-region $AWS_DEFAULT_REGION \
--amazonec2-vpc-id $AWS_VPC_ID \
--amazonec2-subnet-id $AWS_SUBNET_ID \
--amazonec2-zone $AWS_ZONE \
--amazonec2-ami $AWS_AMI \
--engine-label type=slave \
--swarm \
--swarm-discovery token://$TOKEN \
jmeter-slave-$iterator
# Set env variables to point to newly created machine
eval "$(docker-machine env jmeter-slave-$iterator)"
# Get public ip
ip=$(docker-machine ip jmeter-slave-$iterator)
# Run jmeter on new machine
echo -e "
Run JMeter in Server Mode:
docker run
--detach
--publish 1099:1099
--env IP=$ip
hhcordero/docker-jmeter-server"
docker run \
--detach \
--publish 1099:1099 \
--env IP=$ip \
hhcordero/docker-jmeter-server
# Concatenate slave ip addresses
server_ips+="$ip,"
let "iterator += 1"
done
echo "Slave IP's, for use in JMeter Master: $(echo $server_ips | sed 's/,*$//')"
exit
Raw
3_app_do.env
#!/bin/bash
export TOKEN=[auto-populate by running launch_jmeter_master_do]
export DIGITALOCEAN_ACCESS_TOKEN=[access token here]
export DIGITALOCEAN_REGION=[region here]
export DIGITALOCEAN_SIZE=[memory here]
Raw
3_launch_jmeter_master_do
#!/bin/bash
# Generate token - use swarm for service discovery
TOKEN=$(docker run --rm swarm create)
sed -i "/export TOKEN=/c\export TOKEN=$TOKEN" ./app_do.env
# Initialise env variables
source ./app_do.env
# Provision new machine
echo -e "
Provision machine for JMeter (Master):
docker-machine create
--driver digitalocean
--digitalocean-access-token $DIGITALOCEAN_ACCESS_TOKEN
--digitalocean-region $DIGITALOCEAN_REGION
--digitalocean-size $DIGITALOCEAN_SIZE
--engine-label type=master
--swarm
--swarm-master
--swarm-discovery token://$TOKEN
jmeter-master"
docker-machine create \
--driver digitalocean \
--digitalocean-access-token $DIGITALOCEAN_ACCESS_TOKEN \
--digitalocean-region $DIGITALOCEAN_REGION \
--digitalocean-size $DIGITALOCEAN_SIZE \
--engine-label type=master \
--swarm \
--swarm-master \
--swarm-discovery token://$TOKEN \
jmeter-master
echo "To manage swarm, make sure to run: eval \"\$(docker-machine env --swarm jmeter-master)\""
exit
Raw
3_launch_jmeter_slave_do
#!/bin/bash
# Default to 1 if no parameter is given
count=${1-1}
# Max of 20 machines only, change accordingly
max=20
if ! [[ $count =~ ^[0-9]+$ ]] || ! [[ "$count" -gt 0 && "$count" -le "$max" ]]; then
echo "[ERROR] Parameter is not a positive integer (must be 1 to $max)." >&2; exit 1
fi
if [ "$count" -gt "$max" ]; then
count=$max
fi
# Initialise env variables
source ./app_do.env
iterator=1
while [ "$iterator" -le "$count" ]; do
# Provision new machine
echo -e "
Provision machine for JMeter (Slave):
docker-machine create
--driver digitalocean
--digitalocean-access-token $DIGITALOCEAN_ACCESS_TOKEN
--digitalocean-region $DIGITALOCEAN_REGION
--digitalocean-size $DIGITALOCEAN_SIZE
--engine-label type=slave
--swarm
--swarm-discovery token://$TOKEN
jmeter-slave-$iterator"
docker-machine create \
--driver digitalocean \
--digitalocean-access-token $DIGITALOCEAN_ACCESS_TOKEN \
--digitalocean-region $DIGITALOCEAN_REGION \
--digitalocean-size $DIGITALOCEAN_SIZE \
--engine-label type=slave \
--swarm \
--swarm-discovery token://$TOKEN \
jmeter-slave-$iterator
# Set env variables to point to newly created machine
eval "$(docker-machine env jmeter-slave-$iterator)"
# Get public ip
ip=$(docker-machine ip jmeter-slave-$iterator)
# Run jmeter on new machine
echo -e "
Run JMeter in Server Mode:
docker run
--detach
--publish 1099:1099
--env IP=$ip
hhcordero/docker-jmeter-server"
docker run \
--detach \
--publish 1099:1099 \
--env IP=$ip \
hhcordero/docker-jmeter-server
# Concatenate slave ip addresses
server_ips+="$ip,"
let "iterator += 1"
done
echo "Slave IP's, for use in JMeter Master: $(echo $server_ips | sed 's/,*$//')"
exit
@hhcordero
Copy link
Author

hhcordero commented Jul 8, 2016 •
edited
Loading

Seems like connectivity issue, did you set the --env IP=$IP parameter when you run the jmeter-client (slave) container?

@burzum619
Copy link

It turned out my security group did not have 1099 inbound port open. Once I took care of that, it worked perfect. Thank you for your work, great job!

@jesperrasmussen
Copy link

Hi Hector!

I'm trying to build a small shellscript, to interactively take a location of a test and run it over a number of servers - to automatically run the setup you're describing.

I noticed that the actual execution of the test might have an issue - you write the following:

$ docker run
--detach
--publish 1099:1099
--volume /load_tests/$TEST_DIR:/load_tests/$TEST_DIR
--env TEST_DIR=$TEST_DIR
--env TEST_PLAN=$TEST_PLAN
--env IP=$IP
--env REMOTE_HOSTS=$REMOTE_HOSTS
--env constraint:type==master
hhcordero/docker-jmeter-master

However, the image hhcordero/docker-jmeter-master is not available on Hub I think?

My guess would be that it should be the "docker-jmeter-client" image, but when I change to using that - Docker tells me:

docker: Error response from daemon: Error: image library/pre-create:latest not found.

Do you know what the cause of this is? Has anything changed regarding the images? :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment