Ceph RGW multisite tests suite

Running Tests Locally

you run the tests locally from your "build" directory in your test environment
then you need to add a "test_multi.conf" file:

[DEFAULT]
num_zonegroups=1
num_zones=1
num_ps_zones=0
gateways_per_zone=1
no_bootstrap=false
log_level=5

Note that you would need "num_ps_zones=1" to run the pubsub tests. But in your case you can keep it at zero. The First time you run tests, you would need to have "no_bootstrap=false", but in any consequent run, you would need to set it to "no_bootstrap=true", so that the cluster is not re-created. If you want to change the setup (e.g. number of gateways per zone), you would need to re-create the cluster. I do that by running:

$ /path/to/ceph/src/mstop.sh c1 && /path/to/ceph/src/mstop.sh c2

(assuming I have 2 gateways). Setting "no_bootstrap=false" and invoking the test again.

To invoke the tests (after the conf file is ready), I use:

$ MON=1 OSD=1 MGR=0 MSD=0 RGW_MULTI_TEST_CONF=./test_multi.conf nosetests -s /path/to/ceph/src/test/rgw/test_multi.py

To run only notification and pubsub related tests, I use the -m flag for nosetest:

$ MON=1 OSD=1 MGR=0 MSD=0 RGW_MULTI_TEST_CONF=./test_multi.conf nosetests -s /path/to/ceph/src/test/rgw/test_multi.py -m "test_ps_*"

To run a specific test (e.g. "test_ps_s3_versioned_deletion"), use the following format:

$ MON=1 OSD=1 MGR=0 MSD=0 RGW_MULTI_TEST_CONF=./test_multi.conf nosetests -s /path/to/ceph/src/test/rgw/test_multi.py:test_ps_s3_versioned_deletion

The multisite setup is used in teuthology as well, however, in teuthology it cannot run tests that push notification to HTTP, AMQP, etc. This is why there is a variable called: "skip_push_tests" which is set to "True" in the file. If you run the tests locally, you can enable the "push" tests, by setting it to "False".

AMQP

In order to run the amqp tests you need to install rabbitmq:

$ sudo dnf install rabbitmq-server

dont need to do anything else, since the test is running the rabbitmq broker by itself.

kafka

In order to run the kafka tests you need to install kafka. this is little more complex than rabbitmq.
install kafka: download and unzip [1]
the tests are running zookeeper and kafka binaries (see the init_kafka() function), so you don't need to run them yourself
unless you want to tests security, there is no need to change anything with the kafka configuration file
even if you don't test security, you would still need to create one file on the kafka config directory, called "kafka_client_jaas.conf":

KafkaClient {
          org.apache.kafka.common.security.plain.PlainLoginModule required
          username="alice"
          password="alice-secret";
      };

when you run kafka tests you need to set the KAFKA_DIR env variable, e.g.:

$ MON=1 OSD=1 MGR=0 MSD=0 KAFKA_DIR=/path/to/kafka RGW_MULTI_TEST_CONF=./test_multi.conf nosetests -s /path/to/ceph/src/test/rgw/test_multi.py

[1] https://www.apache.org/dyn/closer.cgi?path=/kafka/2.6.0/kafka-2.6.0-src.tgz

yuvalif/rgw_multisite_tests.md

Running Tests Locally

AMQP

kafka