yuvalif

Cluster 1

start the cluster using mstart.sh:

MON=1 OSD=1 MDS=0 MGR=0 RGW=1 ../src/mstart.sh cluster1 -n -d

create a default realm:

bin/radosgw-admin -c run/cluster1/ceph.conf realm create --rgw-realm=myrealm --default

create a default master zonegroup:

yuvalif

run vstart with memstore (stop and run vstart before every test):

env MON=1 OSD=1 MDS=0 MGR=1 RGW=1 NFS=0 TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=134217728  ../src/vstart.sh -n -X  --nolockdep --without-dashboard -o debug_ms=0 -o debug_objecter=0  --memstore -o memstore_device_bytes=$((5 * 1024*1024*1024)) --rgw_frontend "beast tcp_nodelay=1 request_timeout_ms=0"

./bin/ceph osd set noscrub
./bin/ceph osd set nodeep-scrub

create a user for the test (make sure there is no bucket limit for it):

yuvalif

Expose Object to Lua Scripts in Ceph's Object Gateway

Goal

Ceph is a distributed storage system that supports: block, file, and object storage. All types of storage use the RADOS backend storage system. S3 compliant object storage is provided by the Object Gateway (a.k.a. the RADOS Gateway or the RGW).

Currently the RGW exposes metadata of the objects being uploaded or fetched through it to Lua scripting. This allows for different applications to be implemented in Lua, without changing the C++ codebase. This has great value, both in the skillset needed fro mthe application developrs, and the time it takes for these applications to be developed.

In this project, we will be enhancing our Lua binding in the RGW so that we can expose the content of an object when doing GET operations.

yuvalif

• install diff2html: npm install -g diff2html-cli
• for html view of commit run: git --no-pager show | diff2html -s side -i stdin -o stdout > .html

yuvalif

Minikube

install minikube

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-latest.x86_64.rpm
rpm -Uvh minikube-latest.x86_64.rpm

install and run libvirt

dnf install @virtualization
systemctl start libvirtd

yuvalif

• run kafka broker

bin/kafka-server-start.sh config/server.properties

• configure a topic to point to the kafka broker

aws --endpoint-url http://localhost:8000 sns create-topic --name=kafka_hstopic   --attributes='{"push-endpoint": "kafka://localhost", "persistent": "true"}'

• install hsbench (golang is needed for it)
• start a vstart cluster

yuvalif

• start vstart cluster:

MON=1 OSD=1 MDS=0 MGR=0 RGW=1 ../src/vstart.sh -n -d

• run elasticserach in a container:

podman run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.13.2

yuvalif

start cluster:

MON=1 OSD=1 MDS=0 MGR=0 RGW=1 ../src/vstart.sh -n -d 

start HTTP endpoint:

wget https://gist.githubusercontent.com/mdonkers/63e115cc0c79b4f6b8b3a6b797e485c7/raw/a6a1d090ac8549dac8f2bd607bd64925de997d40/server.py
python server.py 10900

yuvalif

Client Driven Geo-redundancy Library for Ceph Object Storage - Testing

Prerequisites

Please look here, to see how to run a Ceph cluster locally with the binaries you built. You probably would also like to check the developer guide and learn more on how to build Ceph and run it locally.

Basic RGW Testing

I usually use this command to start a new (hence the '-n' flag) local Ceph cluster with an RGW:

MON=1 OSD=1 RGW=1 MGR=0 MDS=0 ../src/vstart.sh -n -d

yuvalif

Security

• user/password with plaintext SASL
• SSL with the following options: disable certificate verification (useful for testing), provide explicit CA location
• other security options are optional

Idle Connection Cleanup

• based on idle timeout

Error Handling

• reconnection of closed/failed connection: probably just basic conf without failover