Set up Aerospike on Docker Swarm

aerospike_setup.md

Set up Aerospike/docker:

Set up EC2-Instances. I used i3.4xlarge. You need the following inbound rules on the Security Group:

• TCP 3000 - 3004 (Aerospike heartbeat, gossip etc)
• TCP 2377 (Docker Swarm cluster management, Raft)
• TCP/UDP 7946 (Docker Swarm control plane, gossip service discovery)
• UDP 4789 (Docker Swarm overlay network traffic)
• ESP (50) (Docker Swarm overlay network encryption, select "Custom Protocol" and type in 50)

See:
https://docs.docker.com/engine/swarm/swarm-tutorial/#open-protocols-and-ports-between-the-hosts
https://www.aerospike.com/docs/operations/configure/network/
https://gist.github.com/BretFisher/7233b7ecf14bc49eb47715bbeb2a2769

I have this set up under the "aerospike" security group

ON ALL NODES: Mount the SSD devices (this is i3.4xlarge)

sudo mkfs.ext4 /dev/nvme0n1
sudo mkfs.ext4 /dev/nvme1n1
sudo mkdir /mnt0
sudo mkdir /mnt1
sudo mount -t ext4 /dev/nvme0n1 /mnt0
sudo mount -t ext4 /dev/nvme1n1 /mnt1

ON ALL NODES: Install docker

sudo yum install docker -y

ON ALL NODES: make docker accessible without sudo

sudo usermod -aG docker ec2-user
sudo su - ec2-user  # apply changes

ON ALL NODES: start docker daemon and increase file limits for docker containers

sudo service docker start
sudo vim /etc/sysconfig/docker # increase file limits. I do OPTIONS="--default-ulimit nofile=102400:409600"
sudo service docker restart

Now pick a node you want to be the docker swarm master. Only do this on single node.

ON MASTER: Initialize docker swarm

docker swarm init --advertise-addr `hostname -i`

ON ALL NON-MASTER NODES. It should tell you the $TOKEN and $MASTER_IP when you initialized the swarm.

docker swarm join --token $TOKEN $MASTER_IP:2377

Verify that all nodes are in the Swarm

docker node ls

ON MASTER: fetch docker-compose and aerospike conf from github

git clone https://github.com/cozos/aerospike-docker-swarm.git
cd aerospike-docker-swarm

Deploy the aerospike server.

docker stack deploy -c aerospike.yml aerospike

Wait for a couple seconds and then check aerospike node info

docker run --net aerospike_aerospikenetwork aerospike/aerospike-tools asadm -h aerospikedb -e info

run AQL on aerospike db

docker exec -it `docker ps -q --filter "name=aerospike_aerospikedb"` aql

try some statements

insert into test.foo (PK, foo) values ('123','my string')
select * from test.foo

More info:

https://github.com/cozos/aerospike-docker-swarm/blob/master/README.md https://www.aerospike.com/docs/deploy_guides/docker/ https://www.aerospike.com/docs/deploy_guides/docker/orchestrate/ https://www.aerospike.com/docs/deploy_guides/docker/recommendations/

Now you can try accessing Aerospike cluster from Java client

in profiles/pom.xml:

    <dependency>
        <groupId>com.aerospike</groupId>
        <artifactId>aerospike-client</artifactId>
        <version>4.1.11</version>
    </dependency>

package profiles.lib;

import com.aerospike.client.AerospikeClient;
import com.aerospike.client.Bin;
import com.aerospike.client.Key;
import com.aerospike.client.Record;

public class AerospikeWriter {
  public static String AEROSPIKE_HOST = "172.18.225.153";
 
  public static void main(String [] args) {
    AerospikeClient client = new AerospikeClient(AEROSPIKE_HOST, 3000);

    Key key = new Key("test", "foo", "putgetkey");
    Bin bin1 = new Bin("foo", "falala");

    // // Write a record
    client.put(null, key, bin1);

    // Read a record
    Record record = client.get(null, key);

    System.out.println(record);

    client.close();
  }
}