ELK - Remote clusters example.md

ELK - Remote clusters example

How to setup an Elasticsearch remote clusters lab.

Docker compose

Notes:

• Adapt the volumes definition
• Run with docker-compose up

File docker-compose.yml:

version: '3.7'

services:

  # Alpha cluster: http://alpha:9200
  elastic_alpha:
    image: "docker.elastic.co/elasticsearch/elasticsearch:7.4.2"
    networks:
      elastic:
        aliases:
          - alpha
    volumes:
      - .../elastic_alpha/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - .../elastic_alpha/data:/usr/share/elasticsearch/data
    environment:
      - "ES_JAVA_OPTS=-Xms250m -Xmx250m"

  # Beta cluster: http://beta:9200
  elastic_beta:
    image: "docker.elastic.co/elasticsearch/elasticsearch:7.4.2"
    networks:
      elastic:
        aliases:
          - beta
    volumes:
      - .../elastic_beta/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - .../elastic_beta/data:/usr/share/elasticsearch/data
    environment:
      - "ES_JAVA_OPTS=-Xms250m -Xmx250m"

  # Proxy cluster: http://proxy:9200
  elastic_proxy:
    image: "docker.elastic.co/elasticsearch/elasticsearch:7.4.2"
    networks:
      elastic:
        aliases:
          - proxy
    volumes:
      - .../elastic_proxy/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - .../elastic_proxy/data:/usr/share/elasticsearch/data
    environment:
      - "ES_JAVA_OPTS=-Xms250m -Xmx250m"

  kibana:
    image: "docker.elastic.co/kibana/kibana:7.4.2"
    networks:
      elastic:
        aliases:
          - kibana
    ports:
      - "5601:5601"
    volumes:
      - .../kibana/kibana.yml:/usr/share/kibana/config/kibana.yml


networks:
  elastic:
    driver: bridge

Cluster alpha

File elasticsearch.yml:

node:
  name: alpha

discovery:
  seed_hosts: alpha

cluster:
  name: alpha_cluster
  initial_master_nodes: alpha

network:
  host: 0.0.0.0

Cluster beta

File elasticsearch.yml:

node:
  name: beta

discovery:
  seed_hosts: beta

cluster:
  name: beta_cluster
  initial_master_nodes: beta

network:
  host: 0.0.0.0

Cluster proxy

File elasticsearch.yml:

node:
  name: proxy

discovery:
  seed_hosts: proxy

cluster:
  name: proxy_cluster
  initial_master_nodes: proxy
  # Remote clusters definition
  remote:
    # Alpha cluster
    alpha_cluster:
      seeds: alpha:9300
      skip_unavailable: true
    # Beta cluster
    beta_cluster:
      seeds: beta:9300
      skip_unavailable: true

network:
  host: 0.0.0.0

Kibana

• Kibana is bound to the proxy cluster
• Kibana have full access to the proxy cluster (e.g. GET, POST, DELETE, ...)
• Kibana have read-only access to the remote clusters alpha_cluster and beta_cluster (e.g. GET)

File kibana.yml:

server:
  host: "0.0.0.0"

elasticsearch:
  hosts:
    - "http://proxy:9200"