Skip to content

Instantly share code, notes, and snippets.

View glenacota's full-sized avatar

Guido Lena Cota glenacota

View GitHub Profile
@glenacota
glenacota / elastic_exercise_shard_allocation
Last active March 3, 2019 21:57
An exercise to practice with shard allocation strategies in Elasticsearch.
# GOAL: Allocate the shards in a way that satisfies a given set of requirements
# INITIAL SETUP: /
# Download the latest 6.x version of Elasticsearch and Kibana
# Deploy the cluster `eoc-06-cluster`, with three nodes named `node1`, `node2`, and `node3`
# Configure the Zen Discovery module of each node so that they can communicate with each other
# Connect a Kibana instance to `node3`
# Start the cluster
# Create the index `hamlet-1` with two primary shards and one replica
# Add some documents to `hamlet-1` by running the following _bulk command
@glenacota
glenacota / elastic_exercise_deploy_and_configure
Last active March 3, 2019 21:57
An exercise to practice with the deployment and configuration of an Elasticsearch cluster.
# GOAL: Deploy an Elasticsearch cluster that satisfies a given set of requirements
# INITIAL SETUP: /
# Download the latest 6.x version of Elasticsearch and Kibana
# Deploy the cluster `eoc-01-cluster`, so that it satisfies the following requirements: (i) has three nodes, named `node1`, `node2`, and `node3`, (ii) all nodes are eligible master nodes
# Configure the nodes to avoid the split brain scenario
# Configure `node1` so that the node (i) is a data node but not an ingest node, (ii) is bound to the network address "192.168.0.100" and HTTP port "9201", (iii) doesn't allow swapping on its host
# Configure the Zen Discovery module of `node2` and `node3` to use the address and default transport port of `node1`
# Configure the JVM settings of each node so that it uses a minimum and maximum of 8 GB for the heap
# Configure the logging settings of each node so that (i) the logs directory is not the default one, (ii) the log level for transport-related events is set to "debug"
@glenacota
glenacota / elastic_exercise_security
Last active March 3, 2019 21:58
An exercise to practice with security in Elasticsearch.
# GOAL: Secure a cluster and an index using Elasticsearch Security
# INITIAL SETUP: (i) a running Elasticsearch cluster with at least one node and a Kibana instance, (ii) no index named `hamlet`
# Copy-paste the following instructions into your Kibana console, and work directly from there
# Enable xPack security on the cluster
# Set the password of the `elastic` and `kibana` built-in users, by using the pattern "{{username}}-password" (e.g., "elastic-password")
# Login to Kibana using the `elastic` user credentials
# Create the index `hamlet` and add some documents by running the following _bulk command
PUT hamlet/_doc/_bulk
{"index":{"_index":"hamlet","_id":0}}
@glenacota
glenacota / elastic_exercise_mapping
Last active February 22, 2019 09:13
An extended exercise that covers the "Mapping and Text Analysis" objective of the Elastic exam.
# ** EXAM OBJECTIVES: MAPPINGS AND TEXT ANALYSIS **
# (remove, if present, any `hamlet*` index and index template)
# Create the index `hamlet_1`, with one primary shard and no replicas
# Define the mapping for `hamlet_1`, satisfying the following criteria: (i) has a type "_doc" with three string fields named `speaker`, `line_number`, and `text_entry`; (ii) only `text_entry` is analysed; (iii) `text_entry` has a multi-field named `english`, associated with the built-in "english" analyzer; (iv) no aggregations supported by `line_number`
# Populate `hamlet_1` by running the _bulk command with the request-body below
{"index":{"_index":"hamlet_1","_id":0}}
{"line_number":"1.1.1","speaker":"BERNARDO","text_entry":"Whos there?"}
{"index":{"_index":"hamlet_1","_id":1}}
{"line_number":"1.1.2","speaker":"FRANCISCO","text_entry":"Nay, answer me: stand, and unfold yourself."}
{"index":{"_index":"hamlet_1","_id":2}}
@glenacota
glenacota / elastic_exercise_indexing_and_mapping
Last active February 21, 2019 21:57
An extended exercise that covers the "Indexing Data" and part of the "Mapping and Text Analysis" objectives of the Elastic exam.
# ** EXAM OBJECTIVES: INDEXING DATA + MAPPINGS AND TEXT ANALYSIS **
# (remove, if present, any `hamlet*` index and index template)
# Create the index `hamlet_raw`, with one primary shard and four replicas
# Index in `hamlet_raw` a document that satisfies the following criteria: (i) has id "1"; (ii) has default type; (iii) has a field `line` with value "To be, or not to be: that is the question"
# Update the document with id "1" by adding the field `line_number` with value "3.1.64"
# Index in `hamlet_raw` a new document without specifying any id. The fields of this document are: (i) `text_entry` with value "Whether tis nobler in the mind to suffer"; (ii) `line_number` with value "3.1.66"
# Update the precedent document by setting `line_number` to "3.1.65"
# (in one request) Update all documents in `hamlet_raw` by adding a new field `speaker` with value "HAMLET"
# Update the document with id "1" by renaming the field `line` into `text_entry`
# Delete the `hamlet_raw` index
@glenacota
glenacota / elastic_exercise_config_and_admin
Last active February 20, 2019 22:31
An extended exercise that covers the "Installation and Configuration" and "Cluster Administration" objectives of the Elastic exam.
# ** EXAM OBJECTIVES: INSTALLATION AND CONFIGURATION + CLUSTER ADMINISTRATION **
# Our goal is to deploy an Elasticsearch cluster named `training-cluster`, which satisfies all the requirements that follows
# Add three nodes to `training-cluster`, and name them `node1`, `node2`, and `node3`
# Configure each node to be eligible as a master node
# Configure each node to be a data node, but not an ingest node
# Configure each node to disable swapping on its host
# Configure the JVM on each node to use a minimum and maximum of 8 GB for the heap
# Bind `node1` to the network address "192.168.0.100"
# Configure the Zen Discovery module of `node2` and `node3` to use the address (and default transport port) of `node1`
# Configure `training-cluster` so as to avoid the split brain scenario
@glenacota
glenacota / 2es_v5-1kibana.yml
Created February 2, 2019 17:23
Docker-compose file to build up an ElasticSearch cluster with 2 nodes running elasticsearch version 5.6.14.
version: '3'
services:
esv5node1:
image: docker.elastic.co/elasticsearch/elasticsearch:5.6.14
container_name: esv5node1
environment:
- cluster.name=elastic-cluster
- node.name=esv5node1
- bootstrap.memory_lock=true
- xpack.security.enabled=false
@glenacota
glenacota / 1master-2datanodes_different_zones-1kibana.yml
Last active August 25, 2022 08:50
Docker-compose file to build up an ElasticSearch cluster with 1 master node, 2 datanodes supporting allocation awareness, and a Kibana instance. (v6.5.4)
version: '3'
services:
master:
image: docker.elastic.co/elasticsearch/elasticsearch:6.5.4
container_name: master
environment:
- cluster.name=elastic-cluster
- node.name=master
- node.master=true
- node.data=false
@glenacota
glenacota / 1master-2datanodes_hot_warm-1kibana.yml
Created February 2, 2019 17:16
Docker-compose file to build up an ElasticSearch cluster with 1 master node, 2 datanodes supporting a hot/warm architecture, and a Kibana instance. (v6.5.4)
version: '3'
services:
master:
image: docker.elastic.co/elasticsearch/elasticsearch:6.5.4
container_name: master
environment:
- cluster.name=elastic-cluster
- node.name=master
- node.master=true
- node.data=false
@glenacota
glenacota / 2clusters-with-1-node.yml
Last active February 2, 2019 17:01
Docker-compose file to build up two ElasticSearch clusters: the first with 1 node and a Kibana instance, the second only with 1 node. (v6.5.4)
version: '3'
services:
esnode-earth:
image: docker.elastic.co/elasticsearch/elasticsearch:6.5.4
container_name: esnode-earth
environment:
- cluster.name=elastic-cluster-earth
- node.name=esnode-earth
- bootstrap.memory_lock=true
- ES_JAVA_OPTS=-Xms512m -Xmx512m