Ian Blenke ianblenke

What

Unit files to deploy an ElasticSearch cluster on CoreOS via Fleet.

Service discovery & registration is done via etcd.

[email protected] provides a dumb discovery service by registering an elasticsearch host if it should be up. [email protected] registers the service only if it is running.

A service & timer unit for elasticsearch curator is provided which does some housekeeping.

Registrator and SkyDNS

We use progrium/registrator and yaronr/skydns (SkyDNS2) to publish information about Docker containers to DNS via A and SRV records. All nodes runs skydns and registrator, and first three nodes are inserted as NS-s into Route53 DNS for services.cluster-name.domain.io. Note, v4 registrator must be used until registrator/124 is resolved.

$ host -t srv mysql-1.services.deis.r53.acp.io deis-6-1.eu.r53.acp.io
Using domain server:
Name: deis-6-1.eu.r53.acp.io
Address: 54.171.239.227#53
Aliases:

First, read the CoreOS page on ECS:

https://coreos.com/docs/running-coreos/cloud-providers/ecs/

Install the aws commandline tool:

which aws || pip install awscli

Make sure you have your AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY defined in your shell environment.

Allow routing allocations:

curl -XPUT localhost:9200/_cluster/settings -d '{
                "transient" : {
                    "cluster.routing.allocation.enable" : "all"
                }
        }'

	#!/bin/bash -e
	tmpdir=/tmp/add-xfleet-conflicts
	mkdir -p $tmpdir
	fleetctl list-units -fields=unit -no-legend \| grep -v -e '@\\|deis' \| cut -d. -f1-2 \| sort \| uniq \| while read name ; do
	max=$(fleetctl list-machines -no-legend \| wc -l)
	count=$(fleetctl list-units -fields=unit -no-legend \| grep -e "^$name" \| wc -l)
	fleetctl list-units -fields=unit -no-legend \| grep -e "^$name" \| sort -n \| while read service; do
	tmpfile=$tmpdir/$service
	fleetctl cat $service >\| $tmpfile
	if ! grep X-Fleet $tmpfile > /dev/null; then

	#cloud-config
	coreos:
	units:
	- name: rethinkdb-create-fleet-units.service
	command: start
	content: \|
	[Unit]
	After=docker.service
	ConditionFileIsExecutable=/srv/rethinkdb-create-fleet-units.sh
	ConditionFileNotEmpty=/srv/[email protected]

	$ systemctl cat etcd
	# /usr/lib64/systemd/system/etcd.service
	[Unit]
	Description=etcd

	[Service]
	User=etcd
	PermissionsStartOnly=true
	Environment=ETCD_DATA_DIR=/var/lib/etcd
	Environment=ETCD_NAME=%m

	sudo btrfs filesys show

	sudo btrfs filesys balance start -dusage=1 /var/lib/docker