byteshiva

Requirements

• no upfront installation/agents on remote/slave machines - ssh should be enough
• application components should use third-party software, e.g. HDFS, Spark's cluster, deployed separately
• configuration templating
• environment requires/asserts, i.e. we need a JVM in a given version before doing deployment
• deployment process run from Jenkins

Solution

byteshiva

module CustomParser
  module Util

    def my_echo_method(param1, param2)
      puts "Hello #{param1}"
      puts "Hello #{param2}"


    end

byteshiva

#!/bin/bash
### Set default parameters
# somewhat follows the conventions from: http://michal.karzynski.pl/blog/2013/10/29/serving-multiple-django-applications-with-nginx-gunicorn-supervisor/
# designed to run on a debian symbiosis box if your interested why its /srv
action=$1
domain=$2
appname=$3
owner=$(who am i | awk '{print $1}')
sitesEnable='/etc/nginx/sites-enabled/'
sitesAvailable='/etc/nginx/sites-available/'

byteshiva

{
    "acl_datacenter": "test-1",
    "acl_default_policy": "deny",
    "acl_down_policy": "deny",
    "acl_master_token": "1234",
    "acl_ttl": "30s",
    "bootstrap_expect": 3,
    "check_update_interval": "5m",
    "datacenter": "test-1",
    "data_dir": "/etc/consul.d/data",

byteshiva

{
    "services": [
    {
      "id": "red0",
      "name": "redis",
      "tags": [
        "master"
      ],
      "address": "127.0.0.1",
      "port": 6000,

byteshiva

- What do Etcd, Consul, and Zookeeper do?
  - Service Registration:
    - Host, port number, and sometimes authentication credentials, protocols, versions
      numbers, and/or environment details.
  - Service Discovery:
    - Ability for client application to query the central registry to learn of service location.
  - Consistent and durable general-purpose K/V store across distributed system.
    - Some solutions support this better than others.
    - Based on Paxos or some derivative (i.e. Raft) algorithm to quickly converge to a consistent state.
    - Centralized locking can be based on this K/V store.

byteshiva

set -x

mkdir -p /tmp/consul
cd /tmp/consul

# Consul Configuration File
tee consul.json <<CONSULCONFIG
{
    "bootstrap_expect": 3,
    "client_addr": "0.0.0.0",

byteshiva

{
  "AWSTemplateFormatVersion": "2010-09-09",

  "Description": "AWS CloudFormation template to create the needed AWS resources for Labs 3 & 4 of the ECS Microservices Bootcamp for re:Invent 2015",

  "Mappings": {

    "AWSRegionToAMI": {
      "eu-west-1": {
        "AMI": "ami-a10897d6"

byteshiva

{
  "bootstrap": false,
  "bootstrap_expect": 1,
  "datacenter": "east-aws",
  "data_dir": "/opt/consul/data",
  "log_level": "INFO",
  "node_name": "60f81dc58442",
  "server": true,
  "telemetry": {
    "dogstatsd_addr": "127.0.0.1:8125",

byteshiva

# Quick Measure
grep 'cpu ' /proc/stat | awk '{usage=($2+$4)*100/($2+$4+$5)} END {print usage "%"}'
# 1.37339%

# Quicker
top -bn1 | grep "Cpu(s)" | \
           sed "s/.*, *\([0-9.]*\)%* id.*/\1/" | \
           awk '{print 100 - $1"%"}'
# 1.4%