Matt Digan mdigan

Internet Scale Services Checklist

A checklist for designing and developing internet scale services, inspired by James Hamilton's 2007 paper "On Desgining and Deploying Internet-Scale Services."

http://mvdirona.com/jrh/talksandpapers/jamesrh_lisa.pdf

Basic tenets

Does the design expect failures to happen regularly and handle them gracefully?
Have we kept things as simple as possible?

CLONE REPO

git clone [email protected]:basho-labs/marqs-ingest.git

SET ENV VARS

Use python script to query Marathon and set env vars (or maybe do it instead in each step so they're not dependent on this step?)
- KAFKA_BROKERS
- KAFKA_TOPIC
- MESOS_MASTER
RIAK_DIRECTOR_HOST

Node.js - npm Cheat Sheet

(Full description and list of commands at - https://npmjs.org/doc/index.html)

##List of less common (however useful) NPM commands

######Install a package and also update package.json with the installed version and package name.

	#!/usr/local/bin/python

	import os
	os.environ["PYSPARK_SUBMIT_ARGS"] = \
	"--packages com.basho.riak:spark-riak-connector_2.10:1.6.0 \
	pyspark-shell"

	import findspark, time
	findspark.init()

	####################################
	# BASIC REQUIREMENTS
	# http://graphite.wikidot.com/installation
	# http://geek.michaelgrace.org/2011/09/how-to-install-graphite-on-ubuntu/
	# Last tested & updated 10/13/2011
	####################################

	cd
	sudo apt-get update
	sudo apt-get upgrade

	# Load balancer configuration
	upstream exampleApp {
	# Directs to the process with least number of connections.
	least_conn;

	# One failed response will take a server out of circulation for 20 seconds.
	server 127.0.0.1:10080 fail_timeout=20s;
	server 127.0.0.1:10081 fail_timeout=20s;
	#server 127.0.0.1:10082 fail_timeout=20s;
	#server 127.0.0.1:10083 fail_timeout=20s;