goliatone

Configuring AWS ECS to have access to AWS EFS

If you would like to persist data from your ECS containers, i.e. hosting databases like MySQL or MongoDB with Docker, you need to ensure that you can mount the data directory of the database in the container to volume that's not going to dissappear when your container or worse yet, the EC2 instance that hosts your containers, is restarted or scaled up or down for any reason.

Don't know how to create your own AWS ECS Cluster? Go here!

New Cluster

Sadly the EC2 provisioning process doesn't allow you to configure EFS during the initial config. After your create your cluster, follow the guide below.

New Task Definition for Web App

If you're using an Alpine-based Node server like duluca/minimal-node-web-server follow this guide:

Disclaimer

I'm still very new to Kafka, eventsourcing, stream processing, etc. I'm in the middle of building my first production system with this stuff and am writing this at the request of a few folks on Twitter. So if you do have experience, please do me and anyone else reading this a favor by pointing out things I get wrong :)

Inspirations

The Log — http://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying
Turning the database inside out — http://www.confluent.io/blog/2015/03/04/turning-the-database-inside-out-with-apache-samza/
Why local state is a fundamental primitive in stream processing — http://radar.oreilly.com/2014/07/why-local-state-is-a-fundamental-primitive-in-stream-processing.html
Samza
Various functional systems been exposed to over past year or so, React, Flux, RX, etc.

Creating a redis Module in 15 lines of code!

A quick guide to write a very very simple "ECHO" style module to redis and load it. It's not really useful of course, but the idea is to illustrate how little boilerplate it takes.

Step 1: open your favorite editor and write/paste the following code in a file called module.c

#include "redismodule.h"
/* ECHO <string> - Echo back a string sent from the client */
int EchoCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {

2 This bar chart uses Chart.js to plot 2008 data land composition in Georgian Counties.

It uses the CartoDB SQL API to pull these data, used to make this map into a chart.

Source: Georgia Polygons + Georgian Census Demographics

Data Store: CartoDB

Color: ColorBrewer-Diverging

	var mongoose = require('mongoose');
	var Schema = mongoose.Schema;

	//Database connection
	var uristring = 'mongodb://localhost/test';
	var mongoOptions = { };

	mongoose.connect(uristring, mongoOptions, function (err, res) {
	if (err) {
	console.log('Error when connecting to: ' + uristring + '. ' + err);

	# You don't need Fog in Ruby or some other library to upload to S3 -- shell works perfectly fine
	# This is how I upload my new Sol Trader builds (http://soltrader.net)
	# Based on a modified script from here: http://tmont.com/blargh/2014/1/uploading-to-s3-in-bash

	S3KEY="my aws key"
	S3SECRET="my aws secret" # pass these in

	function putS3
	{
	path=$1

	/*
	* Dump block 0 of a MIFARE RFID card using a RFID-RC522 reader
	* Uses MFRC522 - Library to use ARDUINO RFID MODULE KIT 13.56 MHZ WITH TAGS SPI W AND R BY COOQROBOT.
	-----------------------------------------------------------------------------
	* Pin layout should be as follows:
	* Signal Pin Pin Pin
	* Arduino Uno Arduino Mega MFRC522 board
	* ------------------------------------------------------------
	* Reset 9 5 RST
	* SPI SS 10 53 SDA

	#!/bin/bash

	# Check for -debug flag
	if [[ $* == -debug ]]; then
	set -x
	fi

	export LGR_NO_TIMESTAMP=true

	VERSION_FILE="./.version"

	SHOBJ_CFLAGS ?= -fno-common -g -ggdb
	SHOBJ_LDFLAGS ?= -shared -Bsymbolic
	CFLAGS = -Wall -g -fPIC -lc -lm -Og -std=gnu99
	CC=gcc

	all: example.so

	example.so: example.o
	$(LD) -o $@ example.o $(SHOBJ_LDFLAGS) $(LIBS) -lc

	*.xo
	*.so
	*.o