jwerak

Snowplow

Quick PoC of sending tracker data to custom Snowplow Collector.

python simple_snowflow_collector.py

Its convenient to extract below binaries to single directory in your $HOME, e.g. ~/bin/ and add this line to your ~/.bashrc:

export PATH=~/bin:$PATH

	Oct 06 17:11:49 ip-10-0-1-7.us-west-2.compute.internal bash[1083]: I1006 17:11:49.280273 11 master.cpp:1315] Received registration request from [email protected]:35840
	Oct 06 17:11:49 ip-10-0-1-7.us-west-2.compute.internal bash[1083]: I1006 17:11:49.280331 11 master.cpp:1361] Framework 20141006-162033-117506058-5050-7-0001 ([email protected]:35840) already registered, resending acknowledgement
	Oct 06 17:11:49 ip-10-0-1-7.us-west-2.compute.internal bash[1083]: I1006 17:11:49.700490 12 master.cpp:1315] Received registration request from [email protected]:51795
	Oct 06 17:11:49 ip-10-0-1-7.us-west-2.compute.internal bash[1083]: I1006 17:11:49.700567 12 master.cpp:1361] Framework 20141006-162033-117506058-5050-7-0004 ([email protected]:51795) already registered, resending acknowledgement
	Oct 06 17:11:50 ip-10-0-1-7.us-west-2.compute.internal bash[1083]:

	{
	"schedule": "R\/2014-10-05T18:50:00Z\/PT2M",
	"name": "chronosdockejob",
	"container": {
	"type": "DOCKER",
	"image": "quay.io/appuri/marathon-test"
	},
	"cpus": "0.1",
	"mem": "20",
	"uris": [],

	Oct 07 10:43:42 ip-10-0-1-80.us-west-2.compute.internal bash[970]: I1007 10:43:42.225401 14 hierarchical_allocator_process.hpp:563] Recovered cpus():1; mem():498; disk():2879; ports():[31000-32000] (total allocatable: cpus():1; mem():498; disk():2879; ports():[31000-32000]) on slave 20141007-102413-1342242826-5050-6-0 from framework 20141007-102413-1342242826-5050-6-0000
	Oct 07 10:43:43 ip-10-0-1-80.us-west-2.compute.internal bash[970]: I1007 10:43:43.143812 8 master.cpp:1315] Received registration request from [email protected]:57089
	Oct 07 10:43:43 ip-10-0-1-80.us-west-2.compute.internal bash[970]: I1007 10:43:43.143895 8 master.cpp:1361] Framework 20141007-102413-1342242826-5050-6-0001 ([email protected]:57089) already registered, resending acknowledgement
	Oct 07 10:43:44 ip-10-0-1-80.us-west-2.compute.internal bash[970]: I1007 10:43:44.144242 9 master.cpp:1315] Received registration request from scheduler-

	I've found this inspiring, but I would use this instead to add value to key:


	def dot_set(s,v):
	return reduce(lambda x,y: {y: x}, reversed(s.split('.')), v)

	print dot_set('1.a.b.c', 'test')

	do_set will return
	{'1': {'a': {'b': {'c': 'test'}}}}

	#!/bin/bash
	curl https://gist.githubusercontent.com/veverjak/b8607526f071f296bd8c/raw/52dd97718f1f7e46c5f7d04888ca02c74bea7225/get_file_content.py -o /app.py

	pip install flask

	while true; do
	etcdctl --peers `cat /etc/ip`:4001 get /test > /test
	if [[ -n $PID ]];
	then kill -9 $PID;
	fi