Skip to content

Instantly share code, notes, and snippets.

View eddies's full-sized avatar

Edwin Shin eddies

  • TRA
  • in the space-time continuum
View GitHub Profile
@eddies
eddies / graphite.md
Last active April 14, 2018 13:08 — forked from caged/graphite.md

Installation of Graphite on OS X Mountain Lion (30 May 2013)

Prerequisites

Install Python

brew install python --framework
pip install --upgrade distribute
@eddies
eddies / Install Graphite on Ubuntu 12.04.md
Last active December 17, 2015 22:38 — forked from bhang/install_graphite_statsd_ubuntu_precise.sh
Installation notes for Graphite on Ubuntu 12.04

Installation of Graphite on Ubuntu 12.04

Prerequisites

  • Apache 2

System level dependencies for Graphite

sudo apt-get install memcached python-dev python-pip sqlite3 libcairo2 \
 libcairo2-dev python-cairo pkg-config
@eddies
eddies / patch-github-oauth.txt
Created July 3, 2013 07:21
Updating an existing GitHub authorization via HTTP PATCH to public_repo only (e.g. prose.io, which defaults to the more permissive "repo").
# List your authorizations (substituting your own GitHub username for <user>)
curl -u <user> https://api.github.com/authorizations
# Update the scope of a specific authorization (substitute a specific id for <id>) to "public_repo"
curl -u <user> --request PATCH https://api.github.com/authorizations/<id> --header "Content-Type:application/json" --data '{"scopes":["public_repo"]}'
# Reference: http://developer.github.com/v3/oauth/#oauth-authorizations-api
@eddies
eddies / snsToSlack.js
Last active October 7, 2015 04:51 — forked from terranware/snsToSlack.js
AWS Lambda function to Slack Channel hookup
var https = require('https');
var util = require('util');
exports.handler = function(event, context) {
console.log(JSON.stringify(event, null, 2));
console.log('From SNS:', event.Records[0].Sns.Message);
var postData = {
"channel": "#aws-sns",
"username": "AWS SNS via Lamda :: DevQa Cloud",
@eddies
eddies / test_spark.py
Created November 4, 2015 07:32
py.test sanity test of spark
import pytest
def test_spark_sanity_check(sc, tmpdir):
csv = tmpdir.mkdir(__name__).join('one_liner.csv')
csv.write("foo,bar,baz")
data = sc.textFile(str(csv))
line_count = data.count()
assert line_count == 1
import org.apache.commons.io.IOUtils
import java.net.URL
import java.nio.charset.Charset
// Zeppelin creates and injects sc (SparkContext) and sqlContext (HiveContext or SqlContext)
// So you don't need create them manually
// load map data
val myMapText = sc.parallelize(
IOUtils.toString(
@eddies
eddies / GIF-Screencast-OSX.md
Created February 29, 2016 07:07 — forked from dergachev/GIF-Screencast-OSX.md
OS X Screencast to animated GIF

OS X Screencast to animated GIF

This gist shows how to create a GIF screencast using only free OS X tools: QuickTime, ffmpeg, and gifsicle.

Screencapture GIF

Instructions

To capture the video (filesize: 19MB), using the free "QuickTime Player" application:

@eddies
eddies / reactive_map.js
Created March 31, 2016 02:26 — forked from granturing/reactive_map.js
Sample reactive Leaflet code for Zeppelin
<!-- place this in an %angular paragraph -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/0.7.5/leaflet.css" />
<div id="map" style="height: 800px; width: 100%"></div>
<script type="text/javascript">
function initMap() {
var map = L.map('map').setView([30.00, -30.00], 3);
L.tileLayer('http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
@eddies
eddies / gtb
Created April 5, 2016 09:58 — forked from dschuetz/gtb
Hackish script to compute Google Authenticator tokencodes and launch a Tunnelblick VPN with the appropriate computed password+tokencode
#!/usr/bin/python
import hmac, struct, time, base64, hashlib # for totp generation
import re, sys, subprocess # for general stuff
from getopt import getopt, GetoptError # pretending to be easy-to-use
#
# gtb - Google(auth) + Tunnelblick
#
@eddies
eddies / setup-notes.md
Created July 29, 2016 08:00
Spark 2.0.0 and Hadoop 2.7 with s3a setup

Standalone Spark 2.0.0 with s3

###Tested with:

  • Spark 2.0.0 pre-built for Hadoop 2.7
  • Mac OS X 10.11
  • Python 3.5.2

Goal

Use s3 within pyspark with minimal hassle.