Skip to content

Instantly share code, notes, and snippets.

View jewer's full-sized avatar

Joshua Ewer jewer

View GitHub Profile
@jewer
jewer / BloomFilterCoderFactory.java
Last active November 12, 2016 19:39
Building a bloom filter in Google Dataflow
import com.google.cloud.dataflow.sdk.coders.Coder;
import com.google.cloud.dataflow.sdk.coders.CoderFactory;
import com.google.cloud.dataflow.sdk.coders.SerializableCoder;
import com.google.common.hash.BloomFilter;
import java.util.Collections;
import java.util.List;
public class BloomFilterCoderFactory implements CoderFactory {
@Override
import subprocess
number_legit_AP = 1
output = subprocess.check_output('gobbledegook', shell=True)
if output != number_legit_AP:
send_email()
@jewer
jewer / regression.scala
Last active March 25, 2016 16:50
logistic regression in spark
import org.apache.spark.ml.feature._
import org.apache.spark.ml.classification._
import org.apache.spark.mllib.regression.LabeledPoint
import org.apache.spark.mllib.linalg.Vector
import org.apache.spark.sql._
import org.apache.spark.mllib.classification.{LogisticRegressionWithLBFGS, LogisticRegressionModel}
import org.apache.spark.ml.feature.VectorAssembler
//helper function
def load(path: String, sqlContext: SQLContext): DataFrame = {
@jewer
jewer / regression.nbpy
Last active March 22, 2016 15:39
sample logistic regression in pyspark
{
"cells": [
{
"cell_type": "code",
"execution_count": 65,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
@jewer
jewer / gulpfile.js
Created January 15, 2016 20:41
simplest possible useful gulp file
var gulp = require('gulp'),
util = require('gulp-util'),
jshint = require('gulp-jshint'),
mocha = require('gulp-mocha');
var srcFiles = './src/**/*.js';
var testFiles = './test/**/*.js';
gulp.task('test', function() {
return gulp.src(testFiles, {read: false})
@jewer
jewer / Dockerfile
Created January 6, 2016 19:37
Example of using nginx to directly proxy to redis
FROM debian:latest
# Update the repository and install Redis Server
RUN apt-get update
RUN apt-get install -y redis-server libreadline-dev libncurses5-dev libpcre3-dev libssl-dev perl make build-essential
RUN apt-get install -y lua-nginx-redis
ADD nginx.conf /etc/nginx/nginx.conf
RUN apt-get install -y curl screen vim
@jewer
jewer / gist:7916530
Last active December 31, 2015 01:49
Scala to (naively) parse base64 encoded querystring values (which include '=' characters)
def expandQueryString(s: String) : Array[(String, String)] =
s.split("\\?").last.split("&").map(_.split("=", -1))
.map(x =>{
if(x.length > 1) Some(x.head, x.tail.mkString("="))
else None
}).flatten
.filterNot(_._2 == "")
@jewer
jewer / gist:7763016
Last active December 30, 2015 02:29
hack to get querystring values
def parse(uri: String) : Map[String,Seq[String]] = {
if (uri == null || uri.isEmpty) {
return Map()
}
def parsePairs(pair: String) : (String, String) = {
val bits = pair.decodeUrl.split("=")
(bits(0), bits(1))
}
@jewer
jewer / gist:5782020
Last active December 18, 2015 12:19
command-line python for uploading s3 packages
import boto.s3
from boto.s3.key import Key
import sys
import math
def percent_cb(complete, total):
sys.stdout.write(str(math.trunc(complete/max(total, 1)) * 100) + '%..')
if total == complete:
print 'done!'
@jewer
jewer / Simple Exception handling
Created April 17, 2012 22:54
What's wrong with this code?
public class TaxCalculationException : ApplicationException
{
private string _zipCode;
public TaxCalculationException(string zipCode, Exception ex)
: base("An error occurred while processing tax.", ex)
{
_zipCode = zipCode;
}
}