Jeremiah Malina jjmalina

Description

This simple script will take a picture of a whiteboard and use parts of the ImageMagick library with sane defaults to clean it up tremendously.

The script is here:

#!/bin/bash
convert "$1" -morphology Convolve DoG:15,100,0 -negate -normalize -blur 0x1 -channel RBG -level 60%,91%,0.1 "$2"

Results

General Background and Overview

Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
Models and Issues in Data Stream Systems
Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
[Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep=rep1&t

Flask + uWSGI + nginx Primer

I've been having trouble with serving a Flask app via uWSGI and nginx, so I thought I'd put together some of the basics to help out others.

How this shit works

Flask is managed by uWSGI.
uWSGI talks to nginx.

Sublime Text 2 – Useful Shortcuts (Mac OS X)

General

⌘T	go to file
⌘⌃P	go to project
⌘KB	toggle side bar
⌘⇧P	command prompt
⌃ `	python console
⌘⇧N	new window (useful for new project)

Locate the section for your github remote in the .git/config file. It looks like this:

[remote "origin"]
	fetch = +refs/heads/*:refs/remotes/origin/*
	url = [email protected]:joyent/node.git

Now add the line fetch = +refs/pull/*/head:refs/remotes/origin/pr/* to this section. Obviously, change the github url to match your project's URL. It ends up looking like this:

	def urlses(cl: ClassLoader): Array[java.net.URL] = cl match {
	case null => Array()
	case u: java.net.URLClassLoader => u.getURLs() ++ urlses(cl.getParent)
	case _ => urlses(cl.getParent)
	}

	val urls = urlses(getClass.getClassLoader)
	println(urls.filterNot(_.toString.contains("ivy")).mkString("\n")

	// data comes from here http://stat-computing.org/dataexpo/2009/the-data.html
	// download 1994.csv.bz2 and unpack by running: cat 1994.csv.bz2 \| bzip2 -d > 1994.csv
	// 1994.csv should be ~5.2 million lines and 500MB
	// importing all rows into leveldb took ~50 seconds on my machine

	// there are two main techniques at work here:
	// 1: never create JS objects, leave the data as binary the entire time (binary-split does this)
	// 2: group lines into 16 MB batches, to take advantage of leveldbs batch API (byte-stream does this)

	var level = require('level')

	gifify() {
	if [[ -n "$1" ]]; then
	if [[ $2 == '--good' ]]; then
	ffmpeg -i $1 -r 10 -vcodec png out-static-%05d.png
	time convert -verbose +dither -layers Optimize -resize 600x600\> out-static*.png GIF:- \| gifsicle --colors 128 --delay=5 --loop --optimize=3 --multifile - > $1.gif
	rm out-static*.png
	else
	ffmpeg -i $1 -s 600x400 -pix_fmt rgb24 -r 10 -f gif - \| gifsicle --optimize=3 --delay=3 > $1.gif
	fi
	else

	var serialport = require('node-serialport')

	var sp = new serialport.SerialPort("/dev/ttyO3", {
	parser: serialport.parsers.raw,
	baud: 9600
	})

	sp.on('data', function(chunk) {
	console.log(chunk.toString('hex'), chunk.toString(), chunk)
	})

	if not sinatra.incoming ( ngx.req.get_method() , ngx.var.uri ) then
	return ngx.exit ( 404 )
	end