Skip to content

Instantly share code, notes, and snippets.

View pingles's full-sized avatar

Paul Ingles pingles

View GitHub Profile
@pingles
pingles / emacs.rb
Created July 27, 2011 12:09
Homebrew Emacs for OSX Lion with native full-screen
require 'formula'
class Emacs < Formula
url 'http://ftp.gnu.org/pub/gnu/emacs/emacs-23.3.tar.bz2'
md5 'a673c163b4714362b94ff6096e4d784a'
homepage 'http://www.gnu.org/software/emacs/'
if ARGV.include? "--use-git-head"
head 'git://repo.or.cz/emacs.git'
else
@pingles
pingles / hdfs_copy_and_merge.clj
Created August 1, 2011 08:05
Copying and merging globs in HDFS
(defn copy-and-merge [src-path wtr]
(let [fs (.getFileSystem (Path. src-path) config)]
(let [paths (map #(.getPath %) (.globStatus fs (Path. src-path)))]
(doseq [p paths] (with-open [is (.open fs p)] (stream/copy is wtr))))))
@pingles
pingles / exp_smooth.clj
Created August 21, 2011 09:29
Exponential smooth for incanter
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; SMOOTHING
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn exponential-smooth
"
Smooths a sequence with an exponentially decreasing weight.
Arguments:
x -- a sequence of numbers
(def app
(-> #'main-routes
(wrap-request-logging)
(wrap-exception-logging)
(wrap-nested-params)))
(defn repeatedly-consume
"Repeatedly consume from url and pass message to handler"
[handler url]
(mq/with-context context 2
(with-open [s1 (-> context
(mq/socket mq/sub)
(mq/subscribe)
(mq/connect url))]
(while true
(handler (mq/recv s1))))))
(defn consumer-seq
"Using 0mq context repeatedly consume from url"
[context url]
(let [socket (-> context
(zmq/socket zmq/sub)
(zmq/subscribe "weblog")
(zmq/connect url))]
(repeatedly #(zmq/receive socket))))
(let [cs (consumer-seq ctx url)]
(defn make-publisher
"Returns a 0mq fn that can be called to publish messages."
([context url]
(make-publisher context url identity))
([context url f]
(let [socket (-> context
(socket pub)
(bind url))]
(fn [msg]
(publish socket (f msg))))))
(defevent WebRequest [http_referer :string
request :string
status :int])
(def success-rpm
"SELECT COUNT(1) as success_count
FROM uswitch_requests.win:time(60 seconds)
WHERE status >= 200 AND status < 400
OUTPUT SNAPSHOT EVERY 1 SECONDS")
(new-event WebRequest :request "/blah" :http_referer "-" :status 200)
var HDFS = require('node-hdfs');
var client = new HDFS({host:"default", port:0});
var hdfs_path = "/tmp/helloworld.txt"
var buffer = new Buffer("Hello, World!\n")
client.write(hdfs_path, function(writer) {
writer.once("open", function(err, handle) {
writer.write(buffer);
writer.end();