Skip to content

Instantly share code, notes, and snippets.

View ryanramage's full-sized avatar

Ryan Ramage ryanramage

View GitHub Profile
@max-mapper
max-mapper / readme.md
Created September 28, 2011 02:01
SLEEP - syncable.org

Your API does REST, but can it SLEEP?

SLEEP (Syncable Lightweight Event Emitting Persistence) is an emerging standard for distributed data sync using HTTP and JSON. A generalized version of CouchDB's much lauded built-in replication, SLEEP extends the REST architecture to define a way in which databases can offer syncable JSON APIs that foster open data innovation by allowing developers to replicate entire databases over the net.


SLEEP comes from the Apache CouchDB project which is now widely known for it's multi-master streaming HTTP + JSON replication. This is possible in part because of the CouchDB _changes feed, which is a particular API that lets you see if there have been any changes made to the database since last time you synchronized. CouchDB can efficiently implement the _changes feed because of one subtle difference between it and most other databases: it stores a history of all changes that happen to the database, including deletes.

If you synchronize data from a remote source and then the

@dch
dch / rebar.md
Created October 24, 2012 13:13
build a stand-alone erlang rebar.cmd script with no dependencies

Pre-requisites

  • install MSYSGIT and add <gitdir>/bin/cmd to path
  • install Erlang/OTP minimum R15B02 (fixes escript swallowing stdout on exit, and adds line numbers to errors)
  • add Erlang/OTP to path

get fresh rebar

git clone git://github.com/basho/rebar.git

cd rebar

@janl
janl / couchdb-plugins.md
Created October 29, 2012 16:30
The CouchDB Plugin System

The CouchDB Plugin System

This document describes a possible future CouchDB Plugin system.

This is a work in progress, all the feedback is welcome. Please simplify and rewrite this as you see fit.

Once this is a solid description of what we want to build, we cna start building it.

High Level User-Oriented Walkthrough

@ryanramage
ryanramage / UMD.js
Last active December 17, 2015 19:19
Make sure you replace FILLME with values.
(function (root, factory) {
if (typeof exports === 'object') {
module.exports = factory( require('FILLME_A'), require('FILLME_B'), require('FILLME_C'));
} else if (typeof define === 'function' && define.amd) {
define(['FILLME_A', 'FILLME_B', 'FILLME_C'],factory);
} else {
root.FILLME_GLOBAL_NAME = factory(root.FILLME_A, root.FILLME_B, root.FILLME_C);
}
}(this, function (FILLME_A, FILLME_B, FILLME_C) {
@max-mapper
max-mapper / index.js
Last active May 9, 2021 02:20
fast loading of a large dataset into leveldb
// data comes from here http://stat-computing.org/dataexpo/2009/the-data.html
// download 1994.csv.bz2 and unpack by running: cat 1994.csv.bz2 | bzip2 -d > 1994.csv
// 1994.csv should be ~5.2 million lines and 500MB
// importing all rows into leveldb took ~50 seconds on my machine
// there are two main techniques at work here:
// 1: never create JS objects, leave the data as binary the entire time (binary-split does this)
// 2: group lines into 16 MB batches, to take advantage of leveldbs batch API (byte-stream does this)
var level = require('level')
#!/bin/sh
remove_dangling() {
echo "Removing dangling images ..."
docker rmi $(docker images -f dangling=true -q)
}
remove_stopped_containers() {
echo "Removing stopped containers ..."
docker rm $(docker ps -qa)
@juliangruber
juliangruber / extend.plist
Created May 30, 2016 20:30
Here is how to accept custom protocol links like dat://LINK in an @electronjs app, in osx. For this to work you need to properly package the app into a `.app`, and place the CFBundleURLTypes spec into it's Info.plist. Then, move the app into another directory using Finder (!). Here we're using electron-packager.
<key>CFBundleURLTypes</key>
<array>
<dict>
<key>CFBundleURLName</key>
<string>Dat Link</string>
<key>CFBundleURLSchemes</key>
<array>
<string>dat</string>
</array>
</dict>