Skip to content

Instantly share code, notes, and snippets.

View soldair's full-sized avatar

Ryan Day soldair

  • Google
  • San Jose CA
View GitHub Profile
@soldair
soldair / gist:6703702
Created September 25, 2013 18:15
npm global broken?
sudo npm install -g npm
npm http GET https://registry.npmjs.org/npm
npm ERR! cb() never called!
npm ERR! not ok code 0
@soldair
soldair / package.json
Last active December 19, 2015 23:19
multilevel ending does not error active streams
{
"name": "multilevel-end-test",
"description": "analyze end behavior",
"version": "0.0.0",
"repository": {
"url": "[email protected]:6034011"
},
"main": "index.js",
"scripts": {
"test": "tape test/*.js"
@soldair
soldair / db.js
Last active February 7, 2022 01:01
backup a running node levelup process
var levelup = require('level')
var zlib = require('zlib')
var fs = require('fs')
var through = require('through')
//hot backup
db = levelup(...)
var backingup = false
@soldair
soldair / gist:5951789
Created July 8, 2013 19:34
process large file with node
process.stdin.pipe(require('line-stream')()).pipe(require('through')(function(line){
// do analysis on a line
})).pipe(process.stdout)
@soldair
soldair / gist:5943759
Created July 7, 2013 15:10
me learns pythons
// write a log file!
file = open(logname,'a+');
file.write("log line!\n");
// stringify json
import json
json.dumps("hi");
@soldair
soldair / gist:5911432
Created July 2, 2013 17:42
what i do instead of using the "async" module
// do series or exactly N parallel its the same
var c = jobs.length
, job = function(){
doThing(function(){
next()
})
}, next = function(){
c--;
if(!c) done();
else job()
@soldair
soldair / leak_levelup.js
Created March 22, 2013 23:42
levelup writeStream doesn't pause when behind. this leads to huge memory leaks importing lots of data especially from stdin.
var level = require('levelup')
, db = level('./streamtest.db')
var dbws = db.createWriteStream()
, c = 0
, start = Date.now()
, intervalCount = 0
, bytes = 0
, paused = false
@soldair
soldair / vertica_copy.js
Last active December 14, 2015 23:49
how to run a copy command with data from any node stream into to a vertica database
var through = require('through');
// pure js vertica driver. im not into the coffee script but this is so useful even i domnt hold it against them
var vertica = require('vertica');
var stream = through();
//
// pipe some bar delimited rows of data to your stream.
// until you call stream.end() the data will not commit.
// because you can use logical streams rather than "real" streams..
// .. you can break a job up simply by ending the stream and starting another copy command ..
@soldair
soldair / epeerinvalid.log
Created March 14, 2013 06:26
epeerinvalid due to peer dependency added to flatiron-cli-config
make: Leaving directory `/usr/local/lib/node_modules/qrcode/node_modules/canvas/build'
/usr/local/bin/qrcode -> /usr/local/lib/node_modules/qrcode/bin/qrcode
npm WARN package.json [email protected] 'contributers' should probably be 'contributors'
npm ERR! peerinvalid The package flatiron does not satisfy its siblings' peerDependencies requirements!
npm ERR! peerinvalid Peer [email protected] wants flatiron@~0.1.9
npm ERR! peerinvalid Peer [email protected] wants flatiron@~0.1.9
npm ERR! System Linux 3.2.0-23-generic
npm ERR! command "/usr/local/bin/node" "/usr/local/bin/npm" "install" "-g" "qrcode"
npm ERR! cwd /usr/local/lib/node_modules
@soldair
soldair / pr
Created February 20, 2013 19:25
function pr () {
local repo=`git remote -v | grep -m 1 "(push)" | sed -e "s/.*github.com[:/]\(.*\)\.git.*/\1/"`
local branch=`git name-rev --name-only HEAD`
if [ "$1" != "" ]; then
local branch=$1
fi
echo "... creating pull request for branch \"$branch\" in \"$repo\""
xdg-open https://github.com/$repo/pull/new/$branch
}