Skip to content

Instantly share code, notes, and snippets.

@ankurcha
ankurcha / spark-config.json
Last active December 17, 2015 18:49
Spark configuration options
{
"home": null,
"local_dir": null,
"buffer_size": 65536,
"kryo": {
"buffer_size_mb": 10,
"registrator": null
},
"parallelism": null,
"test": {
function Average(left, right) {
var result = new Array();
if (left instanceof Array && right instanceof Array) {
for (var i=0; i<left.length || i<right.length; i++) {
result[i] = Average((i<left.length ? left[i] : 0.0), (i<right.length ? right[i] : 0.0));
}
} else if (left instanceof Array) {
for (var i=0; i<left.length; i++) {
result[i] = Average(left[i], (i==0 ? right : 0.0));
@ankurcha
ankurcha / Readme.md
Created July 19, 2013 21:23
Implementation of the Google Youtube Data API interface * All Channels have a one to one mapping with the entry in the credentialStore. * The refreshing process is managed by the GoogleAuthorizationFlow api client.
  • All Channels have a one to one mapping with the entry in the credentialStore.
  • The refreshing process is managed by the GoogleAuthorizationFlow api client.
#!/usr/bin/env ruby
def check_require(name)
begin
gem name
rescue Gem::LoadError
# not installed
system("gem install #{name}")
Gem.clear_paths
end
require name
@ankurcha
ankurcha / groovyw.sh
Created August 13, 2013 20:58
Groovy wrapper
#!/bin/sh
GROOVY_VERSION=${$GROOVY_VERSION:-"2.1.6"}
if [ -z "$@" ]; then
echo "No argument specified"
exit 1
fi
if [ ! -f "$@" ]; then
@ankurcha
ankurcha / backupStatus.json
Last active October 23, 2015 06:11
Script to backup tokumx database to a directory and/or S3.
{
"percent" : 6.297823041677475,
"bytesDone" : NumberLong("22117130240"),
"files" : {
"done" : 8,
"total" : 166
},
"current" : {
"source" : "/var/lib/mongodb_historical11/local_oplog_rs_p15_id__ddfdbfc0_1_19.tokumx",
"dest" : "/backup11/local_oplog_rs_p15_id__ddfdbfc0_1_19.tokumx",
@ankurcha
ankurcha / remove_db.js
Created April 25, 2014 19:19
This is a set of scripts we use to reliably drop sharded databases on a MongoDB cluster. The reason for doing this is because mongos is very unreliable when it comes to cleanly and reliably dropping stuff.
function removeDB(n) {
flushConfigs();
var colls = findChunkedCollections(n);
colls.forEach(function(coll) {
if(!db.getSisterDB(n).getCollection(coll).exists()) {
print("collection did not exist for chunk: " + n + "." + coll)
getPrimaries().forEach(function(p){
p.getDB(n).createCollection(coll);
});
@ankurcha
ankurcha / api-agent.conf
Created June 19, 2014 09:30
logstash agent.conf
input {
file {
type => "analytics-api"
path => "/var/log/analytics-api/analytics-api.log"
}
file {
type => "mongos"
path => ["/var/log/mongodb/mongos_rolling.log", "/var/log/mongodb/mongos_historical.log", "/var/log/mongodb/mongos_collector.log"]
}
file {
/*
var context = cubism.context(),
cube = context.cube("https://data.brightcove.com/v1/accounts/8523")
cube.metric("video_view")
*/
cubism_contextPrototype.analytics_api = function(url) {
if (!arguments.length) url = "";
var source = {},
context = this;
@ankurcha
ankurcha / knife_dl.sh
Created August 28, 2014 22:00
Function to download and extract cookbook
function knife_dl() {
local packages="$@"
cd ~/chef/cookbooks
for c in ${packages}; do
knife cookbook site download ${c};
tar zxf ${c}*.tar.gz;
rm ${c}-*.tar.gz;
done
}