Skip to content

Instantly share code, notes, and snippets.

View retrography's full-sized avatar

Mahmood S. Zargar retrography

  • VU Amsterdam
  • Amsterdam, Netherlands
  • X @mszargar
View GitHub Profile
@retrography
retrography / rh_install.R
Created August 5, 2015 17:33
Install RHadoop
install.packages(c('RCurl','rJava','RJSONIO', 'itertools', 'digest','Rcpp','httr','functional','devtools','plyr','reshape2','R.methodsS3','hydroPSO','pryr',"bitops","functional","reshape2","stringr","caTools"))
Sys.setenv(HADOOP_CMD="/usr/bin/hadoop")
Sys.setenv(HADOOP_STREAMING="/usr/hdp/2.2.0.0-2041/hadoop-mapreduce/hadoop-streaming.jar")
library("devtools")
install_github("RevolutionAnalytics/rmr2/pkg")
install_github("RevolutionAnalytics/rhdfs/pkg")
install_github("RevolutionAnalytics/plyrmr/pkg")
@retrography
retrography / dockering.sh
Last active August 29, 2015 14:27
Docker Configuration and Use on OS X
# see http://docs.docker.com/installation/mac/
# see https://docs.docker.com/reference/commandline/cli/
brew cask install dockertoolbox
docker-machine create --driver virtualbox default
# To start the machine
docker-machine start default
# To stop the machine
@retrography
retrography / mongo_docker.sh
Last active August 29, 2015 14:27
Importing Yelp's Academic Database into MongoDB
# brew cask install dockertoolbox
# docker-machine create --driver virtualbox default
# To list vm environment variables for the machine and export them as global environment variables
docker-machine env default
eval "$(docker-machine env default)"
# Create a data volume
docker create -v /data --name mongodata mongo
@retrography
retrography / json2csv.R
Last active February 26, 2021 23:09
JSON to CSV convertor. Uses `jsonlite` R package, flattens all the hierarchical structure and converts all remaining lists/arrays into strings.
#!/usr/bin/Rscript
if (!require(jsonlite, quietly = TRUE, warn.conflicts = FALSE)) {
stop("This program requires the `jsonlite` package. Please install it by running `install.packages('jsonlite')` in R and then try again.")
}
args <- commandArgs(trailingOnly = TRUE)
if (length(args) == 0) {
input <- readLines(file("stdin"))
@retrography
retrography / R-benchmark-25.R
Created August 24, 2015 01:36
Test R's speed to ensure OpenBlas/MKL/ATLAS integration
# R Benchmark 2.5 (06/2008) [Simon Urbanek]
# version 2.5: scaled to get roughly 1s per test, R 2.7.0 @ 2.6GHz Mac Pro
# R Benchmark 2.4 (06/2008) [Simon Urbanek]
# version 2.4 adapted to more recent Matrix package
# R Benchmark 2.3 (21 April 2004)
# Warning: changes are not carefully checked yet!
# version 2.3 adapted to R 1.9.0
# Many thanks to Douglas Bates (bates@stat.wisc.edu) for improvements!
# version 2.2 adapted to R 1.8.0
# version 2.1 adapted to R 1.7.0
# Some necessary packages!
install.packages(c("devtools","doParallel","doMC","statnet","igraph","dplyr","reshape","reshape2","plyr","tidyr","lubridate","jsonlite","magrittr","stringi","stringr","data.table","sqldf"))
# topicmodels needs gsl
install.packages(c("RTextTools","topicmodels","glmnet","Zelig"))
library(devtools)
install_github("retrography/OrientR")
@retrography
retrography / shell-sshpass-install.md
Last active November 21, 2015 21:05 — forked from arunoda/gist:7790979
Installing SSHPass

Installing SSHPASS

SSHPass is a tiny utility, which allows you to provide the ssh password without using the prompt. This will very helpful for scripting. SSHPass is not good to use in multi-user environment. If you use SSHPass on your development machine, it don't do anything evil.

Installing on Ubuntu

apt-get install sshpass

Installing on OS X

@retrography
retrography / mux.sh
Created September 19, 2015 20:09
Mux two files (audio and video) into a new container with ffmpeg
ffmpeg -i video.mp4 -i audio.m4a -map 0:0 -map 1:0 -c copy out.mp4
@retrography
retrography / git-diff-add.sh
Created September 21, 2015 21:37
The additions between two commits in git
git diff 4ce3aad2428b4d6e2e0a6eac743cfda190cd88eb 2279001c8ff1acc156637d8d7eb4e9c81b198ea8 | grep '^+' | gsed -r -e 's/^\+\+\+ b(.*)$/\n\1\n/' -e 's/^\+//'
@retrography
retrography / respawn.sh
Created September 23, 2015 05:13
One-liner that respawns a crash-prone process every time it quits
while true; do ./scraper.rb; done