Gregor Sturm grst

We set up a distributed network, create a lot of pandas dataframes on the workers, then create a single logical dask.dataframe out of their futures. We then do distributed dataframe filtering operations.

$ dscheduler
$ dworker 127.0.0.1:8786
$ dworker 127.0.0.1:8786
$ dworker 127.0.0.1:8786

Archlinux setup guide

This guide is based on the official Archlinux [Installation Guide][1], [Beginners' Guide][2] and my personal experience.

Partition layout

SDD partitions	fs	size	mountpoint
/dev/nvme0n1p1	FAT	512M	-
/dev/nvme0n1p2	LUKS	MAX	-

Initialize

gitflow	git
`git flow init`	`git init`
	`git commit --allow-empty -m "Initial commit"`
	`git checkout -b develop master`

Connect to the remote repository

Virtual Hosts on nginx (CSC309)

When hosting our web applications, we often have one public IP address (i.e., an IP address visible to the outside world) using which we want to host multiple web apps. For example, one may wants to host three different web apps respectively for example1.com, example2.com, and example1.com/images on the same machine using a single IP address.

How can we do that? Well, the good news is Internet browsers

The fastest remote directory rsync over ssh archival I can muster (40MB/s over 1gb NICs)

This creates an archive that does the following:

rsync (Everyone seems to like -z, but it is much slower for me)

a: archive mode - rescursive, preserves owner, preserves permissions, preserves modification times, preserves group, copies symlinks as symlinks, preserves device files.
H: preserves hard-links
A: preserves ACLs

	#' Convert counts to transcripts per million (TPM).
	#'
	#' Convert a numeric matrix of features (rows) and conditions (columns) with
	#' raw feature counts to transcripts per million.
	#'
	#' Lior Pachter. Models for transcript quantification from RNA-Seq.
	#' arXiv:1104.3889v2
	#'
	#' Wagner, et al. Measurement of mRNA abundance using RNA-seq data:
	#' RPKM measure is inconsistent among samples. Theory Biosci. 24 July 2012.

	# RPKM versus TPM
	#
	# RPKM and TPM are both normalized for library size and gene length.
	#
	# RPKM is not comparable across different samples.
	#
	# For more details, see: http://blog.nextgenetics.net/?e=51

	rpkm <- function(counts, lengths) {
	rate <- counts / lengths

	#! perl

	# Store this file as /usr/lib/urxvt/perl/xkr-clipboard
	#
	# To enable, add to ~/.Xresources:
	#
	# URxvt.perl-ext-common: default,xkr-clipboard
	# URxvt.iso14755: false
	# URxvt.keysym.Shift-Control-C: perl:clipboard:copy
	# URxvt.keysym.Control-Insert: perl:clipboard:copy

	#!/bin/bash +x

	# Source: http://www.gabsoftware.com/tips/automatically-reconnect-to-your-vpn-on-linux/

	# Description:
	# Make the script executable "chmod +x /path/to/the/script.sh
	# Put the script in .profile or .bashrc so it can be run on user login:
	# Example: echo "/path/to/the/script.sh start &" >> .bashrc
	# The script can be bound to shortcut keys with these commands:
	# /path/to/the/script.sh start # starts and monitors VPN connection

	" Reference: http://stackoverflow.com/questions/7159607 "

	import os
	import sys


	def listdir(path):
	"""
	recursively walk directory to specified depth
	:param path: (str) path to list files from