max6cn

max6cn

from random import randint
from operator import add

times = 10
# Step 1:  Rooling dice and record result into array
genseq = map (lambda _: randint(1,6) , [i for i in range(times)] )

#Steap 2: Count occurances

# bmap : map a number into bitmap array, eg: 2 -> [0,1,0,0,0,0](one hot encoding)

max6cn

Viewing all git diffs with vimdiff

git config --global diff.tool vimdiff
git config --global difftool.prompt false
git config --global alias.d difftool

max6cn

body {
  background-color: #fdf6e3;
  color: black;
  font-family: 'Source Serif Pro',Arial, sans-serif;
  font-size: 18px;
  line-height: 140%;
  text-shadow: 0px 1px 1px #fff;    

}
h3 { font-size: 180%;

max6cn

body {
    font-family: 'Source Serif Pro','Lucida Grande', 'Lucida Sans Unicode', 'Geneva',
                 'Verdana', sans-serif;
    font-size: 18px;
    background-color: #fdf6e3;

    line-height: 140%;
}
#page-wrapper {
    background-color: #fdf6e3;

max6cn

1. Hadoop

• Install (Comming Soon!)

• Configuration(Comming Soon!)

• Test Hadoop Installation

yarn jar $YARN_EXAMPLES/hadoop-mapreduce-examples-2.5.0.jar pi 80 100000000

2. iPython

max6cn

Decompress a bzip file

bzip2 -d ../enwiki-20150304-pages-articles-multistream.xml.bz2
## takes about 30min
pv ../enwiki-20150304-pages-articles-multistream.xml.bz2 |pbunzip2 > enwiki.xml

takes 3 min(12 core)

max6cn

Demo A. Text Search ( Introdution to programming and parallel computing)

Summary:

Search keywords in a text file

Tools used:

Hadoop(hdfs), Apache Spark, gcc

Language:

C,Shell Script, Python

Task I : Searching keywords in file and counting its frequency

• Oldschool / Standard way Link

max6cn

tmux shortcuts & cheatsheet

start new:

tmux

start new with session name:

tmux new -s myname