Skip to content

Instantly share code, notes, and snippets.

View feitang0's full-sized avatar

Fei Tang feitang0

  • UCAS
  • BeiJing, China
View GitHub Profile
@feitang0
feitang0 / docker-cgroups.sh
Created November 30, 2015 12:07
Devices cgroup isn't mounted
# From: https://github.com/docker/docker/issues/8791
cgroupfs_mount() {
# see also https://github.com/tianon/cgroupfs-mount/blob/master/cgroupfs-mount
if grep -v '^#' /etc/fstab | grep -q cgroup \
|| [ ! -e /proc/cgroups ] \
|| [ ! -d /sys/fs/cgroup ]; then
return
fi
if ! mountpoint -q /sys/fs/cgroup; then
mount -t tmpfs -o uid=0,gid=0,mode=0755 cgroup /sys/fs/cgroup
@feitang0
feitang0 / linux-trick.sh
Created November 19, 2015 08:59
Linux trick
# add lines to the beginning of file
sed -i '1s/^/<added text>\n/' file
@feitang0
feitang0 / select-fast-mirror.md
Last active November 19, 2015 08:54
ubuntu mirror

deb mirror://mirrors.ubuntu.com/mirrors.txt precise main restricted universe multiverse
deb mirror://mirrors.ubuntu.com/mirrors.txt precise-updates main restricted universe multiverse
deb mirror://mirrors.ubuntu.com/mirrors.txt precise-backports main restricted universe multiverse
deb mirror://mirrors.ubuntu.com/mirrors.txt precise-security main restricted universe multiverse

add above on the top in /etc/apt/sources.list

@feitang0
feitang0 / spark-on-yarn.md
Last active November 15, 2015 07:48
Spark on Yarn

Spark on Yarn

yarn-client

yarn-client

yarn-cluster

yarn-cluster

@feitang0
feitang0 / spark-learn-note.md
Last active November 15, 2015 07:54
Spark Learn Note

Spark学习笔记

集群环境的挑战

  1. 并行化
  2. 单点失败的处理(节点宕机,个别节点缓慢)
  3. 多用户共享,动态进行计算资源的分配

MapReduce--简单通用和自动容错的批处理计算模型,不适合交互计算,迭代计算和流计算