Skip to content

Instantly share code, notes, and snippets.

View byronyi's full-sized avatar
:octocat:
Just for fun

Bairen Yi byronyi

:octocat:
Just for fun
View GitHub Profile
docker run --rm -it -v `pwd`/hadoop:/hadoop -e "HADOOP_HOME=/hadoop" -v `pwd`/java:/java -e "JAVA_HOME=/java" -v `pwd`/spark:/spark -e "SPARK_HOME=/spark" -v `pwd`/conf:/conf -e "HADOOP_CONF_DIR=/conf" --link resource-manager --link name-node ubuntu
/spark/bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn-cluster --num-executors 3 --driver-memory 4g --executor-memory 2g --executor-cores 1 /spark/lib/spark-examples*.jar 10
docker run -d -v `pwd`/hadoop:/hadoop -e "HADOOP_HOME=/hadoop" -v `pwd`/java:/java -e "JAVA_HOME=/java" -v `pwd`/conf:/conf -e "HADOOP_CONF_DIR=/conf" --name=name-node -h name-node -p 50070:50070 ubuntu bash -c "/hadoop/bin/hdfs namenode -format && /hadoop/bin/hdfs namenode"
docker run -d -v `pwd`/hadoop:/hadoop -e "HADOOP_HOME=/hadoop" -v `pwd`/java:/java -e "JAVA_HOME=/java" -v `pwd`/conf:/conf -e "HADOOP_CONF_DIR=/conf" --link name-node ubuntu bash -c "/hadoop/bin/hdfs datanode"
docker run -d -v `pwd`/hadoop:/hadoop -e "HADOOP_HOME=/hadoop" -v `pwd`/java:/java -e "JAVA_HOME=/java" -v `pwd`/conf:/conf -e "HADOOP_CONF_DIR=/conf" --name=resource-manager -h resource-manager -p 8088:8088 ubuntu bash -c "/hadoop/bin/yarn resourcemanager"
docker run -d -v `pwd`/hadoop:/hadoop -e "HADOOP_HOME=/hadoop" -v `pwd`/java:/java -e "JAVA_HOME=/java" -v `pwd`/conf:/conf -e "HADOOP_CONF_DIR=/conf" --link resource-manager --link name-node ubuntu bash -c "/hadoop/bin/yarn nodemanager"
docker run --rm -it -v `pwd`/hadoop:/
# -*- mode: ruby -*-
# vi: set ft=ruby :
# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure(2) do |config|
# The most common configuration options are documented and commented below.
# For a complete reference, please see the online documentation at
@byronyi
byronyi / iunzip.py
Last active August 29, 2015 14:22 — forked from andrix/iunzip.py
import itertools
from operator import itemgetter
def iunzip(iterable):
"""Iunzip is the same as zip(*iter) but returns iterators, instead of
expand the iterator. Mostly used for large sequence"""
_tmp, iterable = itertools.tee(iterable, 2)
iters = itertools.tee(iterable, len(_tmp.next()))
return (itertools.imap(itemgetter(i), it) for i, it in enumerate(iters))
@byronyi
byronyi / ming.md
Last active August 29, 2015 14:21

Government in the Ming Dynasty

Legacies of the Hongwu Emperor

  • During the 276 years of the Ming China's population doubled
  • Destructive domestic warfare was largely avoided
  • Great achievements education and philosophy, literature and art, reflected the high cultural level of the elite gentry society
  • Did not attempt a continuation of the Song but tried in theory to go back to the models of Han and Tang
@byronyi
byronyi / qing.md
Last active August 29, 2015 14:21

The Qing Success Story

  • Manchu tribes, one of the descendants of Jurchen
  • Manchuria in the 16th century had been brought under Chinese type of intensive agriculture only in the southernmost region
  • The Ming had recognized the frontier nature of this region by organizing it in the military districts rather than under a civil administration only
  • In their rise to power the Manchus took full advantage of their strategic position on a frontier where they could learn Chinese ways and yet not be entirely subjected to Chinese rule

The Manchu Conquest

#!/usr/bin/env python
from __future__ import division, print_function, unicode_literals
import os
from datetime import datetime
from tempfile import mkdtemp
import joblib
import tempfile
from joblib import Memory
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title>K-means 聚类分析</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/require.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js"></script>
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Fresh rustc-serialize v0.3.12
Fresh toml v0.1.20
Compiling racer v0.0.1 (file:///Users/byronyi/.vim/bundle/racer)
Running `rustc src/main.rs --crate-name racer --crate-type bin -C opt-level=3 -g --out-dir /Users/byronyi/.vim/bundle/racer/target/release --emit=dep-info,link -L dependency=/Users/byronyi/.vim/bundle/racer/target/release -L dependency=/Users/byronyi/.vim/bundle/racer/target/release/deps --extern toml=/Users/byronyi/.vim/bundle/racer/target/release/deps/libtoml-e4d393d734aaa578.rlib`
src/racer/ast.rs:51:5: 53:7 error: mismatched types:
expected `syntax::ast::Crate`,
found `core::result::Result<syntax::ast::Crate, syntax::diagnostic::FatalError>`
(expected struct `syntax::ast::Crate`,
found enum `core::result::Result`) [E0308]
src/racer/ast.rs:51 with_error_checking_parse(source_str, |p| {