Skip to content

Instantly share code, notes, and snippets.

View chenyanzhe's full-sized avatar

Yanzhe chenyanzhe

View GitHub Profile
@chenyanzhe
chenyanzhe / crawler.sh
Created December 24, 2013 03:25
Gather Geo-Tagged Tweets using Twitter Streaming API
#/bin/bash
# The KEY and SECRET below are invalidate
# Place your correct key here
# Consumer Key
key='x4Ew64IOWg5haePO78w82w'
# Consumer Secret
secret='zXsAy05cGpm1p2r4tJrpqPS7XOfQ5Z4Ga1uSMSCm0Q'
# Access Token
@chenyanzhe
chenyanzhe / Preferences.sublime-settings
Created December 24, 2013 13:34
Sublime Settings - User
{
"font_size": 12,
"ignored_packages":
[
""
],
"rulers":
[
80
],
@chenyanzhe
chenyanzhe / Default.sublime-keymap
Created December 24, 2013 13:35
Sublime Key Bindings - User
[
{ "keys": ["j", "j"], "command": "exit_insert_mode",
"context":
[
{ "key": "setting.command_mode", "operand": false },
{ "key": "setting.is_widget", "operand": false }
]
},
{ "keys": ["g", "t"], "command": "next_view",
@chenyanzhe
chenyanzhe / split.py
Last active January 1, 2016 08:59
Split all files in the corresponding folder into 4 smaller parts accordingly by lines.
#!/usr/bin/env python3
# Usage:
# .
# ├── amazon
# │ ├── amazon_aa
# │ ├── amazon_ab
# │ ├── amazon_ac
# | ...
# │ ├── amazon_bu
@chenyanzhe
chenyanzhe / run
Created December 29, 2013 13:18
MapMatch launcher scripts adapted from Spark run-example script
#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
@chenyanzhe
chenyanzhe / fetch-log
Last active January 1, 2016 17:59
Fetch the latest application log from the Spark cluster and target specific information
#!/bin/bash
# spark.hosts lists all the nodes in the cluster
SPARK_DIR=/home/yanzhe/spark-0.8.1-incubating
LOG_NUM=`ls -t $SPARK_DIR/work | head -n 1`
# tricks to highlight the application id
red='\e[0;31m'
NC='\e[0m' # No Color
@chenyanzhe
chenyanzhe / MaxTemperature.java
Last active January 2, 2016 07:18
HelloWorld of Hadoop
package cn.edu.sjtu.ch02;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
/**
@chenyanzhe
chenyanzhe / login.exp
Created January 27, 2014 02:42
password-less ssh for koding vm
#!/usr/bin/expect -f
set password <password>
set timeout -1
spawn ssh vm-0.<username>.koding.kd.io
expect "*?assword*"
send -- "$password\r"
interact
@chenyanzhe
chenyanzhe / mvim
Created January 28, 2014 06:18
MacVim startup script
#!/bin/sh
#
# This shell script passes all its arguments to the binary inside the
# MacVim.app application bundle. If you make links to this script as view,
# gvim, etc., then it will peek at the name used to call it and set options
# appropriately.
#
# Based on a script by Wout Mertens and suggestions from Laurent Bihanic. This
# version is the fault of Benji Fisher, 16 May 2005 (with modifications by Nico
# Weber and Bjorn Winckler, Aug 13 2007).
@chenyanzhe
chenyanzhe / version-check.sh
Created January 29, 2014 11:43
Simple script to list version numbers of critical development tools
#!/bin/bash
# Simple script to list version numbers of critical development tools
export LC_ALL=C
bash --version | head -n1 | cut -d" " -f2-4
echo "/bin/sh -> `readlink -f /bin/sh`"
echo -n "Binutils: "; ld --version | head -n1 | cut -d" " -f3-
bison --version | head -n1
if [ -e /usr/bin/yacc ];
then echo "/usr/bin/yacc -> `readlink -f /usr/bin/yacc`";