Skip to content

Instantly share code, notes, and snippets.

View ee08b397's full-sized avatar
🎾

Songxiao Zhang ee08b397

🎾
  • BlackRock
  • New York, New York
  • 15:43 (UTC -04:00)
View GitHub Profile
@ee08b397
ee08b397 / git_hint.md
Last active November 5, 2015 02:20
What should be added and what shouldn't

Why I cannot push?

Sometimes when you want to fork a branch and push your exisiting work, github rejects your push (sometimes caused by being offline). git st looks fine.

When I tried to push, it showed this:
 ! [rejected]        master -> master (fetch first)
error: failed to push some refs to 'https://github.com/something.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
@ee08b397
ee08b397 / commit_write.md
Last active December 5, 2015 01:21
Writing good commit messages

From erlang/otp

Good commit messages serve at least three important purposes:

  • To speed up the reviewing process.

  • To help us write a good release note.

  • To help the future maintainers of Erlang/OTP (it could be you!), say five years into the future, to find out why a particular change was made to the code or why a specific feature was added.

@ee08b397
ee08b397 / gist:590b6f179690d05b07af
Created December 9, 2015 19:00 — forked from sontek/gist:af82d281e618074ca8a6
Cassandra benchmark
import time
import uuid
import yappi
import timeit
import logging
import random
from threading import Event
from cassandra import ConsistencyLevel
from cassandra.cluster import Cluster
@ee08b397
ee08b397 / fix.md
Last active December 13, 2015 23:41
Ubuntu-desktop stuck: displayport blinking got display panic

After reboot, gnome got stuck right after the purple "ubuntu" with several dots below, then only one non-flicking cursor on the top left. Then into recovery mode, try startx it says

X: cannot stat /etc/X11/X (No such file or directory), aborting

Fix

sudo dpkg-reconfigure xserver-xorg-lts-utopic

This restores the symlink /etc/X11/X -> /usr/bin/Xorg. I've tried to modify lightdm to gdm doesn't work. And xorg dpkg reconfigure doesn't work because I don't have it installed. I found I have xserver-xorg-lts-utopic.

@ee08b397
ee08b397 / tut.md
Created January 13, 2016 23:37
[ubuntu] reduce wifi channel overlapping

LinSSID is a graphical and functional wifi scanner for Ubuntu Linux, which is similar to iwscanner and Inssider. It can be easily installed in Ubuntu and Linux Mint via PPA repository. (Kismet didn't work for my Ubuntu 15)

sudo add-apt-repository -y ppa:wseverin/ppa
sudo apt-get update; sudo apt-get -y install linssid
@ee08b397
ee08b397 / Spark-SingleRDDfrmMultipleFiles
Created January 20, 2016 17:10 — forked from mkanchwala/Spark-SingleRDDfrmMultipleFiles
Spark : How to create a Single RDD from Multiple Files
Best way to create Mutliple files into a single RDD
==================================
val fileRDD = sc.textFile(filename).repartition(1)
Where the filename is the location of your directory only.
@ee08b397
ee08b397 / Kafka MultiNode - MultiBroker Cluster.md
Created January 20, 2016 17:10 — forked from mkanchwala/Kafka MultiNode - MultiBroker Cluster.md
Create Kafka Multi Node, Multi Broker Cluster

Setting up Flume NG, listening to syslog over UDP, with an S3 Sink

My goal was to set up Flume on my web instances, and write all events into s3, so I could easily use other tools like Amazon Elastic Map Reduce, and Amazon Red Shift.

I didn't want to have to deal with log rotation myself, so I setup Flume to read from a syslog UDP source. In this case, Flume NG acts as a syslog server, so as long as Flume is running, my web application can simply write to it in syslog format on the specified port. Most languages have plugins for this.

At the time of this writing, I've been able to get Flume NG up and running on 3 ec2 instances, and all writing to the same bucket.

Install Flume NG on instances

@ee08b397
ee08b397 / ncdc.sh
Created February 2, 2016 00:27 — forked from aligusnet/ncdc.sh
Download a weather dataset from the National Climatic Data Center (NCDC, http://www .ncdc.noaa.gov/). Prepare it for examples of "Hadoop: The Definitive Guide" book by Tom White. http://www.amazon.com/Hadoop-Definitive-Guide-Tom-White/dp/1449311520 Usage: ./ncdc.sh 1901 1930 # download wheather datasets for period from 1901 to 1930.
#!/usr/bin/env bash
# global parameters
g_tmp_folder="ncdc_tmp";
g_output_folder="ncdc_data";
g_remote_host="ftp3.ncdc.noaa.gov";
g_remote_path="pub/data/noaa";