soulmachine / gist:269b0302acaa161b470e67f2ff243646

Created February 1, 2017 22:55 — forked from slawdan/gist:2848568

敏感词列表

	王
	江
	周
	胡
	刘
	李
	吴
	毛
	温
	习

soulmachine / gist:a61b5d3f690b117b037c9654623bd451

Created September 9, 2016 06:16 — forked from stuart11n/gist:9628955

rename git branch locally and remotely

	git branch -m old_branch new_branch # Rename branch locally
	git push origin :old_branch # Delete the old branch
	git push --set-upstream origin new_branch # Push the new branch, set local branch to track the new remote

soulmachine / ubuntu-mount-new-disk.md

Created August 17, 2016 08:42 — forked from gaoyifan/ubuntu-mount-new-disk.md

ubuntu 添加新硬盘

查看硬盘：

# fdisk -l
...
Disk /dev/sdb: 274.9 GB, 274877906944 bytes
255 heads, 63 sectors/track, 33418 cylinders, total 	536870912 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes

soulmachine / keybase.md

Created July 25, 2016 18:04

Keybase proof

I hereby claim:

I am soulmachine on github.
I am soulmachine (https://keybase.io/soulmachine) on keybase.
I have a public key whose fingerprint is 1F53 41FB 3CD9 D064 FB2E 0D4D 5B13 D2E7 709F 1721

To claim this, I am signing this object:

soulmachine / UserDailySnapshot.scala

Created June 18, 2016 15:13

	package com.cmcm.bdp

	import java.io.FileInputStream
	import java.net.InetAddress

	import com.databricks.spark.avro._
	import com.maxmind.geoip2.DatabaseReader
	import org.apache.spark._
	import org.apache.spark.sql.{DataFrame, Row}
	import org.apache.spark.sql.expressions.Window

soulmachine / MyConcurrentHashMap.java

Last active February 1, 2016 02:00

My implementations of HashMap and ConcurrentHashMap。难点，MyConcurrentHashMap 的 get() 可以不用加锁，大大提高了性能

	import java.util.concurrent.locks.ReentrantLock;

	public class MyConcurrentHashMap<K,V> {
	private static final int DEFAULT_CONCURRENCY_LEVEL = 16;

	/**
	* The default initial capacity - MUST be a power of two.
	*/
	static final int DEFAULT_INITIAL_CAPACITY = 256;

soulmachine / MyArrayBlockingQueue.java

Last active January 31, 2016 06:38

My implementations of BlockingQueue

	import java.util.concurrent.locks.Condition;
	import java.util.concurrent.locks.ReentrantLock;

	// Reference: http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/locks/Condition.html
	public class MyArrayBlockingQueue<E> {
	final ReentrantLock lock = new ReentrantLock();
	final Condition notFull = lock.newCondition();
	final Condition notEmpty = lock.newCondition();

	final Object[] items;

soulmachine / WordCountPercentage.java

Last active November 12, 2019 18:50

Calculate Word Count Percentage

	package me.soulmachine;

	import org.apache.hadoop.conf.Configuration;
	import org.apache.hadoop.conf.Configured;
	import org.apache.hadoop.fs.Path;
	import org.apache.hadoop.io.LongWritable;
	import org.apache.hadoop.io.Text;
	import org.apache.hadoop.mapreduce.Job;
	import org.apache.hadoop.mapreduce.Mapper;
	import org.apache.hadoop.mapreduce.Reducer;

soulmachine / WordCount.java

Last active August 29, 2015 14:14

Polished code based on http://wiki.apache.org/hadoop/WordCount

	package me.soulmachine;

	import org.apache.hadoop.conf.Configuration;
	import org.apache.hadoop.conf.Configured;
	import org.apache.hadoop.fs.Path;
	import org.apache.hadoop.io.IntWritable;
	import org.apache.hadoop.io.LongWritable;
	import org.apache.hadoop.io.Text;
	import org.apache.hadoop.mapreduce.Job;
	import org.apache.hadoop.mapreduce.Mapper;

soulmachine / Mutual.scala

Last active September 15, 2017 12:52

使用 Spark 计算新浪微博的互相关注的关系。输入为一个文本文件，每行的格式为 userid1, userid2, userid3,...,useridN, 表示用户userid1关注了userid2, userid3,...,useridN。输出也是一个文本文件，每行格式为 userid1, userid2，表示这两个用户互相关注了。

	package com.yanjiuyanjiu.weibo

	import org.apache.spark.SparkContext._
	import org.apache.spark.{SparkContext, Logging}


	/**
	* 查找互相关注的关系。
	*
	* 输入为一个文本文件，每行的格式为 userId1, userId2, userId3,..., userIdN，表示 userId1 关注了 userId2, userId3, ..., userIdN

Frank Dai soulmachine

ubuntu 添加新硬盘

Keybase proof