Skip to content

Instantly share code, notes, and snippets.

View feynmanliang's full-sized avatar
💭
sponge

Feynman Liang feynmanliang

💭
sponge
161 followers · 72 following
View GitHub Profile
@feynmanliang
feynmanliang / Solution.java
Created February 22, 2012 23:41
Solution to 2s complement csfall11
/*
Please write complete compilable code.
Your class should be named Solution
Read input from standard input (STDIN) and print output to standard output(STDOUT).
For more details, please check http://www.interviewstreet.com/recruit/challenges/faq/view#stdio
*/
import java.util.Scanner;
public class Solution {
@feynmanliang
feynmanliang / Solution.java
Created February 23, 2012 01:22
csfall11-insertionsort
/*
Please write complete compilable code.
Your class should be named Solution
Read input from standard input (STDIN) and print output to standard output(STDOUT).
For more details, please check http://www.interviewstreet.com/recruit/challenges/faq/view#stdio
*/
import java.util.Scanner;
public class Solution {
@feynmanliang
feynmanliang / calendar.java
Created March 8, 2012 22:24
Quick calendar hack-up for friend
import java.util.Scanner;
public class calendar {
// initialize static scanned
public static Scanner input = new Scanner(System.in);
public static void main(String[] args) {
int month = getMonth();
int year = getYear();
@feynmanliang
feynmanliang / Unit3-SudokuCheck.py
Created July 15, 2012 02:18
Sudoku Checker
# SPECIFICATION:
#
# check_sudoku() determines whether its argument is a valid Sudoku
# grid. It can handle grids that are completely filled in, and also
# grids that hold some empty cells where the player has not yet
# written numbers.
#
# First, your code must do some sanity checking to make sure that its
# argument:
#
@feynmanliang
feynmanliang / SPARK-5016 Perf Testing.md
Last active August 29, 2015 14:24

Cluster

  • Spark 1.4 + 1da3c7f
  • Databricks Cloud
  • 8 Workers, EC2 Spot instances
  • Workers: 240 GB Memory, 32 Cores
  • Driver: 30 GB Memory, 4 Cores

Data

@feynmanliang
feynmanliang / TungstenCache_V0.diff
Last active September 3, 2015 23:36
diff --git a/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java b/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
index 6c02004..83d47c7 100644
--- a/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
+++ b/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
@@ -577,7 +577,9 @@ public String toString() {
StringBuilder build = new StringBuilder("[");
for (int i = 0; i < sizeInBytes; i += 8) {
build.append(java.lang.Long.toHexString(Platform.getLong(baseObject, baseOffset + i)));
- build.append(',');
+ if (i <= sizeInBytes-1) {
@feynmanliang
feynmanliang / TungstenCache V1.diff
Last active September 4, 2015 05:35
diff --git a/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java b/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
index 6c02004..83d47c7 100644
--- a/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
+++ b/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
@@ -577,7 +577,9 @@ public String toString() {
StringBuilder build = new StringBuilder("[");
for (int i = 0; i < sizeInBytes; i += 8) {
build.append(java.lang.Long.toHexString(Platform.getLong(baseObject, baseOffset + i)));
- build.append(',');
+ if (i <= sizeInBytes-1) {
@feynmanliang
feynmanliang / Spark_OnlineLDA_wikipedia_example.scala
Last active April 10, 2020 11:11
import org.apache.spark.ml.feature.{CountVectorizer, RegexTokenizer, StopWordsRemover}
import org.apache.spark.mllib.clustering.{LDA, OnlineLDAOptimizer}
import org.apache.spark.mllib.linalg.Vector
import sqlContext.implicits._
val numTopics: Int = 100
val maxIterations: Int = 100
val vocabSize: Int = 10000
@feynmanliang
feynmanliang / Spark_PrefixSpan_ASL_example.scala
Last active July 8, 2016 13:14
import org.apache.http.client.methods.HttpGet
import org.apache.http.impl.client.{BasicResponseHandler, HttpClientBuilder}
import org.apache.spark.mllib.fpm.PrefixSpan
// sequence database
val sequenceDatabase = {
val url = "http://www.philippe-fournier-viger.com/spmf/datasets/SIGN.txt"
val client = HttpClientBuilder.create().build()
val request = new HttpGet(url)
val response = client.execute(request)
@feynmanliang
feynmanliang / SPARK-10478_microbenchmarks.md
Last active October 10, 2015 19:57

[SPARK-10478] SoftmaxFunction.eval Benchmarks Before/After PR 8648

Num Examples Num Classes Before Average Runtime (ms) After Average Runtime (ms)
1000 5 5.3315174999999995 0.4096546
1000 50 6.7308793 4.1289975
1000 250 21.5866622 21.25164
10000 5 4.2153461 4.145396
10000 50 42.2423785 41.0198213
10000 250 215.2192236 208.61861249999998
100000 5 43.517224899999995 41.2843315