Skip to content

Instantly share code, notes, and snippets.

View erikerlandson's full-sized avatar

Erik Erlandson erikerlandson

View GitHub Profile
@erikerlandson
erikerlandson / nats.scala
Created November 6, 2014 22:15
Scala REPL, WTF?
import scala.language.higherKinds
object nat {
trait Nat {
/** The type of this Nat object. */
type Self <: Nat
/** Add another Nat to this one. */
type + [_ <: Nat] <: Nat
@erikerlandson
erikerlandson / threadlocal.scala
Created November 24, 2014 22:40
Minimal ThreadLocal example in Scala
val local:ThreadLocal[String] = new InheritableThreadLocal[String]
local.set("value")
println(s"${local.get()}")
@erikerlandson
erikerlandson / inheritable.scala
Created November 24, 2014 22:56
ThreadLocal vs InheritableThreadLocal example
import scala.concurrent._
import ExecutionContext.Implicits.global
val local1:ThreadLocal[String] = new InheritableThreadLocal[String]
val local2:ThreadLocal[String] = new ThreadLocal[String]
local1.set("foo")
local2.set("bar")
println(s"main: local1= ${local1.get()}")
@erikerlandson
erikerlandson / thread_local_members.scala
Created November 24, 2014 23:05
ThreadLocal structures as class members
import scala.concurrent._
import ExecutionContext.Implicits.global
// global ThreadLocal structures
val local1:ThreadLocal[String] = new InheritableThreadLocal[String]
val local2:ThreadLocal[String] = new ThreadLocal[String]
local1.set("foo")
local2.set("bar")
@erikerlandson
erikerlandson / transitive.scala
Created November 25, 2014 13:39
transitive membership of ThreadLocal
import scala.concurrent._
import ExecutionContext.Implicits.global
// global ThreadLocal structures
val local1:ThreadLocal[String] = new InheritableThreadLocal[String]
val local2:ThreadLocal[String] = new ThreadLocal[String]
local1.set("foo")
local2.set("bar")
@erikerlandson
erikerlandson / override_childValue.scala
Created November 25, 2014 14:31
override childValue for ThreadLocal
import scala.concurrent._
import ExecutionContext.Implicits.global
// ThreadLocal structures as class members
class Context {
private val local: ThreadLocal[String] = new InheritableThreadLocal[String] {
override protected def childValue(parent: String): String = {
println(s"cloning parent ${parent}")
s"(${parent})"
}
@erikerlandson
erikerlandson / demo.scala
Created November 25, 2014 15:27
java.util.Properties copy constructor doesn't
[eje@localhost spark]$ scala
Welcome to Scala version 2.10.3-20130923-e2fec6b28dfd73482945ffab85d9b582d0cb9f17 (OpenJDK 64-Bit Server VM, Java 1.7.0_71).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import java.util.Properties
import java.util.Properties
scala> val p = new Properties
p: java.util.Properties = {}
@erikerlandson
erikerlandson / axioms.scala
Created December 1, 2014 21:19
Exploring how to organize a formalism around a choice of axiom systems
// A formalism that might be defined using different axiom systems
// Not intended to be instantiated directly
trait Formalism {
// from axiom system A/B
def A: Unit
def B: Unit
// from axiom system C/D
def C: Unit
@erikerlandson
erikerlandson / random_forest_example.scala
Created December 18, 2014 18:45
Spark random forest example
// starting Spark REPL at top of source tree:
// $ cd ~/git/spark
// $ ./bin/spark-shell
//
import org.apache.spark.mllib.tree.RandomForest
import org.apache.spark.mllib.util.MLUtils
// Load and parse the data file.
val data = MLUtils.loadLibSVMFile(sc, "data/mllib/sample_libsvm_data.txt")
// Split the data into training and test sets (30% held out for testing)
val splits = data.randomSplit(Array(0.7, 0.3))
@erikerlandson
erikerlandson / kmedoids.scala
Created January 6, 2015 19:12
A k-medoids implementation for Spark RDDs
def kMedoids[T :ClassTag, U >: T :ClassTag](
data: RDD[T],
k: Int,
metric: (U,U) => Double,
sampleSize: Int = 10000,
maxIterations: Int = 10,
resampleInterval: Int = 3
): (Seq[T], Double) = {
val n = data.count