Leif Wickland leifwickland
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def appendAt[K,V](m: scala.collection.mutable.Map[K,List[V]], k: K, v: V) = m(k) = (m.getOrElse(k, List[V]()) :+ v) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| type Graph[T] = scala.collection.mutable.MultiMap[T,T] | |
| def newGraph[T]: Graph[T] = new scala.collection.mutable.HashMap[T,scala.collection.mutable.Set[T]] with Graph[T] |
leifwickland
/ gist:1009250
Created
June 5, 2011 18:26
SBT Project.scala to slurp in CHD3 Hadoop Core
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import sbt._ | |
| class Project(info: ProjectInfo) extends DefaultProject(info) { | |
| val clouderaRepo = "cloudera release" at "https://repository.cloudera.com/content/repositories/releases" | |
| val cdhVer = "0.20.2-cdh3u0" | |
| val hadoopCore = "org.apache.hadoop" % "hadoop-core" % cdhVer % "provided" | |
| } |
leifwickland
/ gist:1010403
Created
June 6, 2011 14:57
Example of using futures to schedule work for later
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import scala.actors.Futures | |
| (Futures.awaitAll(9999999,1.to(100000).map(x => Futures.future{Thread.currentThread.getName}): _*).foldLeft(Set[String]()) {(set, name) => set + name.get.toString}).foreach(println); |
leifwickland
/ build.sbt
Created
June 7, 2011 18:38
build.sbt which will pull in hadoop and hbase from Cloudera while excluding Thrift.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| organization := "Example" | |
| name := "example" | |
| scalaVersion := "2.9.0-1" | |
| resolvers += Resolver.url("ClouderaRepo", url("https://repository.cloudera.com/content/repositories/releases")) | |
| libraryDependencies ++= Seq( | |
| "org.apache.hadoop" % "hadoop-core" % "0.20.2-cdh3u0", |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| object MD5 { | |
| def hash(s: String) = { | |
| val m = java.security.MessageDigest.getInstance("MD5") | |
| val b = s.getBytes("UTF-8") | |
| m.update(b, 0, b.length) | |
| new java.math.BigInteger(1, m.digest()).toString(16) | |
| } | |
| } |
leifwickland
/ ThreadLocal.scala
Created
June 25, 2011 04:57
— forked from r/ThreadLocal.scala
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| class ThreadLocal[T](init: => T) extends java.lang.ThreadLocal[T] with Function0[T] { | |
| override def initialValue:T = init | |
| def apply = get | |
| def :=(value: T) { set(value) } | |
| } |
leifwickland
/ gist:1072762
Created
July 8, 2011 20:44
Example of slamming a site with lots of HTTP HEAD requests in Scala
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // Note to self: Use https://github.com/scalaj/scalaj-http next time instead. | |
| import java.net._ | |
| import actors.Future | |
| import actors.Futures._ | |
| object woobox extends App { | |
| val codes = ('a'.to('z').map(_.toString) ++ 0.to(9).map(_.toString)).combinations(6) | |
| val blockSize = 100 | |
| var i = 0 |
leifwickland
/ gist:1090136
Created
July 18, 2011 17:43
Recommended way of pimping Scala types (rather than using implicit conversions) according to Jorge's laws in http://dl.dropbox.com/u/1679797/Scalathon/CanBeConfusing.pdf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| class ByteArrayWrapper(bytes: Array[Byte]) { | |
| def fromUtf8: String = { | |
| "Converted to String." | |
| } | |
| } | |
| object ByteArrayWrapper { | |
| implicit def makeByteArrayWrapper(bytes: Array[Byte]): ByteArrayWrapper = new ByteArrayWrapper(bytes) | |
| } | |
| object byteImplicit extends App { |
leifwickland
/ gist:1099861
Created
July 22, 2011 17:01
Here's one solution I found for fighting with type erasure warnings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| object typeErasure { | |
| def main(args: Array[String]) { | |
| val evenOdd = 1.to(10) | |
| .map{x => if (x%2==0) { Option[Int](x) } else { None } } | |
| .partition{ case Some(_) => true case _ => false } | |
| println(evenOdd) | |
| } | |
| } |
OlderNewer