Skip to content

Instantly share code, notes, and snippets.

@JoshRosen
Created May 24, 2013 18:18
Show Gist options
  • Save JoshRosen/5645465 to your computer and use it in GitHub Desktop.
Save JoshRosen/5645465 to your computer and use it in GitHub Desktop.
name := "Simple Project"
version := "1.0"
scalaVersion := "2.9.2"
libraryDependencies += "org.spark-project" %% "spark-core" % "0.7.0"
resolvers ++= Seq(
"Typesafe Repository" at "http://repo.typesafe.com/typesafe/releases/",
"Spray Repository" at "http://repo.spray.cc/")
import spark.SparkContext
import SparkContext._
object SimpleJob extends App {
val master = "spark://ec2-XXX-XXX-XXX-XXX.compute-1.amazonaws.com:7077",
val sc = new SparkContext(master, "Simple Job", "/root/spark", List("target/scala-2.9.2/simple-project_2.9.2-1.0.jar"))
val logFile = "/usr/share/dict/words"
val lines = sc.textFile(logFile).cache
println("COUNT: %d".format(lines.count))
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment