Skip to content

Instantly share code, notes, and snippets.

@joshjdevl
Created November 4, 2014 21:10
Show Gist options
  • Select an option

  • Save joshjdevl/b9af68b11398fd1823c4 to your computer and use it in GitHub Desktop.

Select an option

Save joshjdevl/b9af68b11398fd1823c4 to your computer and use it in GitHub Desktop.
spark sortby compile error
case class Person(first: String, last: String, age: Int)
val parse = (s: String) => {
val split = s.split("\\|")
Person(split(0), split(1), split(2).toInt)
}
object AgeOrdering extends Ordering[Person] {
def compare(a:Person, b:Person) = a.age compare b.age
}
def main(args: Array[String]) {
...
slidingwindow_words.foreachRDD(foreachFunc = rdd => {
rdd.sortBy(parse, true, 2)(AgeOrdering, classTag[Person])
...
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment