Skip to content

Instantly share code, notes, and snippets.

@gbraccialli
Last active April 5, 2017 22:36
Show Gist options
  • Save gbraccialli/cb9de4ff862adaddfc65a83497ad09fe to your computer and use it in GitHub Desktop.
Save gbraccialli/cb9de4ff862adaddfc65a83497ad09fe to your computer and use it in GitHub Desktop.
case class Person (name: String, age: Int)
val df = sc.parallelize(List(Person("Guilherme", 35), Person("Isabela", 6), Person("Daniel", 3))).toDF
def wordsLengthScala(a: Any): Array[Int] = {
a match {
case s: String => {s.split(" ").map(s => s.length)}
case i: Integer => {Array(i)}
}
}
val wordsLengthScalaUDF = udf(wordsLengthScala _)
df.printSchema
df.show
df.select(wordsLengthScalaUDF(col("name")),wordsLengthScalaUDF(col("age"))).show
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment