Skip to content

Instantly share code, notes, and snippets.

@gbraccialli

gbraccialli/scala_parallel_collection.scala

Last active December 12, 2017 21:34
Show Gist options
  • Save gbraccialli/49b011d2d4fe72e28c8e3996398557e9 to your computer and use it in GitHub Desktop.
Save gbraccialli/49b011d2d4fe72e28c8e3996398557e9 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
scala_parallel_collection.scala
import scala.collection.parallel._
val df = (1 to 10).toDF
val list = Seq(1 to 300).par
list.tasksupport = new ForkJoinTaskSupport(new scala.concurrent.forkjoin.ForkJoinPool(2))
val listR = list.map(l => df.withColumn("l", lit(l.toString)).groupBy("l").count.collect)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment