Skip to content

Instantly share code, notes, and snippets.

@solidpple

solidpple/partitionExample02.scala

Created July 25, 2016 07:24
Show Gist options
  • Save solidpple/264b66671f5a3c221f9bf9cf1dbebf2b to your computer and use it in GitHub Desktop.
Save solidpple/264b66671f5a3c221f9bf9cf1dbebf2b to your computer and use it in GitHub Desktop.
Download ZIP
Raw
partitionExample02.scala
val sc = new SparkContext(...)
val userData = sc.sequenceFile[UserId, UserInfo]("hdfs://...").partitionBy(new HashPartitioner(100)).persist()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment