Skip to content

Instantly share code, notes, and snippets.

@mkanchwala

mkanchwala/gist:faf3f5e034c8638390ed

Created April 30, 2015 08:57
Show Gist options
  • Save mkanchwala/faf3f5e034c8638390ed to your computer and use it in GitHub Desktop.
Save mkanchwala/faf3f5e034c8638390ed to your computer and use it in GitHub Desktop.
Download ZIP
Spark : How to create a Single RDD from Multiple Files
Raw
gistfile1.md

Best way to create Mutliple files into a single RDD

val fileRDD = sc.textFile(filename).repartition(1)

Where the filename is the location of your directory only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment