Skip to content

Instantly share code, notes, and snippets.

@pablete

pablete/gist:a0769107e0469d4cd815

Last active August 29, 2015 14:23
Show Gist options
  • Save pablete/a0769107e0469d4cd815 to your computer and use it in GitHub Desktop.
Save pablete/a0769107e0469d4cd815 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.scala
val conf = new SparkConf().setAppName("Simple Application")
val sc = new SparkContext(conf)
val hadoopConf=sc.hadoopConfiguration;
hadoopConf.set("fs.s3.impl","org.apache.hadoop.fs.s3native.NativeS3FileSystem")
hadoopConf.set("fs.s3.awsAccessKeyId", myAccessKey)
hadoopConf.set("fs.s3.awsSecretAccessKey", mySecretKey)
//Also
hadoopConf.set("fs.s3.awsAccessKeyId", System.getProperty("awsAccessKeyId"))
hadoopConf.set("fs.s3.awsSecretAccessKey", System.getProperty("awsSecretAccessKey"))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment