Skip to content

Instantly share code, notes, and snippets.

@rssanders3

rssanders3/spark-streaming-set-yarn-cluster-configurations.scala

Last active January 24, 2019 17:53
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save rssanders3/ddddf12e58fa813f7fbf1b043c9d1079 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save rssanders3/ddddf12e58fa813f7fbf1b043c9d1079 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
spark-streaming-set-yarn-cluster-configurations.scala
val sparkConf = new SparkConf()
.setAppName("App")
.set("spark.yarn.maxAppAttempts", "1")
.set("spark.yarn.am.attemptFailuresValidityInterval", "2h")
val ssc = new StreamingContext(sparkConf, Seconds(2))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment