Skip to content

Instantly share code, notes, and snippets.

@ashkrit
Created January 26, 2019 05:44
Show Gist options
  • Save ashkrit/cffc999f4ecd2af44e8a796b9d64348c to your computer and use it in GitHub Desktop.
Save ashkrit/cffc999f4ecd2af44e8a796b9d64348c to your computer and use it in GitHub Desktop.
private def checkSparkContext(): Unit = {
val startTime = System.currentTimeMillis()
log.info("Context creation monitor thread started")
while (SparkContextBuilder.isContextNotCreated()) {
Thread.sleep(TimeUnit.SECONDS.toMillis(1))
val total = System.currentTimeMillis() - startTime
log.info("Waiting for spark context from {} seconds", TimeUnit.MILLISECONDS.toSeconds(total))
}
log.info("Spark context creation took {} seconds", TimeUnit.MILLISECONDS.toSeconds(System.currentTimeMillis() - startTime))
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment