Skip to content

Instantly share code, notes, and snippets.

@buildlackey

buildlackey/savedTz

Created September 6, 2019 22:47
Show Gist options
  • Save buildlackey/00995e00cf76c85ffaab3c90fde6c675 to your computer and use it in GitHub Desktop.
Save buildlackey/00995e00cf76c85ffaab3c90fde6c675 to your computer and use it in GitHub Desktop.
Download ZIP
savedTz
Raw
savedTz
val savedTz = spark.conf.get("spark.sql.session.timeZone")
spark.conf.set("spark.sql.session.timeZone", "GMT")
List("1970-01-01T00:00:00-01:00").toDF("timestr").
withColumn("ts", col("timestr").cast("timestamp")).
withColumn("tsAsInt", col("ts").cast("integer")).
withColumn("asUtc", date_format($"ts", "yyyy-MM-dd'T'HH:mm:ssX")).
show(false)
spark.conf.set("spark.sql.session.timeZone", savedTz )
// RESULT:
// +-------------------------+-------------------+-------+--------------------+
// |timestr |ts |tsAsInt|asUtc |
// +-------------------------+-------------------+-------+--------------------+
// |1970-01-01T00:00:00-01:00|1970-01-01 01:00:00|3600 |1970-01-01T01:00:00Z|
// +-------------------------+-------------------+-------+--------------------+
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment