Skip to content

Instantly share code, notes, and snippets.

@Hungsiro506

Hungsiro506/spark_datetime_convert.scala

Created September 11, 2017 12:51
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save Hungsiro506/d12f0acdff34ed8b97f83d92552b6d23 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save Hungsiro506/d12f0acdff34ed8b97f83d92552b6d23 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
spark_datetime_convert.scala
val todayLog_DF = loadData(today,Extensions.ALL).withColumn("Ref",sqlBoolFunc(col("Date")))
.withColumn("Date",from_unixtime(unix_timestamp(col("Date"),"MMM dd yyyy HH:mm:ss"),"MM dd yyyy HH:mm:ss"))
.selectExpr("Name","Date","SessID","Input","Output","Ref")
todayLog_DF
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment