Skip to content

Instantly share code, notes, and snippets.

@YordanGeorgiev

YordanGeorgiev/delete-file-if-it-exists.scala

Created September 10, 2018 14:04
Show Gist options
  • Save YordanGeorgiev/8143e3b2b496f71960423ac517cd6245 to your computer and use it in GitHub Desktop.
Save YordanGeorgiev/8143e3b2b496f71960423ac517cd6245 to your computer and use it in GitHub Desktop.
Download ZIP
[delete file if exists in spark] how-to delete a file if it exists in hdfs spark scala
Raw
delete-file-if-it-exists.scala
val fs = org.apache.hadoop.fs.FileSystem.get(spark.sparkContext.hadoopConfiguration)
if (fs.exists(new org.apache.hadoop.fs.Path(cleanedAdpFileUri)))
fs.delete(new Path(cleanedAdpFileUri), true)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment