Skip to content

Instantly share code, notes, and snippets.

@64lines

64lines/coalesce.py

Created February 19, 2019 22:16
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save 64lines/a548b3bcc53ae72cabab8048936c33de to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save 64lines/a548b3bcc53ae72cabab8048936c33de to your computer and use it in GitHub Desktop.
Download ZIP
Raw
coalesce.py
from pyspark.sql.functions import *
fact_df = fact_df.withColumn('columnname', coalesce(col('columnname'), col('columnname_if_null'), col('columnname_if_null'), col('columnname_if_null'))).alias('fact_df')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment