Skip to content

Instantly share code, notes, and snippets.

@gcsfred

gcsfred/dataframe_using_pandas_udf.py

Created November 15, 2018 12:43
Show Gist options
  • Save gcsfred/a68e6d7d4c01ba6ba6bd8e178b805578 to your computer and use it in GitHub Desktop.
Save gcsfred/a68e6d7d4c01ba6ba6bd8e178b805578 to your computer and use it in GitHub Desktop.
Download ZIP
dataframe using pandas_udf and one hot encode
Raw
dataframe_using_pandas_udf.py
dataframe = dataframe.withColumn('ACOLUMN_not_null', pandas_not_null('ACOLUMN'))
dataframe = one_hot_encode(dataframe, "ACOLUMN_not_null", "ACOLUMN_one_hot")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment