Skip to content

Instantly share code, notes, and snippets.

@bepcyc

bepcyc/AvroSchemaFromDataFrame.scala

Created February 21, 2019 17:28
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save bepcyc/22f5fbd2192a888fd5669bb4006fbd44 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save bepcyc/22f5fbd2192a888fd5669bb4006fbd44 to your computer and use it in GitHub Desktop.
Download ZIP
Generate Avro Schema out of Apache Spark DataFrame
Raw
AvroSchemaFromDataFrame.scala
import org.apache.spark.sql.avro.SchemaConverters
SchemaConverters.toAvroType(df.schema) // add .toString if you need JSON here
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment