Skip to content

Instantly share code, notes, and snippets.

@rajvermacas
Last active August 28, 2024 09:03
Show Gist options
  • Save rajvermacas/a584d03ead53be2af5be98164c9a46f8 to your computer and use it in GitHub Desktop.
Save rajvermacas/a584d03ead53be2af5be98164c9a46f8 to your computer and use it in GitHub Desktop.
Create spark dataframe from database tables
def readTable(spark: SparkSession, tableName: String): DataFrame = {
val url = sys.env.getOrElse("DB_URL", "default")
val user = sys.env.getOrElse("DB_USERNAME", "default")
val password = sys.env.getOrElse("DB_PASSWORD", "default")
spark.read
.format("jdbc")
.option("url", url)
.option("dbtable", tableName)
.option("user", user)
.option("password", password)
.option("driver", "oracle.jdbc.driver.OracleDriver")
.load()
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment