| conf = ( | |
| pyspark.SparkConf() | |
| .setAppName("Iceberg Partitioned Data Write") | |
| .set("spark.jars", jdbc_driver_path) # Include the JDBC driver | |
| .set("spark.jars.packages", "org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.5.2,org.projectnessie.nessie-integrations:nessie-spark-extensions-3.5_2.12:0.91.3,software.amazon.awssdk:bundle:2.17.81,org.apache.hadoop:hadoop-aws:3.3.1") # Include Iceberg, Nessie, AWS SDK, and Hadoop AWS packages | |
| .set("spark.sql.extensions", "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,org.projectnessie.spark.extensions.NessieSparkSessionExtensions") # Corrected Spark session extensions | |
| .set("spark.sql.catalog.nessie", "org.apache.iceberg.spark.SparkCatalog") | |
| .set("spark.sql.catalog.nessie.uri", NESSIE_URI) | |
| .set("spark.sql.catalog.nessie.ref", "main") | |
| .set("spark.sql.catalog.nessie.authentication.type", "NONE") |
| title | Coder Kubernetes Docker-in-Docker Setup with RKE2 |
|---|---|
| author | joshyorko, [@joshyorko](https://github.com/joshyorko), joshua.yorko@gmail.com |
| ENVBOX_VERSION | 0.6.1 |
| RKE2_VERSION | v1.30.6 |
| CNI_USED | Flannel |
| OS_AND_KERNEL | Ubuntu Focal (20.04), Kernel 5.15.0 |
Objective This agent automates the process of creating, configuring, testing, and deploying Robocorp robot projects using RCC CLI commands. It guides users through the entire lifecycle of a Robocorp robot, from initial scaffolding to deployment in Control Room.
Instructions