Skip to content

Instantly share code, notes, and snippets.

@ottomata
Created June 26, 2018 21:15
Show Gist options
  • Save ottomata/d5cddb5abd5eb3785a26113e7e187157 to your computer and use it in GitHub Desktop.
Save ottomata/d5cddb5abd5eb3785a26113e7e187157 to your computer and use it in GitHub Desktop.
{
"metadata": {},
"display_name": "Spark YARN - PySpark",
"interrupt_mode": "signal",
"language": "python",
"env": {
"PYTHONPATH": "/usr/lib/spark2/python/lib/py4j-0.10.6-src.zip:/usr/lib/spark2/python",
"__TOREE_SPARK_OPTS__": "--master=yarn --conf spark.executorEnv.PYTHONPATH=/usr/lib/spark2/python/lib/py4j-0.10.6-src.zip:/usr/lib/spark2/python",
"DEFAULT_INTERPRETER": "PySpark",
"SPARK_HOME": "/usr/lib/spark2",
"PYTHON_EXEC": "/usr/bin/python3",
"PYSPARK_PYTHON": "/usr/bin/python3",
"__TOREE_OPTS__": ""
},
"argv": [
"/srv/home/otto/.local/share/jupyter/kernels/spark_yarn_pyspark/bin/run.sh",
"--profile",
"{connection_file}"
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment