Skip to content

Instantly share code, notes, and snippets.

@dusenberrymw
Last active January 5, 2017 00:29
Show Gist options
  • Save dusenberrymw/286b3de816bfa6f344021c4d2c5a3f02 to your computer and use it in GitHub Desktop.
Save dusenberrymw/286b3de816bfa6f344021c4d2c5a3f02 to your computer and use it in GitHub Desktop.
Python 3 + Spark 1.6 + SystemML Jupyter kernel template (`/usr/local/share/jupyter/kernels/pyspark3_1.6/kernel.json`).
{
"argv": [
"/usr/local/opt/python3/bin/python3.5",
"-m",
"ipykernel",
"-f",
"{connection_file}"
],
"language": "python",
"display_name": "Python 3 Spark 1.6 SystemML",
"env": {
"SPARK_HOME": "/path/to/spark-1.6.2-bin-hadoop2.6",
"PYTHONPATH": "/path/to/spark-1.6.2-bin-hadoop2.6/python/:/path/to/spark-1.6.2-bin-hadoop2.6/python/lib/py4j-0.9-src.zip",
"PYTHONSTARTUP": "/path/to/spark-1.6.2-bin-hadoop2.6/python/pyspark/shell.py",
"PYSPARK_PYTHON": "python3",
"PYSPARK_SUBMIT_ARGS": "--master local[*] --driver-class-path=/path/to/SystemML.jar pyspark-shell"
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment