Skip to content

Instantly share code, notes, and snippets.

@yaravind

yaravind/spark-cluster-windows

Last active November 17, 2017 07:37
Show Gist options
  • Save yaravind/c768fb4300beb9977fa3a9f3d5eb010b to your computer and use it in GitHub Desktop.
Save yaravind/c768fb4300beb9977fa3a9f3d5eb010b to your computer and use it in GitHub Desktop.
Download ZIP
Raw
spark-cluster-windows
# - Prints all options available to start the worker
%SPARK_HOME%\bin\spark-class.cmd org.apache.spark.deploy.worker.Worker --help
# - Prints all options available to start the master
%SPARK_HOME%\bin\spark-class.cmd org.apache.spark.deploy.master.Master --help
# - Start master - Web UI on http://localhost:8080
%SPARK_HOME%\bin\spark-class.cmd org.apache.spark.deploy.master.Master
# - Start worker with 1 core and 64mb ram - Web UI on http://localhost:8081
%SPARK_HOME%\bin\spark-class.cmd org.apache.spark.deploy.worker.Worker -c 1 -m 64m -d C:/aravind/sw/spark-worker-dir spark://ip-address:7077
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment