Skip to content

Instantly share code, notes, and snippets.

@yonglai

yonglai/install_spark_2.1.1.sh

Last active May 29, 2017 20:48
Show Gist options
  • Save yonglai/41419c0b190c09581e17d856a4b40788 to your computer and use it in GitHub Desktop.
Save yonglai/41419c0b190c09581e17d856a4b40788 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
install_spark_2.1.1.sh
#! /bin/bash
echo download scala
wget http://downloads.typesafe.com/scala/2.12.2/scala-2.12.2.tgz
tar xvf scala-2.12.2.tgz
mv scala-2.12.2 /usr/lib
ln -s /usr/lib/scala-2.12.2 /usr/lib/scala
export PATH=$PATH:/usr/lib/scala/bin
scala -version
echo download spark
wget http://d3kbcqa49mib13.cloudfront.net/spark-2.1.1-bin-hadoop2.7.tgz
tar xvf spark-2.1.1-bin-hadoop2.7.tgz
export SPARK_HOME=$HOME/spark-2.1.1-bin-hadoop2.7
export PATH=$PATH:$SPARK_HOME/bin
firewall-cmd --permanent --zone=public --add-port=6066/tcp
firewall-cmd --permanent --zone=public --add-port=7077/tcp
firewall-cmd --permanent --zone=public --add-port=8080-8081/tcp
firewall-cmd --reload
echo 'export PATH=$PATH:/usr/lib/scala/bin' >> .bash_profile
echo 'export SPARK_HOME=$HOME/spark-2.1.1-bin-hadoop2.7' >> .bash_profile
echo 'export PATH=$PATH:$SPARK_HOME/bin' >> .bash_profile
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment