Last active
February 12, 2020 21:18
-
-
Save habedi/24ee508c348c37f9d12dc522f6afe6cb to your computer and use it in GitHub Desktop.
Simple commands to download and extract Apache Spark's pre-built binaries from its websites
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## run the following commands in BASH | |
cd # let's get back to your user's home directory | |
wget -c https://www-us.apache.org/dist/spark/spark-2.4.5/spark-2.4.5-bin-hadoop2.7.tgz # this will download spark | |
tar xvfz spark-2.4.5-bin-hadoop2.7.tgz # this will extract the downloaded file to current directory | |
mv spark-2.4.5-bin-hadoop2.7 spark # renaming the extarcted folder to "spark" | |
# appending the JAVA_HOME and SPARK_HOME environement variables to end of your BASH startup script | |
# we are assuming that our JRE 8 is installed in "/usr/lib/jvm/java-1.8.0-openjdk-amd64" | |
cat >> .bashrc <<'EOF' | |
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64 | |
export PATH=$JAVA_HOME/bin:$PATH | |
export SPARK_HOME="/home/$USER/spark" | |
export PATH=$SPARK_HOME/sbin:$SPARK_HOME/bin:$PATH | |
EOF | |
# let's restart our terminal | |
bash |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment