Last active
June 23, 2024 06:48
-
-
Save domderen/27caeb7eb02118d71279 to your computer and use it in GitHub Desktop.
Installation of apache spark on ubuntu machine.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/sh | |
# installation of Oracle Java JDK. | |
sudo apt-get -y update | |
sudo apt-get -y install python-software-properties | |
sudo add-apt-repository -y ppa:webupd8team/java | |
sudo apt-get -y update | |
sudo apt-get -y install oracle-java7-installer | |
# Installation of commonly used python scipy tools | |
sudo apt-get -y install python-numpy python-scipy python-matplotlib ipython ipython-notebook python-pandas python-sympy python-nose | |
# Installation of scala | |
wget http://www.scala-lang.org/files/archive/scala-2.11.1.deb | |
sudo dpkg -i scala-2.11.1.deb | |
sudo apt-get -y update | |
sudo apt-get -y install scala | |
# Installation of sbt | |
wget http://scalasbt.artifactoryonline.com/scalasbt/sbt-native-packages/org/scala-sbt/sbt//0.12.3/sbt.deb | |
sudo dpkg -i sbt.deb | |
sudo apt-get -y update | |
sudo apt-get -y install sbt | |
# Downloading spark | |
wget http://d3kbcqa49mib13.cloudfront.net/spark-1.0.0.tgz | |
tar -zxf spark-1.0.0.tgz | |
cd spark-1.0.0 | |
# Building spark | |
./sbt/sbt assembly | |
# Clean-up | |
rm scala-2.11.1.deb | |
rm sbt.deb | |
rm spark-1.0.0.tgz | |
rm install.sh |
Hi, everyone
I want to know how to download apache spark in eclipse as open source code for editing its scheduling algorithm
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@sypher123: FYI I forked using spark-1.6 (and adding set -e to stop execution on error)
Thx for this script tho @domderen