Created
January 5, 2019 21:46
-
-
Save allanbatista/d12905d7e71ed1345f8aeba355a9a959 to your computer and use it in GitHub Desktop.
Instalação do spark (helper)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| export SPARK_HOME=/path/to/spark/spark-2.4.0-bin-hadoop2.7 | |
| export PATH=$SPARK_HOME/bin:$PATH | |
| export PYTHONPATH=$SPARK_HOME/python:$PYTHONPATH | |
| export PYSPARK_DRIVE_PYTHON="jupyter" | |
| export PYSPARK_DRIVE_PYTHON_OPTS="notebook" | |
| export PYSPARK_PYTHON=python3 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ pip3 install jupyter py4j --user | |
| $ sudo apt-get install openjdk-8-jdk scala | |
| $ wget http://mirror.nbtelecom.com.br/apache/spark/spark-2.4.0/spark-2.4.0-bin-hadoop2.7.tgz | |
| $ tar xzfv spark-2.4.0-bin-hadoop2.7.tgz | |
| $ export ~/.bashrc |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment