Skip to content

Instantly share code, notes, and snippets.

@autodidacticon

autodidacticon/.bashrc

Created January 22, 2017 00:21
Show Gist options
  • Save autodidacticon/21dfd97516c7b0d440810e21ca10c1e1 to your computer and use it in GitHub Desktop.
Save autodidacticon/21dfd97516c7b0d440810e21ca10c1e1 to your computer and use it in GitHub Desktop.
Download ZIP
EMR pyspark bashrc
Raw
.bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# set the default region for the AWS CLI
export AWS_DEFAULT_REGION=$(curl --retry 5 --silent --connect-timeout 2 http://169.254.169.254/latest/dynamic/instance-identity/document | grep region | awk -F\" '{print $4}')
export JAVA_HOME=/etc/alternatives/jre
# added by Anaconda2 4.2.0 installer
export PATH="/home/hadoop/anaconda2/bin:$PATH"
export PYSPARK_DRIVER_PYTHON=/home/hadoop/anaconda2/bin/jupyter
export PYSPARK_DRIVER_PYTHON_OPTS="notebook --NotebookApp.open_browser=False --NotebookApp.ip='*' --NotebookApp.port=8880"
export PYSPARK_PYTHON=/home/hadoop/anaconda2/bin/python
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment