Skip to content

Instantly share code, notes, and snippets.

@snowindy
Last active May 27, 2016 01:47
Show Gist options
  • Save snowindy/008f3e8b878a23c00679 to your computer and use it in GitHub Desktop.
Save snowindy/008f3e8b878a23c00679 to your computer and use it in GitHub Desktop.
This is bootstrap action script for Amazon EMR helping installing Apache Zeppelin to work with Apache Spark. Works fine with [EMR Release label:emr-4.0.0 Hadoop distribution:Amazon 2.6.0 Applications:Hive 1.0.0, Spark 1.4.1]
#!/bin/bash
#Zeppelin installation on Spark EMR cluster
sudo yum update -y
sudo yum install -y git
# Installing maven: https://gist.github.com/sebsto/19b99f1fa1f32cae5d00
sudo wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo
sudo sed -i s/\$releasever/6/g /etc/yum.repos.d/epel-apache-maven.repo
sudo yum install -y apache-maven
# Installing npm: http://stackoverflow.com/a/27350635/792313
sudo yum install -y nodejs npm --enablerepo=epel
# Building Zeppelin
git clone https://github.com/apache/incubator-zeppelin.git /home/hadoop/zeppelin
cd /home/hadoop/zeppelin
mvn clean package -Pspark-1.4 -Dhadoop.version=2.6.0 -Phadoop-2.6 -Pyarn -DskipTests
cp conf/zeppelin-env.sh.template conf/zeppelin-env.sh
chmod +x conf/zeppelin-env.sh
cat <<EOF >> conf/zeppelin-env.sh
export MASTER=yarn-client
export HADOOP_CONF_DIR=/etc/hadoop/conf
EOF
# Start the Zeppelin daemon
bin/zeppelin-daemon.sh start
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment