Created
September 9, 2019 22:31
-
-
Save galanteh/04f7a35a5c71fc7c41dfc9ba9f9fe0b8 to your computer and use it in GitHub Desktop.
Compilation of Cloudera Apache Livy 0.3.0 on CentOS 7.5
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Choose an Edge node. Login as root. | |
| # This recipe works to compile Apache Livy 0.3.0 from Cloudera repo. | |
| yum -y install maven | |
| cd /opt | |
| wget https://github.com/cloudera/livy/archive/v0.3.0.zip | |
| unzip v0.3.0.zip | |
| cd livy-0.3.0 | |
| yum -y install python-pip | |
| python -m pip install --upgrade pip setuptools wheel | |
| pip install cloudpickle requests flake8 flaky pyparsing pytest | |
| mvn -DskipTests=true clean package |
Submit a testing job
curl -H "Content-Type: application/json" -X POST -d '{"code":"println(sc.parallelize(1 to 5).collect())"}' -i http://localhost:8998/sessions/0/statements
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Starting Livy Server
export SPARK_HOME=/opt/cloudera/parcels/CDH/lib/spark
export HADOOP_CONF_DIR=/etc/hadoop/conf
./bin/livy-server start