Created
November 24, 2014 20:30
-
-
Save elliottcordo/39ebb6274854f2cd71bf to your computer and use it in GitHub Desktop.
emr spark pyspark fix
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| unzip -d tmp1 spark/lib/spark-assembly-1.1.0-hadoop2.4.0.jar | |
| cd tmp1 | |
| #run the line below assuming openjdk is not installed on your EMR cluster (it's probably not) | |
| sudo yum install -y java-1.6.0-openjdk-devel.x86_64 | |
| /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.33.x86_64/bin/jar cvmf META-INF/MANIFEST.MF ../spark/lib/spark-assembly-1.1.0-hadoop2.4.0.jar . |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment