Skip to content

Instantly share code, notes, and snippets.

@ankyhe
ankyhe / SparkWithHadoop.md
Created November 26, 2019 02:55 — forked from afranzi/SparkWithHadoop.md
Spark 2.4.0 with Hadoop 2.8.5

Setup Environmnet variables for Hadoop.

export HADOOP_VERSION=2.8.5
export HADOOP_HOME=${HOME}/hadoop-$HADOOP_VERSION
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export PATH=${HADOOP_HOME}/bin:$PATH

Download Hadoop files.