Install Single Node Hadoop on Ubuntu.md

Readme First

Untuk kejelasan status user saat mengetik command, maka ditentukan hal berikut
Karakter "$" di awal berarti posisi pada user biasa
Karakter "#" di awal berarti posisi pada user root
Tiap command(yang berawalan dengan "$" atau "#") harus dieksekusi dengat urut.
Saat copas/mengetik command, jangan copas karakter "$"atau "#" di awal baris.

Install prequisites

Jika ada prompt untuk masukkan password, masukkan password user Ubuntu.
$ curl https://gist.githubusercontent.com/sayyidyofa/9bb59a9649c110363f4da590e655c37b/raw/10c39bcad1d7732608fca00117b8099e66c270e7/preq.sh > inst && bash inst
$ update-alternatives --config java

Important: perhatikan output dari command kedua, simpan/ingat outpunya dikurangi dengan bagian ../bin/java diakhir. Misal outputnya adalah seperti ini:

There is only one alternative in link group java (providing /usr/bin/java): /usr/lib/jvm/java-11-openjdk-amd64/bin/java
Nothing to configure.

maka simpan bagian usr/lib/jvm/java-11-openjdk-amd64, dari sini string tadi akan direferensikan sebagai $JAVA_HOME

Set up SSH

$ ssh-keygen -t rsa -P "" (enter saja jika ditanya dimana ssh key id_rsa akan disimpan)
$ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

Set up Hadoop, Map Reduce, HDFS

1. edit .bashrc
$ nano ~/.bashrc, pergi ke baris terakhir
   Masukkan kumpulan string berikut:

#HADOOP VARIABLES START
export JAVA_HOME=$JAVA_HOME #INGAT $JAVA_HOME diatas tadi apa
export HADOOP_INSTALL=/usr/local/hadoop
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib"
#HADOOP VARIABLES END

save file tadi dengan CTRL+O lalu Enter lalu exit nano dengan CTRL+X
$ source ~/.bashrc
2. edit hadoop-env.sh
$ nano /usr/local/hadoop/etc/hadoop/hadoop-env.sh
cari line dengan export JAVA_HOME... ganti dengan export JAVA_HOME=$JAVA_HOME #INGAT $JAVA_HOME diatas tadi apa
save file tadi dengan CTRL+O lalu Enter lalu exit nano dengan CTRL+X

3. buat tempdir untuk hadoop
   $ sudo mkdir -p /app/hadoop/tmp && sudo chown $USER:$USER /app/hadoop/tmp

4. edit core-site.xml
$ nano /usr/local/hadoop/etc/hadoop/core-site.xml
   tambahkan kumpulan string berikut diantara tag <configuration> </configuration>

<property>
<name>hadoop.tmp.dir</name>
<value>/app/hadoop/tmp</value>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:54310</value>
<description>The name of the default file system. A URI whose
scheme and authority determine the FileSystem implementation. The
uri’s scheme determines the config property (fs.SCHEME.impl) naming
the FileSystem implementation class. The uri’s authority is used to
determine the host, port, etc. for a filesystem.</description>
</property>

save file tadi dengan CTRL+O lalu Enter lalu exit nano dengan CTRL+X
5. edit mapred-site.xml
$ cp /usr/local/hadoop/etc/hadoop/mapred-site.xml.template /usr/local/hadoop/etc/hadoop/mapred-site.xml && nano /usr/local/hadoop/etc/hadoop/mapred-site.xml
tambahkan kumpulan string berikut diantara tag <configuration> </configuration>

<property>
<name>mapred.job.tracker</name>
<value>localhost:54311</value>
<description>The host and port that the MapReduce job tracker runs
at. If local, then jobs are run in-process as a single map
and reduce task.
</description>
</property>

save file tadi dengan CTRL+O lalu Enter lalu exit nano dengan CTRL+X

6. buat direktori namenode dan datanode
   $ sudo mkdir -p /usr/local/hadoop_store/hdfs/namenode && sudo mkdir -p /usr/local/hadoop_store/hdfs/datanode && sudo chown -R $USER:$USER /usr/local/hadoop_store

7. edit hdfs-site.xml
$ nano /usr/local/hadoop/etc/hadoop/hdfs-site.xml
   tambahkan kumpulan string berikut diantara tag <configuration> </configuration>

<property>
<name>dfs.replication</name>
<value>1</value>
<description>Default block replication.
The actual number of replications can be specified when the file is created.
The default is used if replication is not specified in create time.
</description>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/usr/local/hadoop_store/hdfs/namenode</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/usr/local/hadoop_store/hdfs/datanode</value>
</property>

save file tadi dengan CTRL+O lalu Enter lalu exit nano dengan CTRL+X
8. Inisialisasi HDFS
$ cd /usr/local/hadoop_store/hdfs/namenode
$ hadoop namenode -format

9. Start Hadoop Daemon
   $ cd /usr/local/hadoop/sbin
$ start-all.sh

10. Tes koneksi ke hadoop
    tes dengan pergi ke alamat berikut di browser pada VM: http://localhost:50070/dfshealth.html#tab-overview