Last active
September 8, 2015 11:28
-
-
Save giwa/6c0786edf8258c8d47d8 to your computer and use it in GitHub Desktop.
Run Hadoop on CentOS 7 ref: http://qiita.com/giwa/items/19d62b4233b90c0b6d4f
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ cat /etc/redhat-release | |
| CentOS Linux release 7.1.1503 (Core) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ sudo systemctl stop firewalld |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ source $HOME/.bashrc |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <property> | |
| <name>fs.default.name</name> | |
| <value>hdfs://localhost:9000</value> | |
| </property> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <property> | |
| <name>dfs.replication</name> | |
| <value>1</value> | |
| </property> | |
| <property> | |
| <name>dfs.name.dir</name> | |
| <value>file:///home/hadoop/hadoopdata/hdfs/namenode</value> | |
| </property> | |
| <property> | |
| <name>dfs.data.dir</name> | |
| <value>file:///home/hadoop/hadoopdata/hdfs/datanode</value> | |
| </property> | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ cp $HADOOP_HOME/etc/hadoop/mapred-site.xml.template $HADOOP_HOME/etc/hadoop/mapred-site.xml | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <property> | |
| <name>mapreduce.framework.name</name> | |
| <value>yarn</value> | |
| </property> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <property> | |
| <name>yarn.nodemanager.aux-services</name> | |
| <value>mapreduce_shuffle</value> | |
| </property> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| export JAVA_HOME=/usr/lib/jvm/jre |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ hdfs namenode -format |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ start-dfs.sh |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ start-yarn.sh |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ yum install java-1.7.0-openjdk |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ useradd hadoop |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ su - hadoop |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ mkdir ~/.ssh | |
| $ ssh-keygen -t rsa -b 4096 -C "example@foo.com" | |
| $ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ chmod 0600 ~/.ssh/authorized_keys |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ ssh localhost | |
| $ exit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ cd | |
| $ wget http://apache.claz.org/hadoop/common/hadoop-2.7.0/hadoop-2.7.0.tar.gz | |
| $ tar zxvf hadoop-2.7.0.tar.gz |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #.bashrc | |
| export JAVA_HOME=/usr/lib/jvm/jre | |
| export HADOOP_HOME=/home/hadoop/hadoop-2.7.0 | |
| export HADOOP_INSTALL=$HADOOP_HOME | |
| export HADOOP_MAPRED_HOME=$HADOOP_HOME | |
| export HADOOP_COMMON_HOME=$HADOOP_HOME | |
| export HADOOP_HDFS_HOME=$HADOOP_HOME | |
| export HADOOP_YARN_HOME=$HADOOP_HOME | |
| export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native | |
| export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin | |
| export JAVA_LIBRARY_PATH=$HADOOP_HOME/lib/native:$JAVA_LIBRARY_PATH |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment