Skip to content

Instantly share code, notes, and snippets.

@glatrofa

glatrofa/useful-hadoop-instructions.md

Last active December 13, 2020 08:32
Show Gist options
  • Save glatrofa/1f33c52eba2831a6db9d4043ada596ed to your computer and use it in GitHub Desktop.
Save glatrofa/1f33c52eba2831a6db9d4043ada596ed to your computer and use it in GitHub Desktop.
Download ZIP
Raw
useful-hadoop-instructions.md

These commands have to be axecuted within the "hadoop-x.x.x" folder

  • Start/stop hadoop
sbin/start-dfs.sh
sbin/stop-dfs.sh
  • Create/remove folder
bin/hdfs dfs -mkdir <folder>
bin/hdfs dfs -rm -r  <folder>
  • Send/receive folder/file from hdfs
bin/hdfs dfs -put <local_folder> <hdfs_folder>
bin/hdfs dfs -get <local_folder> <hdfs_folder>
  • Show files contained in a folder
bin/hdfs dfs -cat folder/*
  • Start/stop YARN
sbin/start-yarn.sh
sbin/stop-yarn.sh
  • Start jar file
bin/hadoop jar <path-jar> <jar-MainClass> <jar-parameters>

  • Hadoop pseudo-distributed mode:

    • etc/hadoop/core-site.xml
    • etc/hadoop/hdfs-site.xml

  • Hadoop pseudo-distributed mode with yarn:

    • etc/hadoop/yarn-site.xml
    • etc/hadoop/mapred-site.xml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment