Skip to content

Instantly share code, notes, and snippets.

@prateek

prateek/newdistcp-usage-notes.md

Created November 8, 2014 20:37
Show Gist options
  • Save prateek/dac917ceb8c81b926f9d to your computer and use it in GitHub Desktop.
Save prateek/dac917ceb8c81b926f9d to your computer and use it in GitHub Desktop.
Download ZIP
Raw
newdistcp-usage-notes.md

New Distcp Usage Notes

Make sure you can access the S3 bucket with a command such as:

$ hdfs dfs -put localfile s3n://$BUCKET/

If this works, uncompress the attached jar and try the command below.

$ hadoop jar $MYLIBPATH/hadoop-tools-1.3.0-SNAPSHOT.jar \
  org.apache.hadoop.tools.NewDistCp                     \
  src s3n://$BUCKET/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment