Created
August 21, 2010 14:51
-
-
Save objectoriented/542408 to your computer and use it in GitHub Desktop.
Ensure complete backup or copy of an Amazon S3 bucket (compare file lists on S3 vs your filesystem)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/sh | |
# | |
# Diff file lists to find missing files on local copy | |
# | |
s3cmd ls -r s3://<<..S3 BUCKET..>>/|awk '{ print $4 }' | awk '{sub(/^s3\:\/\/<<..S3 BUCKET..>>\//, "")};1' | sort > /tmp/s3-file-list.txt.sorted | |
find . -type f | awk '{sub(/\.\//, "")};1' | sort > /tmp/my-file-list.txt.sorted | |
diff /tmp/s3-file-list.txt.sorted /tmp/my-file-list.txt.sorted | |
# | |
# Compare file counts | |
# | |
find . -type f|wc | |
s3cmd ls -r s3://<<..S3 BUCKET..>>/|wc |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment