Skip to content

Instantly share code, notes, and snippets.

@SwathiMystery
Created April 15, 2013 15:49
Show Gist options
  • Save SwathiMystery/5389095 to your computer and use it in GitHub Desktop.
Save SwathiMystery/5389095 to your computer and use it in GitHub Desktop.
Monitor the upload to s3, the new file, to the directory /home/ubuntu/data/yyyy/mm/dd/*.*
Goto s3sync directory
$ cd ~/s3sync
$ sudo vim monitor.sh
#!/bin/bash
##...........................................................##
## script to upload to S3BUCKET, once the change is detected ##
##...........................................................##
## AWS Credentials required for s3sync ##
export AWS_ACCESS_KEY_ID=ABCDEFGHSGJBKHKDAKS
export AWS_SECRET_ACCESS_KEY=jhhvftGFHVgs/bagFVAdbsga+vtpmefLOd
export SSL_CERT_DIR=/etc/ssl/certs
#echo "Running monitor.sh!"
echo "[INFO] File or directory modified = $1 "
## Read arguments
PASSED=$1
# Declare the watch path and S3 destination path
watchPath='/home/ubuntu/data'
s3path='bucket-data:'
# Trim watch path from PASSED
out=${PASSED#$watchPath}
outPath=${out#"/"}
echo "[INFO] ${PASSED} will be uploaded to the S3PATH : $s3path$outPath"
if [ -d "${PASSED}" ]
then echo "[SAFEMODE ON] Directory created will not be uploaded, unless a file exists!"
elif [ -f "${PASSED}" ]
then ruby /home/ubuntu/s3sync/s3cmd.rb --ssl put $s3path$outPath ${PASSED} x-amz-storage-class:REDUCED_REDUNDANCY; #USE s3cmd : File
else echo "[ERROR] ${PASSED} is not valid type!!";
exit 1
fi
RETVAL=$?
[ $RETVAL -eq 0 ] && echo "[SUCCESS] Upload successful! " &&
if [ -d "${PASSED}" ]
then echo "[SAFEMODE ON] ${PASSED} is a directory and its not deleted!";
elif [ -f "${PASSED}" ]
then sudo rm -rf ${PASSED}; echo "[SUCCESS] Sync and Deletion successful!";
fi
[ $RETVAL -ne 0 ] && echo "[ERROR] Synchronization failed!!"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment