Skip to content

Instantly share code, notes, and snippets.

@ryantbrown
Created February 1, 2018 07:25
Show Gist options
  • Save ryantbrown/b643f678f174013ae92e488850ce699f to your computer and use it in GitHub Desktop.
Save ryantbrown/b643f678f174013ae92e488850ce699f to your computer and use it in GitHub Desktop.
Bash script to Upload folder to S3
# Set AWS credentials and S3 paramters
AWS_KEY=""
AWS_SECRET=""
S3_BUCKET=""
S3_BUCKET_PATH="/"
S3_ACL="x-amz-acl:private"
function s3Upload
{
path=$1
file=$2
acl=${S3_ACL}
bucket=${S3_BUCKET}
bucket_path=${S3_BUCKET_PATH}
date=$(date +"%a, %d %b %Y %T %z")
content_type="application/octet-stream"
sig_string="PUT\n\n$content_type\n$date\n$acl\n/$bucket$bucket_path$file"
signature=$(echo -en "${sig_string}" | openssl sha1 -hmac "${AWS_SECRET}" -binary | base64)
curl -X PUT -T "$path/$file" \
-H "Host: $bucket.s3.amazonaws.com" \
-H "Date: $date" \
-H "Content-Type: $content_type" \
-H "$acl" \
-H "Authorization: AWS ${AWS_KEY}:$signature" \
"https://$bucket.s3.amazonaws.com$bucket_path$file"
}
# set the path based on the first argument
path=$1
# loop through the path and upload the files
for file in "$path"/*; do
s3Upload "$path" "${file##*/}" "/"
done
@pkhetan
Copy link

pkhetan commented Sep 27, 2018

Hi,
I using this solution to upload files to s3 bucket which is managed by rook. But I am not able to push my files. Below is the response I get when I run the script.

< HTTP/1.1 403 Forbidden
< Accept-Ranges: bytes
< Content-Length: 201
< Content-Type: application/xml
< Date: Thu, 27 Sep 2018 19:51:08 GMT
< X-Amz-Request-Id: tx000000000000000001a08-005bad34ac-76c6d0-objectstore
< Connection: close
< Set-Cookie: BIGipServerAutoProvpool_pp_https_auto=950056714.47873.0000; path=/; Httponly; Secure
<

AccessDeniedtx000000000000000001a08-005bad34ac-76c6d0-objectstore76c6d0-objectstore-objectstore* Closing connection 0
  • TLSv1.2 (OUT), TLS alert, Client hello (1):

PFB my script.

Set AWS credentials and S3 paramters

AWS_KEY="abc"
AWS_SECRET="xyzxyzxyz"
S3_BUCKET="teamfelgen"
S3_BUCKET_PATH="/test/"
S3_ACL="x-amz-acl:private"

function s3Upload
{
path=$1
file=$2

acl=${S3_ACL}
bucket=${S3_BUCKET}
bucket_path=${S3_BUCKET_PATH}

date=$(date +"%a, %d %b %Y %T %z")
content_type="application/octet-stream"
sig_string="PUT\n\n$content_type\n$acl\n$date\n/$bucket$bucket_path$file"
echo "$sig_string"
signature=$(echo -en "${sig_string}" | openssl sha1 -hmac "${AWS_SECRET}" -binary | base64)

curl -kv -X PUT -T "$path/$file"
-H "Host: s3.company-rook.com"
-H "Date: $date"
-H "Content-Type: $content_type"
-H $acl
-H "Authorization: AWS ${AWS_KEY}:$signature"
"https://s3.company-rook.com/$bucket$bucket_path$file"
}

set the path based on the first argument

path="C:/project/script/file"

loop through the path and upload the files

for file in "$path"/; do
s3Upload "$path" "${file##
/}" "/"
done

Note: This is working with my own bucket in amazon s3, but not with the specified bucket.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment