Uploading to S3 in 18 lines of Shell (used to upload builds for http://soltrader.net)

s3.sh

# You don't need Fog in Ruby or some other library to upload to S3 -- shell works perfectly fine
# This is how I upload my new Sol Trader builds (http://soltrader.net)
# Based on a modified script from here: http://tmont.com/blargh/2014/1/uploading-to-s3-in-bash

S3KEY="my aws key"
S3SECRET="my aws secret" # pass these in

function putS3
{
  path=$1
  file=$2
  aws_path=$3
  bucket='my-aws-bucket'
  date=$(date +"%a, %d %b %Y %T %z")
  acl="x-amz-acl:public-read"
  content_type='application/x-compressed-tar'
  string="PUT\n\n$content_type\n$date\n$acl\n/$bucket$aws_path$file"
  signature=$(echo -en "${string}" | openssl sha1 -hmac "${S3SECRET}" -binary | base64)
  curl -X PUT -T "$path/$file" \
    -H "Host: $bucket.s3.amazonaws.com" \
    -H "Date: $date" \
    -H "Content-Type: $content_type" \
    -H "$acl" \
    -H "Authorization: AWS ${S3KEY}:$signature" \
    "https://$bucket.s3.amazonaws.com$aws_path$file"
}

for file in "$path"/*; do
  putS3 "$path" "${file##*/}" "/path/on/s3/to/files/"
done

@miznokruge I guess you are using one of the new regions after January 30, 2014. From http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html

This latest signature version is supported in all regions and any new regions after January 30, 2014 will support only Signature Version 4. For more information, go to Authenticating Requests (AWS Signature Version 4) in the Amazon Simple Storage Service API Reference.

This script using V2 API version, so to use new regions after January 30, 2014 you will need support v4 standard.

How to configure this with wasabi ?

Stoked to come across this. We are using it to upload files from AIX servers, which doesn't support (read: admins won't install) the AWS cli and related stuff. Thanks!

My company is using rook for s3 storage. So I have different base URL for my bucket. "http://abc.rook.com" like this.
so if I create a new bucket with let say "piyush" and put some object inside "test" then my URL becomes like this "http://abc.rook.com/piyush/test/abc.txt". I am able to put object using s3api but not with above curl script.
Please suggest what change I need to do.

Hi chrismdp,

I ran the same script in my environment it is giving out an error.
SignatureDoesNotMatchThe request signature we calculated does not match the signature you provided. Check your key and signing method.AKI*****************LQPUT

Can you please help me with that..
I am also trying to create a download shell script as well if you have any information regarding that do let me know.

Thanks In Advance.
:) 👍

I have the same problem

The request signature we calculated does not match the signature you provided

How to fix that?

For those with root access, just install awscli and be happy doing that in only one line https://aws.amazon.com/getting-started/hands-on/backup-to-s3-cli/

How can I test my connection to S3 bucket

Hi bro ,may you please help with.....

How do I check my connection to S3 is successful or not?