curl get file from private s3 with iam role

get_s3_file.sh

#!/bin/bash

instance_profile=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/`
aws_access_key_id=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile} | grep AccessKeyId | cut -d':' -f2 | sed 's/[^0-9A-Z]*//g'`
aws_secret_access_key=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile} | grep SecretAccessKey | cut -d':' -f2 | sed 's/[^0-9A-Za-z/+=]*//g'`
token=`curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile} | sed -n '/Token/{p;}' | cut -f4 -d'"'`

file="somefile.deb"
bucket="some-bucket-of-mine"
date="`date +'%a, %d %b %Y %H:%M:%S %z'`"
resource="/${bucket}/${file}"
signature_string="GET\n\n\n${date}\nx-amz-security-token:${token}\n/${resource}"
signature=`/bin/echo -en "${signature_string}" | openssl sha1 -hmac ${aws_secret_access_key} -binary | base64`
authorization="AWS ${aws_access_key_id}:${signature}"

curl -s -H "Date: ${date}" -H "X-AMZ-Security-Token: ${token}" -H "Authorization: ${authorization}" "https://s3-us-west-1.amazonaws.com/${resource}" -o "output.deb"

policy-for-bucket

{
	"Version": "2012-10-17",
	"Id": "Policy1441048292119",
	"Statement": [
		{
			"Sid": "Stmt1441048289544",
			"Effect": "Allow",
			"Principal": {
				"AWS": [
					"arn:aws:iam::123456789012:role/your_role_name",
				]
			},
			"Action": "s3:*",
			"Resource": "arn:aws:s3:::some-bucket-of-mine/*"
		}
	]
}

policy-for-iam

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::some-bucket-of-mine",
                "arn:aws:s3:::some-bucket-of-mine/*"
            ]
        },
    ]
}

How to perform the same in an ECS Container with a role attached ? Thanks

this seems to work https://gist.github.com/afunsten/fcabcfe3a77a4958af9bcafba2dfbbb5

Thank you for this! Have you happen to updated this to work with V4?

Hello, this work 99% of the time, however, occationally I got this:

aws authentication requires a valid date or x-amz-date header

No idea where that coming from, anyone else got that ?

P/s: OMG, one machine doesn't have English as main language LC_ALL=en_EN.utf8 date +'%a, %d %b %Y %H:%M:%S %z', Merry Christmas everyone!