AWS S3 guide. keywords: aws s3

aws_s3.md

AWS S3 - Simple Storage Service

Table of contents

• Basic CLI commands
• Permissions
  • Cross-account access
• Hosting a public website
  • Basic setup
  • Changing the URL to use the bucket name in the prefix
  • Adding SSL and boosting performance with AWS CloudFront
  • HTTP redirection
• Gotchas
  • Prefix design is freaking important

Basic CLI commands

Command	Description
aws s3 ls	Lists all buckets.
aws s3 sync s3://mybucket .(1)	Downloads all the content of a bucket in the current directory ..
aws s3 sync ./myFolder s3://mybucket/myBackup(1)	Uploads ./myFolder to s3://mybucket/myBackup.
aws s3 mb s3://your-universally-unique-bucket-name	Creates a new bucket.
aws s3 cp your-local-file.txt s3://your-bucket-name	Uploads file to a bucket.
aws s3 ls s3://your-universally-unique-bucket-name --recursive --human-readable --summarize	List all files in the bucket.

(1) aws s3 sync can only be used to sync folders.

Permissions

Cross-account access

• Open the S3 service.
• Select your bucket and click on the Permissions tab.
• Click the Edit button under the Bucket policy section.

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "Statement1",
			"Effect": "Allow",
			"Principal": {
				"AWS": [
					"arn:aws:iam::1234:root",
					"arn:aws:iam::4567:root"
				]
			},
			"Action": "s3:*",
			"Resource": [
				"arn:aws:s3:::your-bucket",
				"arn:aws:s3:::your-bucket/*"
			]
		}
	]
}

Hosting a public website

WARNING: Make sure that all folders and files have a URL friendly name, i.e., lowercase with allowed URL characters.

Basic setup

Make the bucket public. Under the Permissions tab

• Set the Block public access section to off.
• Configure the Bucket policy as follow:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "PublicReadGetObject",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::YOUR-BUCKET-NAME/*"
    }
  ]
}

When that's done, each file is publicly available. Select the file and you should see their associated URL.

Changing the URL to use the bucket name in the prefix

By default, each file should be accessible at a URL similar to the following:

https://s3-ap-southeast-2.amazonaws.com/your-bucket-name/hello.html

If you wish to get something similar to http://your-bucket-name.s3-ap-southeast-2.amazonaws.com/hello.html instead, you must enable the Static website hosting option on your bucket:

• Select the Properties tab.
• Enable the Static website hosting option.

However, this option does not support SSL. To add SSL back, read the next section

Adding SSL and boosting performance with AWS CloudFront

Please refer to the CloudFront for S3 Static Website section in the AWS CLOUDFRONT GUIDE.

HTTP redirection

Please refer to the Advanced redirect scenarios with S3 section of the AWS ROUTE 53 document.

Gotchas

Prefix design is freaking important

This is a typical S3 newbies mistake. The newbie thinks of S3 as a file system. In reality S3 is more like a key/value pair datatstore. These are two very different mindsets:

• File system mindset: I can organize my files however I want. I'll use the must intuitive file structure to navigate through my files.
• Key/value pair mindset: Which key should I use to efficiently retrieve or list my files.

The file system mindset will inevitably creates issue when the user must list or search objects in S3. Indeed, because S3 does not support regex in prefix (i.e., the start of a file's path), you must design your bucket's path with care. Think about the use cases to list or retrieve your objects so that you can design your prefixes accordingly.