COPY FROM s3 to s3 DIFFERENT ACCOUNTS

gistfile1.txt

from https://blogs.tensult.com/2018/08/06/copy-s3-bucket-objects-across-aws-accounts/

In this blog, I am going to explain about, how to transfer S3 objects from one AWS account to another.

Prerequisites
Two AWS accounts(One for source S3 bucket and another for destination S3 bucket)
Create an IAM user in destination AWS account (see this doc to create IAM user for AWS account).
Configure AWS CLI in local machine with previously created IAM user credentials (see this doc to configure AWS CLI).
Step 1: Get The 12 Digit Destination AWS Account Number
Sign in to destination AWS account. Go to Support → Support center and copy account number from there.

Step 2: Setup Source S3 Bucket
Sign in to source AWS account. Create a bucket in S3(To create bucket, follow this doc). Attach the following policy to the bucket(To attach bucket policy, follow this doc). Upload some test files which are meant to be copied automatically to the destination bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DelegateS3Access",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::DESTINATION_BUCKET_ACCOUNT_NUMBER:root"
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::SOURCE_BUCKET_NAME/*",
                "arn:aws:s3:::SOURCE_BUCKET_NAME"
            ]
        }
    ]
}
view rawcopy-s3-bucket-objects-across-accounts-source-bucket-policy.json hosted with ❤ by GitHub
Step 3: Setup Destination S3 Bucket
Sign in to destination AWS account. Create a bucket in S3(To create bucket, follow this doc).

Step 4: Attach Policy To IAM User In Destination AWS Account
Attach the following policy to the IAM user created previously in the destination AWS account (see this doc to add policy to IAM user).

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::SOURCE_BUCKET_NAME",
                "arn:aws:s3:::SOURCE_BUCKET_NAME/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::DESTINATION_BUCKET_NAME",
                "arn:aws:s3:::DESTINATION_BUCKET_NAME/*"
            ]
        }
    ]
}
view rawcopy-s3-bucket-objects-across-accounts-iam-user-policy.json hosted with ❤ by GitHub
Step 5: Sync S3 Objects To Destination
If above steps are completed, we can copy S3 bucket objects from source account to destination account by using the following AWS CLI command.

aws s3 sync s3://SOURCE-BUCKET-NAME s3://DESTINATION-BUCKET-NAME --source-region SOURCE-REGION-NAME --region DESTINATION-REGION-NAME
The above command should be executed with destination AWS IAM user account credentials only otherwise the copied objects in destination S3 bucket will still have the source account permissions and won’t be accessible by destination account users.

Conclusion
Now we have learnt how to copy S3 bucket objects from one AWS account to another. Try this approach and let me know if you have faced any issues, also if you have any other way to do this task, let me know.

aws s3 sync s3://videostdbwatch s3://originalvideobucket --source-region us-east-1 --region us-east-1