Skip to content

Instantly share code, notes, and snippets.

@mataki
Last active December 18, 2015 13:48
Show Gist options
  • Save mataki/5792311 to your computer and use it in GitHub Desktop.
Save mataki/5792311 to your computer and use it in GitHub Desktop.
AWS の S3 のファイルを別の bucket にコピーする方法 ref: http://qiita.com/mat_aki/items/90205144c3f93758e357
require 'right_aws'
aws_access_key_id, aws_secret_access_key = "key_id", "access_key"
old_bucket_name = 'from-bucket'
new_bucket_name = "to-bucket"
s3 = RightAws::S3.new(aws_access_key_id, aws_secret_access_key)
bucket = s3.bucket(old_bucket_name)
puts "Copy S3 files from '#{old_bucket_name}' to '#{new_bucket_name}'"
total_count = bucket.keys.count
puts "total_count: #{total_count}"
print "Are you sure? (y/N) > "
if gets.strip == 'y'
puts "Start to copy at #{Time.now}"
bucket.keys.each_with_index do |key, i|
print "\r#{i}"
s3.interface.copy(old_bucket_name, key.name, new_bucket_name, key.name)
end
end
require "rubygems"
require 'right_aws'
require "parallel"
aws_access_key_id, aws_secret_access_key = "xxx", "xxx"
new_bucket_email_address = "[email protected]"
old_bucket_name = 'old'
new_bucket_name = "new"
s3 = RightAws::S3.new(aws_access_key_id, aws_secret_access_key)
bucket = RightAws::S3::Bucket.new(s3, old_bucket_name)
puts "Copy S3 files from '#{old_bucket_name}' to '#{new_bucket_name}'"
total_count = bucket.keys.count
puts "total_count: #{total_count}"
print "Are you sure? (y/N) > "
if gets.strip == 'y'
Parallel.each_with_index(bucket.keys, in_processes: 10) do |key, i|
print "\r#{i}: #{key}"
s3.interface.copy(old_bucket_name, key.name, new_bucket_name, key.name, :replace, {'x-amz-grant-full-control' => %!emailAddress="#{new_bucket_email_address}"!, 'x-amz-grant-read' => 'uri=http://acs.amazonaws.com/groups/global/AllUsers'})
end
end
@interu
Copy link

interu commented Jan 16, 2014

アカウントを跨いでコピーする場合は、old_bucketのbucket policyに以下を追加する必要があるよん!

{
"Version": "2008-10-17",
"Id": "Policy1345626759739",
"Statement": [
{
"Sid": "Stmt1345626754496",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<new_bucketのオーナーID12桁>:root"
},
"Action": "s3:",
"Resource": "arn:aws:s3:::<old_bucket_name>/
"
}
]
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment