Skip to content

Instantly share code, notes, and snippets.

@amui
Created April 10, 2015 00:50
Show Gist options
  • Save amui/686e3f3feb8d30c19924 to your computer and use it in GitHub Desktop.
Save amui/686e3f3feb8d30c19924 to your computer and use it in GitHub Desktop.
Set cache-control max-age for S3 objects larger than 5GB
from boto.s3.connection import S3Connection
# cache-control metadata
meta = {'Cache-Control':'max-age=43200'}
connection = S3Connection(KEY, SECRET)
bucket = connection.get_bucket('mui.test.bucket')
key = bucket.lookup('sourcefile.tar')
chunk = 99999999
mark = 0
part = 1
try:
up = bucket.initiate_multipart_upload('tmp/result.tar', metadata=meta)
for r in range(0,((key.size)/chunk)+1):
if (mark+chunk)>key.size:
up.copy_part_from_key(bucket.name,key.name,part,mark,key.size-1)
print "processing blocks",mark,"to",key.size,"::part",part
else:
up.copy_part_from_key(bucket.name,key.name,part,mark,mark+chunk)
print "processing blocks",mark,"to",(mark+chunk),"::part",part
mark = mark+chunk+1
part+=1
up.complete_upload()
except RuntimeError as e:
print '[BROKEN]:', e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment