Skip to content

Instantly share code, notes, and snippets.

@msimonin
Created August 27, 2014 11:50
Show Gist options
  • Select an option

  • Save msimonin/5303d28580a09ee318f1 to your computer and use it in GitHub Desktop.

Select an option

Save msimonin/5303d28580a09ee318f1 to your computer and use it in GitHub Desktop.
After checkpoint script
#!/usr/bin/env python
import boto
import boto.s3.connection
import sys
import os
access_key = 'ZO9OYE6J48X3LPP5JIWZ'
secret_key = 'QlyDGOBkjHa+AmKauCswrEs3kltF7XkiZCNVZG+Y'
conn = boto.connect_s3(
aws_access_key_id = access_key,
aws_secret_access_key = secret_key,
host = '192.168.0.2',
is_secure=False, # uncommmnt if you are not using ssl
calling_format = boto.s3.connection.OrdinaryCallingFormat(),
)
if (len(sys.argv) != 2):
sys.exit(1)
ckpt_dir = sys.argv[1]
print "from python : {}".format(ckpt_dir)
# we assume that path is /.../cid/chkptid
bucket_names = ckpt_dir.split("/")
# create the cid bucket
root = conn.create_bucket(bucket_names[-2])
sub_bucket = bucket_names[-1]
# put files in subbuckets
for f in os.listdir(ckpt_dir):
key = root.new_key(sub_bucket + "/" + f)
key.set_contents_from_filename(ckpt_dir + "/" + f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment