Skip to content

Instantly share code, notes, and snippets.

View garnaat's full-sized avatar

Mitch Garnaat garnaat

View GitHub Profile
@garnaat
garnaat / bucket_du.py
Created September 1, 2011 13:15
Get total bytes in a bucket
# Iterates over all the keys in a bucket and totals the bytes used
# NOTE: if you have a lot of keys, this could take a LONG time.
import boto
def bucket_du(bucket_name):
s3 = boto.connect_s3()
bucket = s3.lookup(bucket_name)
total_bytes = 0
for key in bucket:
@garnaat
garnaat / eucarc.sh
Created October 13, 2011 12:20
Example of eucarc file
EUCA_KEY_DIR=$(dirname $(readlink -f ${BASH_SOURCE}))
export S3_URL=http://173.205.188.130:8773/services/Walrus
export EC2_URL=http://173.205.188.130:8773/services/Eucalyptus
export EC2_PRIVATE_KEY=${EUCA_KEY_DIR}/euca2-garnaat-e5ec560d-pk.pem
export EC2_CERT=${EUCA_KEY_DIR}/euca2-garnaat-e5ec560d-cert.pem
export EC2_JVM_ARGS=-Djavax.net.ssl.trustStore=${EUCA_KEY_DIR}/jssecacerts
export EUCALYPTUS_CERT=${EUCA_KEY_DIR}/cloud-cert.pem
export EC2_ACCESS_KEY='999999999999999999999999999999999999'
export EC2_SECRET_KEY='00000000000000000000000000000000000000'
# This is a bogus value; Eucalyptus does not need this but client tools do.
@garnaat
garnaat / boto.cfg
Created October 13, 2011 12:48
Example showing Eucalyptus credentials in boto config file
[Credentials]
...
euca_access_key_id=999999999999999999999999999999999999
euca_secret_access_key=00000000000000000000000000000000000000
...
[Boto]
...
eucalyptus_host =173.205.188.130
walrus_host = 173.205.188.130
...
@garnaat
garnaat / gist:1284171
Created October 13, 2011 12:57
Connecting to Eucalyptus server in boto (assuming boto.cfg is configured properly)
$ python
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import boto
>>> euca = boto.connect_euca()
>>> euca.get_all_images()
[Image:emi-DFEA10FF, Image:emi-DF0A1104, Image:emi-0CC81777, Image:emi-DF5F1113, Image:emi-DF0B1106, Image:emi-E9E51553,
...
Image:emi-169215C7, Image:emi-622216BC, Image:emi-E4C11567, Image:emi-B153146A, Image:emi-81BC1332]
@garnaat
garnaat / gist:1443559
Created December 7, 2011 16:52
Using get_all_instance_status method in boto
>>> import boto
>>> ec2 = boto.connect_ec2()
>>> stats = ec2.get_all_instance_status()
>>> stats
[InstanceStatus:i-67c81e0c]
>>> stat = stats[0]
>>> stat
InstanceStatus:i-67c81e0c
>>> stat.id
u'i-67c81e0c'
@garnaat
garnaat / gist:1539859
Created December 30, 2011 13:28
Enable debug output with boto and EC2
# This will cause full debug output to go to the console
>>> import boto
>>> boto.set_stream_logger('foo')
>>> ec2 = boto.connect_ec2(debug=2)
@garnaat
garnaat / gist:1596459
Created January 11, 2012 19:57
Retrieve full Instance info from an InstanceInfo object returned by ELB
import boto
elb = boto.connect_elb()
ec2 = boto.connect_ec2()
load_balancers = elb.get_all_load_balancers()
#
# The InstanceInfo object in the LoadBalancer object contains only a small subset
# of information about the Instance. To get the full set of information, you have
@garnaat
garnaat / tag_instance.py
Created February 6, 2012 11:41
Add a new/value tag to an instance
>>> import boto
>>> c = boto.connect_ec2()
>>> reservations = c.get_all_instances()
>>> reservations
[Reservation:r-b73716d6]
>>> instance = reservations[0].instances[0]
>>> instance
Instance:i-366c4354
>>> instance.tags
{}
@garnaat
garnaat / update_key.py
Created February 10, 2012 17:25
Update the content-type of an existing key in S3 using boto
import boto
s3 = boto.connect_s3()
bucket = s3.lookup('mybucket')
key = bucket.lookup('mykey')
# Copy the key onto itself, preserving the ACL but changing the content-type
key.copy(key.bucket, key.name, preserve_acl=True, metadata={'Content-Type': 'text/plain'})
key = bucket.lookup('mykey')
@garnaat
garnaat / download.py
Created February 23, 2012 22:22
Use multiprocess to download objects from S3
"""
"""
import multiprocessing
import boto
import os
import sys
import datetime
import logging
import Queue