Skip to content

Instantly share code, notes, and snippets.

@amatthies
amatthies / boto3_s3_bucket_folders.py
Created June 29, 2015 21:46
List the top folders of a s3 bucket using boto3. https://github.com/boto/boto3/issues/134
import boto3
client = boto3.client('s3')
paginator = client.get_paginator('list_objects')
for result in paginator.paginate(Bucket='edsu-test-bucket', Delimiter='/'):
for prefix in result.get('CommonPrefixes'):
print(prefix.get('Prefix'))
import boto3
from botocore.handlers import disable_signing
resource = boto3.resource('s3')
resource.meta.client.meta.events.register(
'choose-signer.s3.*', disable_signing)
import json
import logging
import logging.config
with open('python-logging.json', 'r') as f:
logging.config.dictConfig(json.load(f))
logger = logging.getLogger(__name__)
logger.info("Logging config loaded from json")
logger.debug("This is a DEBUG message on the logger.")
@amatthies
amatthies / boto3_profiles.md
Last active September 25, 2018 02:26
Handling multiple aws profiles and roles on my local machine

Handling multiple aws profiles and roles on my local machine

I have pretty some aws profiles. And each of those profiles can assume pretty some IAM roles. I write code for instance roles: I trip, when I see aws_access_key_id = in code. ("No! No! No!")

When I started writing aws stuff, I added profile= arguments to all my boto3 constructors... until the roles started to be crucial. In fact, on several AWS accounts I can be several users, who can assume several roles.

Long story short: meanwhile, I handle all profile and (most of the) region stuff locally before coding, not in the code. For an EC2 instance, the needed credentials are "just there" – I want the same on my local machine.

My ~/.aws folder contains two files, credentials and config: