Python script to auto-tag AWS EBS Snapshots and Volumes using AMI and Instance tags

function.py

import copy
import logging
import os

import boto3

logging.basicConfig(level=os.environ.get('LOG_LEVEL', 'INFO'))

ec2 = boto3.client('ec2')
logger = logging.getLogger(__name__)


def tag_snapshots():
    snapshots = {}
    for response in ec2.get_paginator('describe_snapshots').paginate(OwnerIds=['self']):
        snapshots.update([(snapshot['SnapshotId'], snapshot) for snapshot in response['Snapshots']])

    for image in ec2.describe_images(Owners=['self'])['Images']:
        tags = boto3_tag_list_to_ansible_dict(image.get('Tags', []))
        for device in image['BlockDeviceMappings']:
            if 'SnapshotId' in device['Ebs']:
                snapshot = snapshots[device['Ebs']['SnapshotId']]
                snapshot['Used'] = True
                cur_tags = boto3_tag_list_to_ansible_dict(snapshot.get('Tags', []))
                new_tags = copy.deepcopy(cur_tags)
                new_tags.update(tags)
                new_tags['ImageId'] = image['ImageId']
                new_tags['Name'] += ' ' + device['DeviceName']
                if new_tags != cur_tags:
                    logger.info('{0}: Tags changed to {1}'.format(snapshot['SnapshotId'], new_tags))
                    ec2.create_tags(Resources=[snapshot['SnapshotId']], Tags=ansible_dict_to_boto3_tag_list(new_tags))

    for snapshot in snapshots.values():
        if 'Used' not in snapshot:
            cur_tags = boto3_tag_list_to_ansible_dict(snapshot.get('Tags', []))
            name = cur_tags.get('Name', snapshot['SnapshotId'])
            if not name.startswith('UNUSED'):
                logger.warning('{0} Unused!'.format(snapshot['SnapshotId']))
                cur_tags['Name'] = 'UNUSED ' + name
                ec2.create_tags(Resources=[snapshot['SnapshotId']], Tags=ansible_dict_to_boto3_tag_list(cur_tags))


def tag_volumes():
    volumes = {}
    for response in ec2.get_paginator('describe_volumes').paginate():
        volumes.update([(volume['VolumeId'], volume) for volume in response['Volumes']])

    for response in ec2.get_paginator('describe_instances').paginate():
        for reservation in response['Reservations']:
            for instance in reservation['Instances']:
                tags = boto3_tag_list_to_ansible_dict(instance.get('Tags', []))
                for device in instance['BlockDeviceMappings']:
                    volume = volumes[device['Ebs']['VolumeId']]
                    volume['Used'] = True
                    cur_tags = boto3_tag_list_to_ansible_dict(volume.get('Tags', []))
                    new_tags = copy.deepcopy(cur_tags)
                    new_tags.update(tags)
                    new_tags['Name'] += ' ' + device['DeviceName']
                    if new_tags != cur_tags:
                        logger.info('{0} Tags changed to {1}'.format(volume['VolumeId'], new_tags))
                        ec2.create_tags(Resources=[volume['VolumeId']], Tags=ansible_dict_to_boto3_tag_list(new_tags))

    for volume in volumes.values():
        if 'Used' not in volume:
            cur_tags = boto3_tag_list_to_ansible_dict(volume.get('Tags', []))
            name = cur_tags.get('Name', volume['VolumeId'])
            if not name.startswith('UNUSED'):
                logger.warning('{0} Unused!'.format(volume['VolumeId']))
                cur_tags['Name'] = 'UNUSED ' + name
                ec2.create_tags(Resources=[volume['VolumeId']], Tags=ansible_dict_to_boto3_tag_list(cur_tags))


def tag_everything():
    tag_snapshots()
    tag_volumes()


def boto3_tag_list_to_ansible_dict(tags_list):
    tags_dict = {}
    for tag in tags_list:
        if 'key' in tag and not tag['key'].startswith('aws:'):
            tags_dict[tag['key']] = tag['value']
        elif 'Key' in tag and not tag['Key'].startswith('aws:'):
            tags_dict[tag['Key']] = tag['Value']

    return tags_dict


def ansible_dict_to_boto3_tag_list(tags_dict):
    tags_list = []
    for k, v in tags_dict.items():
        tags_list.append({'Key': k, 'Value': v})

    return tags_list


def handler(event, context):
    tag_everything()


if __name__ == '__main__':
    tag_everything()

Thanks for this - it was very helpful.

It crashed at line 28 because my snaps didn't have Names to start with. I fixed by ignoring any pre-existing name.

Modified to add (1/2) (2/2) etc to each volume/snap so you know how many are attached to a given image/instance. e.g.:

AMI:db5 /dev/sdd (4/4) 
AMI:db5 /dev/sdc (3/4) 
AMI:db5 /dev/sdb (2/4) 
AMI:db5 /dev/sda1 (1/4)

https://gist.github.com/danpritts/1089d878a76b14393478a7476476f97b

Amazing script

Worked better than I could have hoped for, thanks!

Great script. Thanks! Is there a way to modify this so that it tags Snaps created manually or from the Lifecycle Manager, so not just AMIs? As currently any others just get tagged with "UNUSED snap-xxxxxxxx"

Is there a way to modify this so that it tags Snaps created manually or from the Lifecycle Manager, so not just AMIs? As currently any others just get tagged with "UNUSED snap-xxxxxxxx"

In risk of outing myself as a beginner. How is this code ran in AWS? I am not sure how to implement it.

@migdotcom you run it on a host that has AWS credentials available. Could be an EC2 instance with an instance role, could be a dev/ops workstation with a CLI profile holding credentials.

@migdotcom you run it on a host that has AWS credentials available. Could be an EC2 instance with an instance role, could be a dev/ops workstation with a CLI profile holding credentials.

Thanks for answering that as I didn't know that I didn't know that. I was going to use it as a lambda function. I meant more in the side of inputs and env I have a batch of ec2 snapshots. I see boto3_tag_list_to_ansible_dict(tags_list) how do I define this list.

You definitely could use it in a lambda function if you wanted to. This just runs against all the snapshots in an account. You don't have to feed it any tags, it looks them up from hosts and volumes.

Hello Brandon, is there a way to display ec2 tags in two separate columns with key in one column and value in another column. Could you please help me with that?

@harishsd1998 yes but that's a pretty basic task that you could do with awscli or boto3, unrelated to the functionality here?

Actually I used boto3 and printed Ec2 instances and the tags in excel format but tags column contain both key and value in a list format. I want to print that key and value in separate columns? Can you please help me with that?

Hi Brandon- There are a lot of snapshots and volumes that I need to tag, I tried your script using lambda but it didn't work.

Function Logs
START RequestId: 37b948f9-d89d-4782-bb3b-5a29b4197967 Version: $LATEST
[ERROR] Runtime.HandlerNotFound: Handler 'lambda_handler' missing on module 'lambda_function'
Traceback (most recent call last):END RequestId: 37b948f9-d89d-4782-bb3b-5a29b4197967
REPORT RequestId: 37b948f9-d89d-4782-bb3b-5a29b4197967 Duration: 1.24 ms Billed Duration: 2 ms Memory Size: 300 MB Max Memory Used: 85 MB

This is like 4 years old and uses python 2.7; I'm pretty sure it needs updating but I don't do AWS infra any more (I work for SUSE Rancher now) so I'm unlikely to fix it myself. Feel free to fork and modify!