Mimic sqs message json from cloud custodian's c7n-mailer

README.md

sqs-message-json.py

Usage

Create SQS message JSON by feeding in the policy yml, giving the policy name if there are multiple policies, and giving it the location of the output directory so it can read the resources.json file.

python sqs-message-json.py \
  policies/ebs-garbage-collect.yml \
  --policy ebs-mark-unattached-deletion \
  --output-dir out > sqs-ebs-garbage-collect.json

Use SQS message JSON to print a template

c7n-mailer-replay \
  --plain sqs-ebs-garbage-collect.json \
  --config mailer.yml \
  --templates templates/ \
  --template-print

Help

✗ python sqs-message-json.py --help
usage: sqs-message-json.py [-h] [--policy POLICY] --output-dir OUTPUT_DIR
                           [--notify-index NOTIFY_INDEX]
                           policy_file

Convert policy yml and output to an sqs message sent by c7n-mailer

positional arguments:
  policy_file           Policy file to create SQS message for

optional arguments:
  -h, --help            show this help message and exit
  --policy POLICY, -p POLICY
                        Policy name if multiple policies. If none provided,
                        pick the first.
  --output-dir OUTPUT_DIR, -s OUTPUT_DIR
                        [REQUIRED] Directory for policy output
  --notify-index NOTIFY_INDEX, -n NOTIFY_INDEX
                        index of notify action if multiple notifies exist.
                        defaults to 0.

Previously

function convert-sqs-msg {
  input=$(basename $1);
  cat $input | jq -r '.Messages[].Body' | base64 -d > $input.zlib;
  printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" | cat - $input.zlib | gzip -dc | jq -M > $input.clean.json
  rm $input.zlib
}

aws sqs receive-message \
  --queue-url https://sqs.us-west-2.amazonaws.com/1234567890/cloud-custodian > sqs-message.json
convert-sqs-msg sqs-message.json
cat sqs-message.clean.json

Above figured out using davidclin's hode to decode sqs message

sqs-message-json.py

#!/usr/bin/env python
import yaml
import json
import argparse
import os
import uuid
from datetime import datetime


if __name__ == "__main__":
  try:
    region = os.environ['AWS_REGION']
  except KeyError:
    region = 'us-east-1'

  template = {
    "event": None,
    "account_id": "snip",
    "account": "snip",
    "region": region,
    "execution_id": str(uuid.uuid4()),
    "execution_start": datetime.utcnow().timestamp(),
  }

  parser = argparse.ArgumentParser(
    description='Convert policy yml and output to an sqs message sent by c7n-mailer'
  )
  parser.add_argument(
    'policy_file', action="store",
    help='Policy file to create SQS message for')
  parser.add_argument(
    '--policy', '-p', action="store",
    help='Policy name if multiple policies. If none provided, pick the first.')
  # TODO: read from s3 bucket as well
  parser.add_argument(
    '--output-dir', '-s', action="store", required=True,
    help='[REQUIRED] Directory for policy output')
  parser.add_argument(
    '--notify-index', '-n', action="store", type=int, default=0,
    help='index of notify action if multiple notifies exist. defaults to 0.')

  args = parser.parse_args()
  assert args.policy_file

  with open(args.policy_file, "r") as f:
    data = yaml.load(f.read(), Loader=yaml.BaseLoader)
  policies = data['policies']

  # if there is only one policy, select it
  # otherwise choose the one using the name as the unique identifier
  policy_name = None
  if len(data['policies']) == 1:
    policy = data['policies'][0]
    policy_name = policy['name']
  else:
    assert args.policy
    policy_name = args.policy
    for pol in policies:
      if pol['name'] == args.policy:
        policy = pol
        break

  template['policy'] = policy

  action = None
  notify_index = 0
  for act in policy['actions']:
    if act['type'] == 'notify':
      if args.notify_index == notify_index:
        action = act
        break
      notify_index += 1

  template['action'] = action

  resources_path = '{}/{}/resources.json'.format(
    args.output_dir,
    policy_name
  )
  with open(resources_path, "r") as f:
    resources = json.loads(f.read())

  template['resources'] = resources

  print(json.dumps(template, indent=2))