AJ Kerrigan ajkerrigan

Generate and record boto3 errors

The testing for developers section of the Cloud Custodian documentation walks through recording and playing back API responses for use in testing. Some cases can be tricky to reproduce in isolation though, because they occur due to concurrency, timing or caching issues that are difficult to replicate.

If you know that a particular error is possible but it's complex to reproduce inside a test, one alternative is forcing the error outside the Custodian test suite. You can record the error response in a way that simulates Custodian's own placebo-based flight recording, and then use

c7n-local

Update: As of boto3 1.28 all of these shenanigans are no longer necessary. I'm keeping this up as a reference but there is simpler guidance available here now.

Big thanks to the folks who got this changed upstream. Relevant reading for those interested: boto/boto3#2099 aws/aws-sdk#229 (and many other related issues/PRs!)

This is an unsupported experimental wrapper for running pull or event-based Cloud Custodian policies

De-Duplicate Redash Dashboard Widgets

Overview

When updating Redash dashboards, it's possible to add widgets rather than replacing them. That leads to a number of duplicate widgets that have the same visualization type and query details. This is a script built on top of redash-toolbelt that removes duplicate widgets, leaving only the most recently updated widget per set.

Usage

Install redash-toolbelt.
(Optional) Set the REDASH_URL and REDASH_API_KEY environment variables.

VisiData Plugin Discovery / Automatic Import

This is an experimental Python snippet that can be used (such as from .visidatarc) to discover VisiData plugins based on entry points. The approach is inspired by the Pluggy framework, and specifically the entry point group naming that flake8 uses for plugin registration.

This means that you could pip-install a plugin package and VisiData would find and import it at startup. I've tested this by packaging up vds3 with poetry and using an entry point to [map the

	# sample poetry plugin section (which equates to setup.py entry point)
	# extracted from https://github.com/ajkerrigan/visidata-plugins/pull/17/files#diff-4335f179817b92f616b8895564efdcb417ef24644176089a2a4be9d213768c2fR20-R21
	# map a plugin name to the importable module that implements it

	[tool.poetry.plugins."visidata.plugin"]
	"vds3" = "visidata_s3"

	import contextlib
	from io import StringIO

	import visidata
	from visidata import vd


	class VDStatusIO(StringIO):
	def write(self, s):
	'''Forward writes to the VisiData message log'''

	"""
	First pass at extracting a mapping of CCM 3.0.1 controls to their associated SOC 2
	control IDs.

	Dependencies: openpyxl pandas
	"""
	import json
	from itertools import islice

	import click

	MIT License

	Copyright (c) 2021 AJ Kerrigan

	Permission is hereby granted, free of charge, to any person obtaining a copy
	of this software and associated documentation files (the "Software"), to deal
	in the Software without restriction, including without limitation the rights
	to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
	copies of the Software, and to permit persons to whom the Software is
	furnished to do so, subject to the following conditions:

	from visidata import JsonSheet, PandasSheet, VisiData

	@VisiData.api
	def open_tfstate(p):
	return JsonSheet(p.name, source=p)

	@VisiData.api
	def open_parquet(p):
	return PandasSheet(p.name, source=p)

	#!/bin/bash
	#
	# Mush mosh and SSM sessions together.
	#
	# Run mosh-server on an EC2 instance as part of an AWS Session Manager SSH
	# session, and feed the output into mosh-client locally. This is helpful
	# if you have an EC2 instance which:
	#
	# * Is publicly addressable
	# * Has a predictable Name tag but a shifting IP