Michael Vasilenko mvasilenko

I've been playing with jq, and I've been having a hard time finding examples of how it works with output from a service like AWS (which I use a lot).

Here is one I use a lot with vagrant-ec2.

When we're launching and killing a lot of instances, the AWS API is the only way to track down which instances are live, ready, dead, etc.

To find instances that are tagged with e.g. {"Key" = "Name", "Value" = "Web-00'} in the middle of a vagrant dev cycle, or a prod launch/replace cycle, you can do something like this:

Golang Workers / Job Queue

A running example of the code from:

This gist creates a working example from blog post, and a alternate example using simple worker pool.

TLDR: if you want simple and controlled concurrency use a worker pool.

	{
	"Version": "2012-10-17",
	"Statement": [
	{
	"Action": "sts:AssumeRole",
	"Principal": {
	"Service": "ec2.amazonaws.com"
	},
	"Effect": "Allow",
	"Sid": ""

	"""
	this uses tornado's own IOLoop (i.e. not asyncio's) and it also uses tornado's own
	concurrency functions such as `tornado.gen.coroutine` instead of `async/await` as
	those are native Python3 functions and not compatible with tornado.
	"""

	from tornado.gen import coroutine
	from tornado.httpclient import AsyncHTTPClient
	from tornado.ioloop import IOLoop

	#!/bin/bash


	## THIS IS THE OLD WAY
	## Nowadays, simply follow the Compose installation instructions in the official documentation:
	## https://docs.docker.com/compose/install/


	# get latest docker compose released tag
	COMPOSE_VERSION=$(curl -s https://api.github.com/repos/docker/compose/releases/latest \| grep 'tag_name' \| cut -d\" -f4)

	# Add this snippet to the top of your playbook.
	# It will install python2 if missing (but checks first so no expensive repeated apt updates)
	# [email protected]

	- hosts: all
	gather_facts: False

	tasks:
	- name: install python 2
	raw: test -e /usr/bin/python \|\| (apt -y update && apt install -y python-minimal)

	{
	"AWSEBDockerrunVersion": "1",
	"Image": {
	"Name": "<AWS_ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com/<NAME>:<TAG>",
	"Update": "true"
	},
	"Ports": [
	{
	"ContainerPort": "443"
	}

	input {
	beats {
	port => 9300
	type => beats
	}
	}
	filter {
	grok {
	match => { "message" => "%{TOMCAT_DATESTAMP:DATETIME} \[%{WORD:level}\]%{SPACE}%{GREEDYDATA:message}" }
	overwrite => [ "message" ]

	#!/usr/bin/expect

	set timeout -1;
	spawn {{django_dir}}/venv/bin/python manage.py changepassword {{admin_user}};
	expect {
	"Password:" { exp_send "{{admin_pass}}\r" ; exp_continue }
	"Password (again):" { exp_send "{{admin_pass}}\r" ; exp_continue }
	eof
	}

	user www-data;
	worker_processes 1;
	pid /var/run/nginx.pid;

	events {
	worker_connections 1024;
	}

	http {
	sendfile on;