Daniel Maher phrawzty

S3 as Yum repo

There are two parts to this:

Managing access to non-public S3 resources.
Building RPM repositories in an automated, deterministic way that Yum can use.

Environment

In general, a CentOS 7 x86_64 box in AWS EC2; in specific, this Packer profile.

Roles and Types

There are a number of "roles" in the Socorro infra. The resource profiles for these roles are not identical, ergo there will be different AWS instance types for different roles.

The roles are:

Admin
Collectors
Crash-Analysis
Elasticsearch
Middleware

Managing multiple user accounts within the cloud-based Socorro infrastructure is a fool's errand; instead, the plan is use a single login (role acccount) with multiple accepted SSH keys (one per user). These keys are managed from the Source of Truth and implanted during the node provisioning step.

In order to keep track of things, however, it will be helpful to tag the public SSH keys with an identifier of the user that possesses the associated private key. Normally this is what the "comment" field is for:

ssh-rsa <big_ol_key> [comment]

The issue here is that the "comment" section isn't exported, announced, or otherwise relevent at all from a system perspective. Instead, I propose adding a small environment variable that does the job:

environment="SSH_KEY=happyuser" ssh-rsa <big_ol_key> [comment]

What it is

Problem: Terraform doesn't play nicely with pre-existing infrastructure.

Solution: Officially there isn't one - but here's a work-around that does the trick.

Summary

Declare a new, temporary resource in your Terraform plan that is nearly identical to the extant resource.
Apply the plan, thus instantiating the temporary "twinned" resource and building a state file.

fs test

TEST: Mount disparate remote storage devices as if they were a single directory.

Environment

Ubuntu 14.04 "Daily cloud image" as obtained on 2014-07-31

https://cloud-images.ubuntu.com/vagrant/trusty/current/trusty-server-cloudimg-amd64-vagrant-disk1.box

SSHfs test

Keybase proof

I hereby claim:

I am phrawzty on github.
I am phrawzty (https://keybase.io/phrawzty) on keybase.
I have a public key whose fingerprint is 1204 786B D9B7 7FA0 618B 82A1 016F 9A65 192B FE03

To claim this, I am signing this object:

	(socorro-virtualenv)[vagrant@localhost ~]$ cat /etc/centos-release
	CentOS release 6.4 (Final)


	(socorro-virtualenv)[vagrant@localhost ~]$ sudo yum install centos-release-SCL
	[...]
	================================================================================
	Package Arch Version Repository Size
	================================================================================
	Installing:

	#!/usr/bin/env bash

	function techo {
	STAMP=`date '+%b %d %H:%M:%S'`
	echo "${STAMP} BOOTSTRAP: ${@}"
	}

	techo "start"
	techo "install puppet yum repo"
	rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm

	#!/usr/bin/env python

	import hashlib
	import logging

	import boto
	import config
	import happybase

	logger = logging.getLogger(__name__)

	Duplicati.CommandLine.exe backup \
	--passphrase=$PASSPHRASE \
	--aws_access_key_id=$AWS_KEY \
	--aws_secret_access_key=$AWS_SECRET \
	--s3-location-constraint=$LOC \
	--aes-encryption-dont-allow-fallback=true \
	$DIR \
	s3://$S3_BUCKET/$DIR

	# Obviously the S3 DIR target can be tweaked; this is just a simple example.