HOWTO: Installing Vault On AWS Linux

This is quick howto for installing vault on AWS Linux, mostly to remind myself. At the end of this tutorial, you'll have a working vault server, using s3 for the backend, self signed certificates for tls, and supervisord to ensure that the vault server is always running, and starts on reboot.

Setting up S3

First things first, let's set up an s3 bucket to use as the storage backend for our s3 instance.

From the AWS Mangement Console, go to the S3 console.
Click on the Create Bucket button
Name it something

IAM Stuff

Next, let's create an IAM Policy with full access to our newly created bucket. We'll also create an IAM Role and IAM User in this step, but this should not be neccessary once Vault v5 is released.

From the AWS Management Console, go the IAM console.
Click on Policies in the sidenav
Click on Create Policy
Select the Policy Generator option, because it's easy.
Select Amazon S3 from the AWS Service dropdown
Select All Actions (*) from the Actions dropdown
Enter the Amazon Resource Name: arn:aws:s3:::<your_bucket_name>
Click Add Statement
Next, repeat steps 5-8, except use the following ARN: arn:aws:s3:::<your_bucket_name>/* (this is required to let vault manage all keys within the bucket)
Click Next Step
Give the policy a name: s3-vault-full-access
Click Add Policy

Next, we create an IAM Role and attach our policy to it. We will use this role as the EC2 instance role later on.

Click on Roles in the side nav
Click Create New Role
Under Select Role Type, select Amazon EC2 from the AWS Service Role section
Attach our newly created s3-vault-full-access policy to the role and click Next Step
Give the role a name: vault-ec2
Lastly, review our role, and click Create Role

Lastly, due to a bug in the current version of vault (v4.1), we create a new user and assign the policy to it. We will then generate access keys for this user to use when initializing vault.

Click on Users from the side nav
Click on Add User
Enter a username: vault
Select Programmatic access from the Select AWS access type section
Click Next: Permissions
Click Attach existing policies directly
Select the s3-vault-full-access policy.
Click Next: Review
Click Create User
Save/download the security credentials on the next screen
Back to the Users screen, and click on our newly created user

Launch EC2

Ok, now it's time to launch an ec2 that will act as our Vault server.

From the AWS Management Console, go to the EC2 console.
Click Launch Instance
Select the most recent Amazon Linux AMI, usually at the top
Select an appropriate size, for this tutorial, I'll use a t2.nano
Click Next: Configure Instance Details
Under IAM role, select the IAM Role we created earlier (s3-vault-full-access)
Click Next: Add Storage
The default storage is fine, so click Next: Add Tags
Give your instance a name tag: vault
Click Next: Configure Security Group
Give your security group a name: vault
Give your security group a description: vault server security group
Click Add Rule
Select Custom TCP Rule and define a port range: 8200
Under source, for the purposes of this tutorial, select My IP. However, in production, you should restrict this port to the security groups of the servers that require access to vault.
Click Review and Launch
Click Launch
If you have an existing key-pair, you can use it, or create a new one and download it
Lastly, click Launch Instance and then View Instances

Option 1. Generating self-signed certificate for vault

Now, we're going to generate a self-signed certificate to use with vault. You can go ahead and skip this step if you already have an ssl certificate to use. Note: these steps were taken from http://www.akadia.com/services/ssh_test_certificate.html

ssh into the ec2 instance

ssh -i <path/to/key.pem> ec2-user@<ec2-dns>

note: if this is a new key, you may receive a permission denied error, in which case, modify the key permissions and try again.

chmod 0700 <path/to/key.pem>

Create a directory to hold the ssl stuff

sudo mkdir -p /etc/vault/ssl
cd /etc/vault/ssl

Generate a private key

openssl req -nodes -new -x509 -keyout server.key -out server.cert -subj '/CN=acme.com/O=ACME INC/C=US/ST=CA/L=Nowhere'

Add the certificate to the certificate manager of any machine that needs to make calls to the vault.

Option 2. Generating a Let's Encrypt certificate (only works with valid domains)

Temporarily enable inbound ports 80 and 443 from Anywhere
Run the following commands

sudo yum update -y
sudo yum install git
sudo git clone https://github.com/certbot/certbot /opt/letsencrypt
cd /etc
sudo mkdir letsencrypt
cd
wget https://dl.eff.org/certbot-auto
chmod a+x certbot-auto
./certbot-auto certonly --standalone --debug -d YOUR_DOMAIN
sudo cp /etc/letsencrypt/live/YOUR_DOMAIN/fullchain.pem /home/ec2-user/.ssl/server.crt
sudo cp /etc/letsencrypt/live/YOUR_DOMAIN/privkey.pem /home/ec2-user/.ssl/server.key
sudo chown ec2-user:ec2-user /home/ec2-user/.ssl/server.*

Remove the inbound rules added in step 1

Installing Vault

Once the instance has finished initializing, it's time to download the Vault binary and unpack it.

update the instance

sudo yum update

install Vault (find the latest binary on the vault project page)

wget https://releases.hashicorp.com/vault/0.8.2/vault_0.8.2_linux_amd64.zip

unzip it

unzip vault_0.8.2_linux_amd64.zip

move the binary

sudo mv vault /usr/local/bin/vault

verify that Vault is ready to go

vault version

create the vault configuration file

touch vault-config.hcl

edit the file

nano vault-config.hcl

define the vault configuration like so.

listener "tcp" {
    address = "0.0.0.0:8200"
    tls_cert_file="/etc/vault/ssl/server.cert"
    tls_key_file="/etc/vault/ssl/server.key"
}

backend "s3" {
    bucket = "<your_bucket_name>"
    region = "us-west-2"
}

disable_mlock=true

exit and save (control+x to exit, y to save, enter to confirm)

Installing supervisord

Next, we install supervisord, which will simplify the whole "let's get Vault running as a service, and have it start on reboot, blah blah blah"

install supervisor

sudo easy_install supervisor

create a new supervisord configuration file

echo_supervisord_conf > supervisord.conf

modify the configuration file
- under [unix_http_server]
  - change ;chmod=0700 to chmod=0766
- change the ;[program:theprogramname] header to [program:vault]
- under [program:vault]
  - change ;command=/bin/cat to command=/usr/local/bin/vault server -config=/home/ec2-user/vault-config.hcl
  - change ;user=chrism to user=ec2-user
  - change ;environment=A="1",B="2" to environment=AWS_ACCESS_KEY_ID="<your_access_key_id",AWS_SECRET_ACCESS_KEY="<your_secret_access_key>", where <your_access_key_id> and <your_secret_access_key> are the credentials you downloaded/wrote down when we created the vault user.

Configuring supervisord

Lastly, we need to configure supervisord to start on init.

create a new init script

sudo touch /etc/init.d/supervisord

edit the file contents:

sudo nano /etc/init.d/supervisord

#!/bin/sh
# Amazon Linux AMI startup script for a supervisor instance
#
# chkconfig: 2345 80 20
# description: Autostarts supervisord.

# Source function library.
. /etc/rc.d/init.d/functions

supervisorctl="/usr/local/bin/supervisorctl"
supervisord="/usr/local/bin/supervisord"
name="supervisor-python"

[ -f $supervisord ] || exit 1
[ -f $supervisorctl ] || exit 1

RETVAL=0

start() {
     echo -n "Starting $name: "
     $supervisord -c /home/ec2-user/supervisord.conf
     RETVAL=$?
     echo
     return $RETVAL
}

stop() {
     echo -n "Stopping $name: "
    $supervisorctl shutdown
     RETVAL=$?
     echo
     return $RETVAL
}

case "$1" in
         start)
             start
             ;;

         stop)
             stop
             ;;

         restart)
             stop
             start
             ;;
esac

exit $REVAL

make the init script executable

sudo chmod +x /etc/init.d/supervisord

add the supervisor init script to chkconfig services

sudo chkconfig --add supervisord

start the supervisord service

sudo service supervisord start
supervisorctl

Initialise the vault

Exit out of the Linux VM

exit

Create a key, require only the one key to unseal the vault.

vault init -key-shares=1 -key-threshold=1

walterheck/howto-installing-vault-on-aws-linux.md