Deploying Deis on AWS

These instructions will get you up and running with Deis and CoreOS in a AWS VPC. There already exists a CloudFormation script to get up and running in AWS. But if you want to get down and dirty, this document will help walk you through the steps.

I tried to build these instructions for both the AWS web console and equivilent AWS CLI. If you find errors, please feel free to comment so I can update them. You can also find me in the #deis IRC channel on Freenode if you have questions.

AWS

Since we will be running CoreOS and Deis inside a VPC we need some AWS setup first. We need a single subnet VPC, some security groups, and instances of CoreOS.

VPC

Use the Start VPC Wizard to create a VPC with single subnet. You can use the default settings. In our examples, we have a IP CIDR block of 172.8.0.0/16 and a public subnet of 172.8.96.0/24.

Using the CLI (be sure to replace the example VPC ID vpc-a01106c2 with the one output from the previous create-vpc command):

$ aws ec2 create-vpc --cidr-block 172.8.0.0/16
VPC	172.8.0.0/16	dopt-674f8b0e	default	pending	vpc-a01106c2
$ aws ec2 create-subnet --vpc-id vpc-a01106c2 --cidr-block 172.8.96.0/24
SUBNET	us-west-2a	251	172.8.96.0/24	pending	subnet-8e8c97fa	vpc-a01106c2

Security Groups

Create security groups for remote SSH access and the Deis router services:

Inbound port 22 for your remote SSH management (this can be limited by IP address)
Inbound port 2222, 80, and 443 TCP for your remote git push SSH and Deis router (this will get assigned to the load balancer)
Inbound all traffic for the current security group (the default group is fine, this allows each CoreOS instance to talk to each other on all ports)

Using the CLI (replacing sg-fdc16c98, sg-a7cd354e and 203.0.113.0/24 for the authorize-security-group-ingress commands):

$ aws ec2 create-security-group --group-name ssh --description "SSH management for CoreOS instances" --vpc-id vpc-a01106c2
sg-fdc16c98	true
$ aws ec2 create-security-group --group-name deislb --description "Deis application services" --vpc-id vpc-a01106c2
sg-a7cd354e	true
$ aws ec2 authorize-security-group-ingress --group-id sg-fdc16c98 --protocol tcp --port 22 --cidr 203.0.113.0/24
true
$ aws ec2 authorize-security-group-ingress --group-id sg-a7cd354e --protocol tcp --port 2222 --cidr 0.0.0.0/0
true
$ aws ec2 authorize-security-group-ingress --group-id sg-a7cd354e --protocol tcp --port 80 --cidr 0.0.0.0/0
true
$ aws ec2 authorize-security-group-ingress --group-id sg-a7cd354e --protocol tcp --port 443 --cidr 0.0.0.0/0
true
$ aws ec2 describe-security-groups --filters Name=vpc-id,Values=vpc-bc9e6bd9
SECURITYGROUPS	SSH management for CoreOS instances	sg-ffdb7f9a	ssh	847672749925	vpc-bc9e6bd9
IPPERMISSIONS	22	tcp	22
IPRANGES	203.0.113.0/24
IPPERMISSIONSEGRESS	-1
IPRANGES	0.0.0.0/0
SECURITYGROUPS	Deis application services	sg-a7cd354e	deislb	847672749925	vpc-bc9e6bd9
IPPERMISSIONS	2222	tcp	22222
IPRANGES	0.0.0.0/0
IPPERMISSIONS	80	tcp	80
IPRANGES	0.0.0.0/0
IPPERMISSIONS	443	tcp	443
IPRANGES	0.0.0.0/0
IPPERMISSIONSEGRESS	-1
IPRANGES	0.0.0.0/0
SECURITYGROUPS	default VPC security group	sg-b4d97dd1	default	847672749925	vpc-bc9e6bd9
IPPERMISSIONS	-1
USERIDGROUPPAIRS	sg-b4d97dd1	847672749925
IPPERMISSIONSEGRESS	-1
IPRANGES	0.0.0.0/0

Instances

You need to start up at least 3 insances of the current alpha CoreOS image.

Image: Search for CoreOS-alpha-338.0.0 to get the AMI for your region.
Instance Type: This should be at least m3.large so there is enough memory per node.
Instance Details:
Start an odd number of instances between 3 and 13.
Check to automatically assign public IP addresses.
Be sure to set the network to your newly created VPN.
Expand the Advanced section and fill out user-data, replacing <token> with your own generated from http://discovery.etcd.io/new (do not remove the #cloud-config comment, it is required):

#cloud-config

coreos:
  etcd:
    discovery: https://discovery.etcd.io/<token>
    addr: $private_ipv4:4001
    peer-addr: $private_ipv4:7001
  units:
    - name: etcd.service
      command: start
    - name: fleet.service
      command: start

Storage: Your main EBS root can be default 8 GB. But you will need to mount another EBS volume at /var/lib/docker that gives you plenty of space for image storage (16 GB is a good start).
Security Groups: Assign the 2 previously created security groups (the default and remote SSH) to the new instances.

We can create these instances using the AWS CLI. First save your cloud config file from above as cloudinit. Then using the AWS CLI, replacing ami-a7d1a197, VolumeSize, security-group-ids, subnet-id, and other options with the correct ones for your environment:

$ aws ec2 run-instances --image-id ami-a7d1a197 --block-device-mappings '[{"DeviceName": "/dev/sda","Ebs": {"VolumeSize": 32}}]' --security-group-ids sg-b4d97dd1 sg-a7cd354e --associate-public-ip-address --count 3 --instance-type m3.large --subnet-id subnet-de170faa --key-name deis --user-data file://cloudinit
847672749925	r-9165bf99
INSTANCES	0	x86_64	None	False	xen	ami-a7d1a197	i-096ed701	m3.large	aki-f837bac8	andy-aws	2014-06-06T02:27:54.000Z	ip-172-8-96-142.us-west-2.compute.internal	172.8.96.142	None	/dev/sda1	ebs	True	None	subnet-de170faa	paravirtual	vpc-bc9e6bd9
MONITORING	disabled
NETWORKINTERFACES	None	eni-98d45bef	847672749925	ip-172-8-96-142.us-west-2.compute.internal	172.8.96.142	True	in-use	subnet-de170faa	vpc-bc9e6bd9
ATTACHMENT	2014-06-06T02:27:54.000Z	eni-attach-a22a2494	True	0	attaching
GROUPS	sg-fdc16c98	sg-b4d97dd1	default
PRIVATEIPADDRESSES	True	ip-172-8-96-142.us-west-2.compute.internal	172.8.96.142
PLACEMENT	us-west-2a	None	default
SECURITYGROUPS	sg-fdc16c98	sg-b4d97dd1	default
STATE	0	pending
STATEREASON	pending	pending
INSTANCES	1	x86_64	None	False	xen	ami-a7d1a197	i-0a6ed702	m3.large	aki-f837bac8	andy-aws	2014-06-06T02:27:54.000Z	ip-172-8-96-143.us-west-2.compute.internal	172.8.96.143	None	/dev/sda1	ebs	True	None	subnet-de170faa	paravirtual	vpc-bc9e6bd9
MONITORING	disabled
NETWORKINTERFACES	None	eni-99d45bee	847672749925	ip-172-8-96-143.us-west-2.compute.internal	172.8.96.143	True	in-use	subnet-de170faa	vpc-bc9e6bd9
ATTACHMENT	2014-06-06T02:27:54.000Z	eni-attach-a32a2495	True	0	attaching
GROUPS	sg-fdc16c98	sg-b4d97dd1	default
PRIVATEIPADDRESSES	True	ip-172-8-96-143.us-west-2.compute.internal	172.8.96.143
PLACEMENT	us-west-2a	None	default
SECURITYGROUPS	sg-fdc16c98	sg-b4d97dd1	default
STATE	0	pending
STATEREASON	pending	pending
INSTANCES	2	x86_64	None	False	xen	ami-a7d1a197	i-0b6ed703	m3.large	aki-f837bac8	andy-aws	2014-06-06T02:27:54.000Z	ip-172-8-96-144.us-west-2.compute.internal	172.8.96.144	None	/dev/sda1	ebs	True	None	subnet-de170faa	paravirtual	vpc-bc9e6bd9
MONITORING	disabled
NETWORKINTERFACES	None	eni-9ad45bed	847672749925	ip-172-8-96-144.us-west-2.compute.internal	172.8.96.144	True	in-use	subnet-de170faa	vpc-bc9e6bd9
ATTACHMENT	2014-06-06T02:27:54.000Z	eni-attach-9f2a24a9	True	0	attaching
GROUPS	sg-fdc16c98	sg-b4d97dd1	default
PRIVATEIPADDRESSES	True	ip-172-8-96-144.us-west-2.compute.internal	172.8.96.144
PLACEMENT	us-west-2a	None	default
SECURITYGROUPS	sg-fdc16c98	sg-b4d97dd1	default
STATE	0	pending
STATEREASON	pending	pending

Load Balancer

This load balancer should be have the following listeners:

TCP 80 to TCP 80
TCP (Secure) 443 to TCP 80 (you can omit this if you are not needing SSL)
TCP 2222 to TCP 2222

Add all EC2 instances to this load balancer. These requests get sent to the Deis routers, which in turn route traffic for the Deis controller, Deis builder, and applications running on Deos. We will later add DNS to this load balancer for the deis.mydomain.com and wildcard *.deis.mydomain.com A records.

Using the CLI:

$ aws elb create-load-balancer --load-balancer-name deis --listeners Protocol=TCP,LoadBalancerPort=2222,InstanceProtocol=TCP,InstancePort=2222 Protocol=TCP,LoadBalancerPort=80,InstanceProtocol=TCP,InstancePort=80 --subnets subnet-8e8c97fa
deis-553741316.us-west-2.elb.amazonaws.com

DNS

Create two new DNS records. If you are using AWS Route53, these should be aliases to the load balancer instances instead of CNAMEs.

deis.mydomain.com which will point to the Deis Control load balancer
*.deis.mydomain.com which will point to the Deis Applications load balancer

Deis

Install

Clone the lastest version of Deis:

git clone https://github.com/deis/deis.git deis`

Then install the CLI:

cd deis/client
sudo python setup.py install
cd ..

Setup

The FLEETCTL_TUNNEL environment variable provides a gateway to use in the datacenter to one of the CoreOS hosts:

export FLEETCTL_TUNNEL=ec2-55-55-55-55.us-west-2.compute.amazonaws.com

Now we can bootstrap the Deis containers. DEIS_NUM_INSTANCES should match the number of EC2 instances launched. DEIS_NUM_ROUTERS should be 3 or more Deis application load balancer routers to run:

DEIS_NUM_INSTANCES=3
DEIS_NUM_ROUTERS=3
make run

Then register the admin user (the first user registered is an admin):

$ deis register http://deis.mydomain.com

You can now login to Deis:

$ deis login http://deis.mydomain.com

add your SSH key for git operations:

$ deis keys:add
Found the following SSH public keys:
1) id_rsa.pub andy
Which would you like to use with Deis? 1
Uploading andy to Deis...done

Create a new cluster named deis to run applications under:

$ deis clusters:create deis deis.mydomain.com --hosts deis.mydomain.com --auth ~/.ssh/id_rsa

Applications

Creating an application requires that application be housed under git already. Navigate to the application root and then:

$ deis create myapp --cluster deis
Creating application... done, created myapp
Git remote deis added

$ git push deis master

Your application will now be built and run inside the Deis cluster! After the application is pushed it should be running at http://myapp.deis.mydomain.com:

$ deis apps:info

andyshinn/DEISAWS.md