Deploying to Elastic Beanstalk via CircleCi 2.0

circleci-2.0-eb-deployment.md

Deploying to Elastic Beanstalk via CircleCi 2.0

I got to here after spending hours trying to deploy to an Elastic Beanstalk instance via CircleCi 2.0 so I thought I'd write up what worked for me to hopefully help others. Shout out to RobertoSchneiders who's steps for getting it to work with CircleCi 1.0 were my starting point.

For the record, I'm not the most server-savvy of developers so there may be a better way of doing this.

Setup a user on AWS IAM to use for deployments

• Add user here
• Set a username and select Programmatic access as the Access type
• Click 'Create group' on the user permissions page
• Set a group name and search for the AdministratorAccess-AWSElasticBeanstalk and AmazonS3FullAccess policies and select them
  • Note: This used to be just AWSElasticBeanstalkFullAccess, but that's since been deprecated
• Create the group so it's assigned to your new user
• Review and create the user

Setup an Elastic Beanstalk application

• 'Create New Application'
• 'Create New Environment'
• 'Create web server'
• On the Environment Information page where it asks you to set the 'Environment name', set it to something with the Git branch name in e.g. BRANCHNAME-my-application
  • I do this as I have a staging branch and the master branch so in our EB config, we'll be replacing BRANCHNAME with the $CIRCLE_BRANCH environment variable provided by CircleCi so when deploying the staging branch for example, it will know to deploy to the staging-my-application environment on Elastic Beanstalk
• Follow the rest of the setup as you require

Add deployment user environment variables to CircleCi

• Project Settings > Environment Variables
  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY

Add .elasticbeanstalk/config.yml config to application code

• Update the values in the below snippet to match what you setup.

branch-defaults:
  master:
    environment: $CIRCLE_BRANCH-my-application
global:
  application_name: My Application
  default_ec2_keyname: XXXXXX
  default_platform: 64bit Amazon Linux 2017.09 v2.6.2 running Ruby 2.4 (Passenger Standalone)
  default_region: eu-west-1
  sc: git

Note: Ensure the application_name is exactly what you called your application in Elastic Beanstalk when you did the 'Create New Application' step.

Also Note: Do not set a profile: value here, the profile will be set based on the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables you setup.

Update your .circleci/config.yml

• The following is what I think is the bare minimum I needed to get only the master or staging branches on your Git repo to deploy.
• Update $CIRCLE_BRANCH-my-application to the environment name you set in Elastic Beanstalk.

version: 2
jobs:
  deploy:
    working_directory: ~/app
    docker:
      - image: circleci/ruby:2.4.3
    steps:
      - checkout

      - run:
          name: Installing deployment dependencies
          working_directory: /
          command: |
            sudo apt-get -y -qq update
            sudo apt-get install python3-pip python3-dev build-essential
            sudo pip3 install awsebcli

      - run:
          name: Deploying
          command: eb deploy $CIRCLE_BRANCH-my-application

workflows:
  version: 2
  build:
    jobs:
      - deploy:
          filters:
            branches:
              only:
                - staging
                - master

Cross fingers

• Commit and let CircleCi do it's thing. If all goes well you should see it updating on the Elastic Beanstalk dashboard as the 'Deploying' step is running in CircleCi.

 Complete output from command python setup.py egg_info:
    error in awsebcli setup command: Invalid environment marker: sys_platform == "win32" and python_version >= "3.6"

For anyone reading, I fixed this error by simply fixing the awsebcli to a known stable version.

      - run:
          name: Install python, pip, and the AWS EB CLI
          command: |
            sudo apt-get install python-pip python-dev
            sudo pip install awsebcli==3.14.11

Great, I can get the eb cli installed but still can't run eb deploy! I can run eb deploy from my local machine, and my .elasticbeanstalk/config.yml file is committed to the repo that CircleCi is checking out. Any idea what is going on, @ryansimms?

@upugo-dev what error do you get when the deploy command runs?

@ryansimms thanks so much for the reply. Sorry I wasn't very clear when I wrote my previous message; I was receiving the

ERROR: This directory has not been set up with the EB CLI
You must first run "eb init".

Which I have now fixed... drumroll please... I'd removed checkout from my circleci .config file, eb init was looking in a completely empty directory! I have now added the checkout line back in and it is working fine deployment-wise, but my app isn't running; I'm getting

┌──────────────────────────────────────────────────┐
│ npm update check failed │
│ Try running with sudo or get access │
│ to the local update config store via │
│ sudo chown -R $USER:$(id -gn $USER) /tmp/.config │
└──────────────────────────────────────────────────┘

for some reason, on my instance node logs. I'm thinking of clearing the npm cache and re-deploying.

I'd removed checkout from my circleci .config file

@upugo-dev whoops! glad deployment is now working as expected. In regards to your app I'm afraid I'm not too versed with npm so probably not the best person to ask, but if you find a solution and think it's helpful for others finding this gist then please do share.

Looking into it, its an issue with connecting to my mongo atlas db, so nothing to do with deployment at all. Deployment working perfectly now thanks in some part to you! cheers

hey, I've had success without locking the awsebcli version by doing:

- run: 
          name: Install python, pip, and the AWS EB CLI
          command: |
            sudo apt update
            sudo apt install -y python-pip python-dev
            sudo pip install awsebcli

credit to: https://discuss.circleci.com/t/eb-command-not-found/24461

this is on circleci docker image - image: circleci/node:10.15.1

Thanks. This worked perfectly.

I was first getting a

ERROR: This directory has not been set up with the EB CLI
You must first run "eb init".

To fix this I had .elasticbeanstalk folder inside my .gitignore file. I simply removed it and pushed the changes and everything seemed to work fine.

 deploy:
    working_directory: ~/zu
    docker:
      - image: circleci/ruby:2.6.2
    steps:
      - checkout
      - run:
          name: Installing deployment dependencies
          working_directory: /
          command: |
            sudo apt-get -y -qq update
            sudo apt-get install python3-pip python3-dev build-essential
            sudo pip3 install awsebcli

worked for me. awsebcli supports python3 now

Thanks for sharing this. It was an excellent starting point for me to use circleci to deploy to Elastic Beanstalk.

If you don't want to store your config file in your repo, you can also use circleci environment variables and use additional eb commands.
Within CircleCI, I created a context called aws and used this to store the application name and environment name as environment variables. I then added the following job to the circleci config.yml file. Make sure you use the aws context when calling the job.

I've also used the build number ($CIRCLE_BUILD_NUM) as the description for the deployment package.

  deploy:
    docker:
      - image: circleci/python:3.7.7
    steps:
      - checkout
      - run:
          name: installing aws dependencies
          command: |
            sudo pip install --upgrade setuptools
            sudo pip install awsebcli --upgrade
      - run:
          name: deploying
          command: |
            eb init --platform 'Python 3.7 running on 64bit Amazon Linux 2' --region 'us-east-2' $BEANSTALK_APPLICATION_NAME
            eb use $BEANSTALK_ENVIRONMENT_NAME
            eb deploy $BEANSTALK_ENVIRONMENT_NAME --message $CIRCLE_BUILD_NUM

Updated gist based on @IsaiahJTurner 's suggestion (cheers!) as it seems Python2 is being phased out which was resulting in errors like:

pyrsistent requires Python '>=3.5' but the running Python is 2.7.13

For those who may experience the error:

No matching distribution found for botocore<1.22.0,>=1.21.0 (from awsebcli)

A temporary fix seems to be replacing sudo pip install awsebcli --upgrade with sudo pip3 install awsebcli botocore==1.19.63 --upgrade as mentioned here.

@DeBraid were you able to solve the linking to s3 issue?