drone.md

Install Drone CLI

• Github drone instance: https://drone.digital.homeoffice.gov.uk
• Gitlab drone instance: https://drone-gitlab.digital.homeoffice.gov.uk/

Download and install the Drone CLI from the official website.

At the time of writing brew install drone installs an old version of Drone that is not compatible with Drone 0.5.

Verify it works as expected:

$ drone --version
drone version 0.5.0+dev

Export the DRONE_SERVER and DRONE_TOKEN variables. You can find your token on the top left corner in your account page.

export DRONE_SERVER=https://drone.digital.homeoffice.gov.uk
export DRONE_TOKEN=123_your_token

If your installation is successful, you should be able to query the current Drone instance:

$ drone info
User: youruser
Email: [email protected]

If the output is Error: you must provide the Drone server address. or Error: you must provide your Drone access token. make sure that you have exported the DRONE_SERVER and DRONE_TOKEN variables properly.

Activate your pipeline

Once you are logged in, navigate to your Account from the top right corner.

Select the repo you want to activate.

Navigate to your repository's settings in Github (or Gitlab) and update the payload url in the web hook section so that it matches this pattern:

https://drone-external.digital.homeoffice.gov.uk/hook?access_token=some_token

Configure your pipeline

In the root folder of your project, create a .drone.yml file with the following content:

pipeline:

  my-build:
    privileged: true
    image: docker:1.11
    environment:
      - DOCKER_HOST=tcp://127.0.0.1:2375
    commands:
      - docker build -t image_name .
    when:
      branch: master
      event: push

services:
  dind:
    image: docker:1.11-dind
    privileged: true
    command:
      - "-s"
      - "overlay"

Commit and push your changes:

$ git add .drone.yml
$ git commit
$ git push origin master

After you pushed, you should be able to see the build failing in Drone with the following message:

ERROR: Insufficient privileges to use privileged mode

The current configuration requires extended privileges to run, but you're repository is not trusted. With a little help from Devops you should get your repository whitelisted.

Once you are good to go, you can trigger a new build by pushing a new commit or through the Drone UI.

This time the build will succeed.

Signature

Drone requires you to sign the Yaml file before injecting secrets into your build environment. You can generate a signature for a named repository like this:

$ drone sign UKHomeOffice/your_repo

This will create a .drone.yml.sig in the current directory. You should commit this file:

$ git add .drone.yml.sig
$ git commit
$ git push origin master

You must re-sign your .drone.yml every time you change it.

Publishing to Quay

Register for a free Quay account using your Github account linked to the Home Office organisation.

Once you've logged into Quay check that you have ukhomeofficedigital under Users and Organisations.
If you do not, [make a request by adding an issue to the Platform Access column] (https://github.com/UKHomeOffice/hosting-platform-bau/projects/1)

Once you have access to view the ukhomeofficedigital repositories, click repositories and click the + Create New Repositories that is:

• public
• empty - no need to create a repo from a Dockerfile or link it to an existing repository

Add your project to the UKHomeOffice Quay account

Raise a ticket for a new UKHomeOffice Quay robot in the Platform Access column. You have to pick a name for it. You should be supplied a robot password in response.

Add the step to publish the docker image to Quay in your Drone pipeline with the supplied docker login but NOT the password:

image_to_quay:
  image: docker:1.11
  environment:
    - DOCKER_HOST=tcp://127.0.0.1:2375
  commands:
    - docker login -u="ukhomeofficedigital+drone_demo" -p=${DOCKER_PASSWORD} quay.io
    - docker tag image_name quay.io/ukhomeofficedigital/node-hello-world:${DRONE_COMMIT_SHA}
    - docker push quay.io/ukhomeofficedigital/node-hello-world:${DRONE_COMMIT_SHA}
  when:
    branch: master
    event: push

Where the image_name in:

docker tag image_name quay.io/ukhomeofficedigital/node-hello-world:${DRONE_COMMIT_SHA}

is the name of the image you tagged previously in the build step.

Since your .drone.yml has changed, you have to sign it before you can push the repository to the remote:

$ drone sign UKHomeOffice/your_repo
$ git add .drone.yml.sig
$ git add .drone.yml
$ git commit
$ git push origin master

The build should fail with the following error:

docker login -u="ukhomeofficedigital+drone" -p=${DOCKER_PASSWORD} quay.io
inappropriate ioctl for device

The error points to the missing DOCKER_PASSWORD environment variable.

You can inject the robot's password supplied to you in your raised ticket to the Platform team with:

$ drone secret add --image="docker:1.11" UKHomeOffice/your_repo DOCKER_PASSWORD your_robot_token

Restarting the build should be enough to make it pass.

Deployments and promotions

Create a step that runs only on deployments:

deploy-to-preprod:
  image: busybox
  commands:
    - /bin/echo hello preprod
  when:
    environment: preprod
    event: deployment

Sign your build and push the changes to your remote repository.

You can deploy the build you just pushed with the following command:

$ drone deploy UKHomeOffice/your_repo 16 preprod

Where 16 is the successful build number you wish to deploy to the preprod environment.

You can pass additional arguments to your deployment as environment variables:

$ drone deploy UKHomeOffice/your_repo 16 preprod -p DEBUG=1 -p NAME=Dan

and use from the step like this:

deploy-to-preprod:
  image: busybox
  commands:
    - /bin/echo hello ${NAME}
  when:
    environment: preprod
    event: deployment

Environments are strings and can be set to any value. When you wish to deploy to several environments you can create a step for each one of them:

deploy-to-preprod:
  image: busybox
  commands:
    - /bin/echo hello preprod
  when:
    environment: preprod
    event: deployment

deploy-to-prod:
  image: busybox
  commands:
    - /bin/echo hello prod
  when:
    environment: prod
    event: deployment

And deploy them accordingly:

$ drone deploy UKHomeOffice/your_repo 15 preprod
$ drone deploy UKHomeOffice/your_repo 16 prod

Read more on environment and constraints

Publishing to Artifactory

Images hosted on Artifactory are private.

You can add the following step in your .drone.yml:

image_to_artifactory:
  image: docker:1.11
  environment:
    - DOCKER_HOST=tcp://127.0.0.1:2375
  commands:
    - docker login -u="regt-build-bot" -p=${DOCKER_ARTIFACTORY_PASSWORD} docker.digital.homeoffice.gov.uk
    - docker tag image_name docker.digital.homeoffice.gov.uk/node-hello-world:${DRONE_COMMIT_SHA}
    - docker push docker.digital.homeoffice.gov.uk/node-hello-world:${DRONE_COMMIT_SHA}
  when:
    branch: master
    event: push

Where the image_name in:

docker tag image_name docker.digital.homeoffice.gov.uk/node-hello-world:${DRONE_COMMIT_SHA}

is the name of the image you tagged previously in the build step.

Since your .drone.yml has changed, you have to sign it before you can push the repository to the remote:

$ drone sign UKHomeOffice/your_repo
$ git add .drone.yml.sig
$ git add .drone.yml
$ git commit
$ git push origin master

You can check that the image has now been published to Artifactory. You will need to login with your Office365 account to access this page.

Drone as a Pull Request builder

Drone pipelines are triggered when events occurs. Event triggers can be as simple as a push, a tagged commit or a pull request or as granular as only for pull request with a named branch test. You can limit the execution of build steps at runtime using the when block. As an example, this block executes only on pull requests:

pr-builder:
  privileged: true
  image: docker:1.11
  environment:
    - DOCKER_HOST=tcp://127.0.0.1:2375
  commands:
    - docker build -t node-hello-world .
  when:
    event: pull_request

Drone will automatically execute that step when a new pull request is raised.

> Read more about Drone conditions

Q&A

Q: The build fails with "ERROR: Insufficient privileges to use privileged mode"

A: Your repository isn't in the trusted list of repositories. Get in touch with Devops and ask them to trust it.

Q: The build fails with "Cannot connect to the Docker daemon. Is the docker daemon running on this host?"

A: Make sure that your steps contain the environment variable DOCKER_HOST=tcp://127.0.0.1:2375 like in this case:

my-build:
  privileged: true
  image: docker:1.11
  environment:
    - DOCKER_HOST=tcp://127.0.0.1:2375
  commands:
    - docker build -t image_name .
  when:
    branch: master
    event: push

Also make sure that you have dind as a service in your .drone.yml:

services:
  dind:
    image: docker:1.11-dind
    privileged: true
    command:
      - "-s"
      - "overlay"

Q: The build fails when uploading to Quay with the error "Inappropriate ioctl for device"

A: This suggests that the docker executable prompted for credentials instead of reading them from the command line. This might be caused by:

• Your .drone.yml not being signed. When you want to inject environment variables in your build you must sign your .drone.yml.
• The secret wasn't injected correctly or the password is incorrect.

Q: As part of my build process I have two Dockerfiles to produce a Docker image. How can I share files between builds in the same step?

A: When the pipeline starts, Drone creates a Docker data volume that is passed along all active steps in the pipeline. If the first step creates a test.txt file, the second step can use that file. As an example, this pipeline uses a two step build process:

pipeline:

  first-step:
    image: busybox
    commands:
      - echo hello > test.txt
    when:
      branch: master
      event: push

  second-step:
    image: busybox
    commands:
      - cat test.txt
    when:
      branch: master
      event: push

Q : What shall I do when my deployment scripts are in another repo?

A: You can leverage the drone-trigger plugin to trigger a build on another repository. If you have a repository with deployment scripts, you can setup Drone on that repository like so:

pipeline:
  deploy_to_uat:
    image: busybox
    commands:
      - echo ${SHA}
    when:
      event: deployment
      environment: uat

Once you are ready, you can sign the .drone.yml and push the changes to the remote repository. In your main repository you can add the following step:

trigger_deploy:
  image: quay.io/ukhomeofficedigital/drone-trigger:latest
  drone_server: https://drone.digital.homeoffice.gov.uk
  repo: UKHomeOffice/deployment_repo
  branch: master
  deploy_to: uat
  params: SHA=${DRONE_COMMIT_SHA}
  when:
    event: deployment
    environment: uat

The settings are very similar to the drone deploy command:

• deploy_to is the environment constraint
• params is a list of comma separated list of arguments. In the command line tool, this is equivalent to -p PARAM1=ONE -p PARAM2=TWO
• repo the repository where the deployment scripts are located

The next time you trigger a deployment on the main repository with:

$ drone deploy UKHomeOffice/your_repo 16 uat

This will trigger a new deployment on the deployment scripts repository.

Q: I can't sign .drone.yml

• Make sure that your repository is activated in Drone
• Make sure your DRONE_SERVER and DRONE_TOKEN are properly set
• Make sure you can successfully connect to Drone by typing drone info