Skip to content

Instantly share code, notes, and snippets.

@deltaepsilon

deltaepsilon/firebase-ci-cd-with-google-cloud-build.md

Created August 20, 2020 10:19
Show Gist options
  • Save deltaepsilon/df6a7093c3f11769d0ef7fe0e6d85dec to your computer and use it in GitHub Desktop.
Save deltaepsilon/df6a7093c3f11769d0ef7fe0e6d85dec to your computer and use it in GitHub Desktop.
Download ZIP
Raw
firebase-ci-cd-with-google-cloud-build.md

Firebase CI/CD with Google Cloud Build

[[Fireline]]: Firebase Functions + Stripe FTW

Fireline is a recent open source project of mine.

It's a series of drop-in Firebase Functions and React hooks that integrate with Stripe to add SaaS payments to web apps.

Firebase Tools CLI

Fireline is built on top of Firebase, so it uses firebase-tools for its deployment.

Create local scripts for all of your Firebase build steps

  • package.json
  • firebase deploy --token $FIREBASE_TOKEN --project $FIREBASE_PROJECT

Set Firebase Functions config programmatically

  • fireabase-config.sh

  • Configure your local development environment with all necessary environment variables

  • Add those variables to your CI/CD deploy environment

      echo "Exporting firebase functions config..."

  npx firebase functions:config:set \
    stripe.sk=$STRIPE_SK \
    stripe.signing_secret.customer=$STRIPE_SIGNING_SECRET_CUSTOMER \
    stripe.signing_secret.invoice=$STRIPE_SIGNING_SECRET_INVOICE \
    stripe.signing_secret.price=$STRIPE_SIGNING_SECRET_PRICE \
    stripe.signing_secret.payment_method=$STRIPE_SIGNING_SECRET_PAYMENT_METHOD \
    stripe.signing_secret.product=$STRIPE_SIGNING_SECRET_PRODUCT \
    stripe.signing_secret.subscription=$STRIPE_SIGNING_SECRET_SUBSCRIPTION \

Create multiple Firebase / GCP projects for serious projects

  • dev project is for local dev as well as a staging deploy to show your clients
  • prod is its own deploy with tighter security

I built Fireline on a single Firebase project because it's too small a project to need a staging/dev build.

Client projects get two Firebase/GCP projects so that clients can test features before releasing them to production.

Dockerfile

  • Dockerfile
  • Start with a thin image
  • Copy over the bare minimum to begin your install
  • Install all dependencies
  • Copy over all code
  • Complete build

Docker caches each build step as a separate layer. Infrequently-changed files can be injected early in the Dockerfile to avoid breaking the layer cache for subsequent commands/layers.

You want your most-frequently-changed files to get passed into the Dockerfile near the end of the file to avoid unnecessary cache invalidations.

FROM mhart/alpine-node:10

WORKDIR /app/functions

COPY ./app/functions/package.json package.json
COPY ./app/functions/yarn.lock yarn.lock
RUN yarn install --pure-lockfile --production=false

WORKDIR /app

COPY ./app/package.json package.json
COPY ./app/yarn.lock yarn.lock
RUN yarn install --pure-lockfile --production=true

ADD ./app /app

RUN yarn && yarn ci:build

Cloudbuild.yaml

  • cloudbuild.yaml

  • Build and tag the current state of the repo

  • Use the built container to set up, test and deploy your code

      steps:
    - name: 'gcr.io/cloud-builders/docker'
  	args: ['build', '-t', 'us.gcr.io/$PROJECT_ID/fireline:latest', '.']
    - name: 'us.gcr.io/$PROJECT_ID/fireline:latest'
  	dir: '/app'
  	args: ['yarn', 'ci:config']
    - name: 'us.gcr.io/$PROJECT_ID/fireline:latest'
  	dir: '/app'
  	args: ['yarn', 'ci:test']
    - name: 'us.gcr.io/$PROJECT_ID/fireline:latest'
  	dir: '/app'
  	args: ['yarn', 'ci:deploy']
  images: ['us.gcr.io/$PROJECT_ID/fireline:latest']
  options:
    env:
  	- 'FIREBASE_DATABASE_URL=$_FIREBASE_DATABASE_URL'
  	- 'FIREBASE_PROJECT=$_FIREBASE_PROJECT'
  	- 'FIREBASE_TOKEN=$_FIREBASE_TOKEN'
  	- 'GOOGLE_APPLICATION_CREDENTIALS=$_GOOGLE_APPLICATION_CREDENTIALS'
  	- 'SERVICE_ACCOUNT_BASE64=$_SERVICE_ACCOUNT_BASE64'
  	- 'STRIPE_PK=$_STRIPE_PK'
  	- 'STRIPE_SK=$_STRIPE_SK'
  	- 'STRIPE_SIGNING_SECRET_CUSTOMER=$_STRIPE_SIGNING_SECRET_CUSTOMER'
  	- 'STRIPE_SIGNING_SECRET_INVOICE=$_STRIPE_SIGNING_SECRET_INVOICE'
  	- 'STRIPE_SIGNING_SECRET_PRICE=$_STRIPE_SIGNING_SECRET_PRICE'
  	- 'STRIPE_SIGNING_SECRET_PRODUCT=$_STRIPE_SIGNING_SECRET_PRODUCT'
  	- 'STRIPE_SIGNING_SECRET_SUBSCRIPTION=$_STRIPE_SIGNING_SECRET_SUBSCRIPTION'
  timeout: 3600s

Cloud Build Dashboard

  • Cloud Build Dashboard
  • Start by configuring triggers.
    • Branch triggers are easy
    • Any push to the branch will kick off a build
    • origin/master builds to staging
    • origin/prod builds to prod
    • Use git to control releases
  • Substitution variables
    • Injected into your Docker container
    • Each trigger has its own variables
  • Secret Servers: Enterprise-grade security
    • Vault is the open-source go-to
    • Using Vault inside of your CI/CD build steps can be expensive because you need to run an http-accessible Vault instance for your build jobs to query
    • I use a free local Vault instance backed by Google Cloud Storage and run in my local environment with Docker Compose
    • I manage my secrets directly in Google Cloud Build

Tips & Tricks

Use Google Container Registry to cache Docker images.

  • Flyerr.co cloudbuild.yaml

  • Caching cut build times from 9 minutes to 4.5 minutes

      steps:
    - name: 'gcr.io/cloud-builders/docker'
  	entrypoint: 'bash'
  	args: ['-c', 'docker pull us.gcr.io/$PROJECT_ID/flyerr:latest-$BRANCH_NAME || exit 0']
    - name: 'gcr.io/cloud-builders/docker'
  	args:
  	  [
  		'build',
  		'-t',
  		'us.gcr.io/$PROJECT_ID/flyerr:latest-$BRANCH_NAME',
  		'--cache-from',
  		'us.gcr.io/$PROJECT_ID/flyerr:latest-$BRANCH_NAME',
  		'.',
  	  ]
    - name: 'gcr.io/cloud-builders/docker'
  	args: ['push', 'us.gcr.io/$PROJECT_ID/flyerr:latest-$BRANCH_NAME']
    - name: 'us.gcr.io/$PROJECT_ID/flyerr:latest-$BRANCH_NAME'
  	dir: '/app'
  	args: ['yarn', 'ci:config']
    - name: 'us.gcr.io/$PROJECT_ID/flyerr:latest-$BRANCH_NAME'
  	dir: '/app'
  	args: ['yarn', 'ci:deploy']
  options:
    env:
  	- 'FIREBASE_APPLICATION_CREDENTIALS=$_FIREBASE_APPLICATION_CREDENTIALS'
  	- 'FIREBASE_DATABASE_URL=$_FIREBASE_DATABASE_URL'
  	- 'FIREBASE_PROJECT=$_FIREBASE_PROJECT'
  	- 'FIREBASE_SERVICE_ACCOUNT_BASE64=$_FIREBASE_SERVICE_ACCOUNT_BASE64'
  	- 'FIREBASE_TOKEN=$_FIREBASE_TOKEN'
  	- 'GOOGLE_APPLICATION_CREDENTIALS=$_GOOGLE_APPLICATION_CREDENTIALS'
  	- 'GOOGLE_PROJECT=$_GOOGLE_PROJECT'
  	- 'GOOGLE_SERVICE_ACCOUNT_BASE64=$_GOOGLE_SERVICE_ACCOUNT_BASE64'
  	- 'ROOT_URL=$_ROOT_URL'
  	- 'TAG=$_TAG'
  timeout: 3600s

Create scripts to debug locally

  • package.json

  • yarn ci:build runs a local build of the container

  • yarn ci:interactive shells into the local build

  • yarn ci:pull pulls the latest image built by Cloud Build

  • yarn ci:latest shells into the latest Cloud Build image for prod debugging

      {
    "name": "@quiver/fireline-parent",
    "version": "1.0.0",
    "main": "index.js",
    "repository": "https://github.com/deltaepsilon/fireline.git",
    "author": "Chris Esplin <[email protected]>",
    "license": "MIT",
    "private": true,
    "scripts": {
  	"build": "docker-compose build",
  	"dev": "docker-compose build workspace && docker-compose run --service-ports --rm workspace zsh",
  	"connect": "docker exec -it workspace-fireline zsh",
  	"ci:login": "npx firebase login:ci --no-localhost",
  	"ci:build": "docker build --tag=fireline .",
  	"ci:interactive": "docker run -it --rm fireline sh",
  	"ci:pull": "docker pull us.gcr.io/fireline-2020/fireline:latest",
  	"ci:latest": "docker run -it --rm us.gcr.io/fireline-2020/fireline:latest sh",
  	"windows:watch": "powershell ./bin/watch.ps1"
    }
  }

Dockerize dev environments

  • docker-compose.yaml

  • One container per service

    • Workspace: Ubuntu-based dev environment
    • Vault: Barebones Vault image for secret management
    • Nginx: Reverse proxy to serve SSL certs locally
    • Certbot: Obtain SSL certs automatically

  • VSCode's Remote Containers Extension can run inside your workspace for a seamless dev environment

  • docker-compose up -d brings up your Docker Compose containers

  • docker-compose down takes your containers down

  • docker-compose run --service-ports --rm workspace sh runs a temporary instance

  • docker exec -it workspace sh shells into an already-running instance. This is useful if you used docker-compose up -d to bring up your containers in daemon mode

  • docker-compose ps lists your running containers

  • alias dc='docker-compose' will save your fingers

      version: '3'
  services:
    workspace:
  	container_name: workspace-fireline
  	build: ./dev/workspace
  	env_file: ./dev/workspace/env.list
  	ports:
  	  - '3000:3000'
  	  - '4000:4000'
  	  - '5000:5000'
  	  - '5001:5001'
  	  - '5002:5002'
  	  - '8080:8080'
  	  - '9000:9000'
  	  - '41000:41000'
  	volumes:
  	  - './app:/app'
  	  - './docs:/app/docs'
    vault:
  	container_name: vault
  	build: ./dev/vault
  	env_file: ./dev/vault/env.list
  	volumes:
  	  - ./dev/vault:/dev/vault
  	  - ./app/vault:/app/vault
  	ports:
  	  - 8200:8200
  	cap_add:
  	  - IPC_LOCK
    nginx:
  	container_name: nginx
  	image: nginx:1.15-alpine
  	depends_on:
  	  - workspace
  	ports:
  	  - '80:80'
  	  - '443:443'
  	volumes:
  	  - ./dev/nginx:/etc/nginx/conf.d
  	  - ./dev/certbot/conf:/etc/letsencrypt
  	  - ./dev/certbot/www:/var/www/certbot
    certbot:
  	container_name: certbot
  	image: certbot/certbot
  	depends_on:
  	  - nginx
  	volumes:
  	  - ./dev/certbot/conf:/etc/letsencrypt
  	  - ./dev/certbot/www:/var/www/certbot
  	  - ./dev/certbot/scripts:/scripts
  	entrypoint: sh /scripts/challenge.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment