Cloud Build and Cloud Run.md

Overview

Cloud Build

Cloud Build is a product of GCP (Google Cloud Platform) used to build software quickly across all languages.

Function:

• Speed up Build with VMs on Cloud without wasting personal computer’s resources.
• Source code put on Local, Github, Cloud Source Repositories or Bitbucket all can use Cloud Build.
• Package your source into Docker containers or non-container artifacts with build tools such as Maven, Gradle, webpack, Go, or Bazel.
• After completing Build, the result is updated on Docker Hub or Container Registry.
• You can be flexible to adjust the stream of Build process according to your intention.

Cloud Run

Cloud Run is a fully managed compute platform that automatically scales your stateless containers. Cloud Run is serverless: it abstracts away all infrastructure management, so you can focus on what matters most—building great applications.

Implementation

Setup

1. Setup project: gcloud config set project [PROJECT_ID]
2. Install Cloud SDK
3. Enable the Cloud Build and Cloud Run APIs.
4. Enable IAM:

• Cloud Build Editor => build images
• Cloud Run Admin => deploy to cloud run
• Cloud SQL, Editor,... if necessary

Preparing source files

• Create a Dockerfile to pack your application into an image.

Example:

# Use the official lightweight Node.js 12 image.
# https://hub.docker.com/_/node
FROM node:12-slim

# Create and change to the app directory.
WORKDIR /usr/src/app

# Copy application dependency manifests to the container image.
# A wildcard is used to ensure both package.json AND package-lock.json are copied.
# Copying this separately prevents re-running npm install on every code change.
COPY package*.json ./

# Install production dependencies.
RUN npm install --only=production

# Copy local code to the container image.
COPY . ./

# Run the web service on container startup.
CMD [ "npm", "start" ]

Build and Deploy

There are 2 ways to build and deploy applications.

1. Use Dockerfile

• Get your Cloud project ID by running the following command:

gcloud config get-value project

• Build image and push it to Container Registry:

gcloud builds submit --tag gcr.io/[project-id]/[image-name]

Run the command from the directory containing the Dockerfile, where project-id is your Cloud project ID.

You can use flags to set options when building.

• Deploy to Cloud Run:

gcloud run deploy --image gcr.io/PROJECT-ID/[image-name] --platform managed

Similar to building, you can also use flags to set options when deploying.

Wait a few moments until the deployment is complete. On success, the command line displays the service URL.

Visit your deployed container by opening the service URL in a web browse.

2. Use a config file (cloudbuild.yaml)

• Create a file called cloudbuild.yaml to build the same Docker image as above. This file contains all the sequential steps to build and deploy your application to Cloud Run instead of using the gcloud command as above.

As for build step, it is recommended that you should use the Kaniko tool to build the image.

Kaniko executes directives in a Dockerfile, with remote layer caching for faster builds. By default, Kaniko will cache layers for 6 hours. To override this, set the builds/kaniko_cache_ttl property.

Example:

  - id: 'build-image'
    name: 'gcr.io/kaniko-project/executor:latest'
    args:
      - --destination=gcr.io/$PROJECT_ID/${_CLOUD_RUN_SERVICE_NAME}
      - --dockerfile=${_APP_DOCKER_FILE}
      - --cache=true

As for deploy, it is recommended that you should use Gcloud tool of Cloud builder

Example:

  - id: "deploy"
    name: gcr.io/cloud-builders/gcloud
    entrypoint: bash
    args:
      - -c
      - |
        gcloud beta run deploy ${_APP_NAME} \
          --platform managed \
          --region asia-northeast1 \
          --allow-unauthenticated \
          --image gcr.io/${PROJECT_ID}/${_APP_NAME} \
          --add-cloudsql-instances ${_INSTANCE_CONNECTION_NAME} \
          --set-env-vars INSTANCE-CONNECTION-NAME="${_INSTANCE_CONNECTION_NAME}"

Use the flags to configure as above.

• Start the build by running the following command:

gcloud builds submit --config path-to/cloudbuild.yaml

View the build details

• Open the Cloud Build page in the Google Cloud Console.
• Select your project and click Open. You will see the Build history page.
• Click on a particular build. You will see the Build details page. You can download your build log and view your image details in Container Registry from this page.

Clean up

• Open the Container Registry page in the Google Cloud Console.
• Select your project and click Open.
• Click on the unuse image, click Delete.

Some notes

Setup Cloudbuild Trigger

The company’s projects using Cloud Run almost use Trigger of Cloud Build to auto-tag and build when pushing code to Github.

This part is set up in the item Trigger of Cloud Build.

Sops file .env

The files .env all are set up sops because of security issues before pushed up to run Cloud Run. In order to encrypt/decrypt, you need to be given permission according to projects.

• Install sops in MacOs:

brew install sops

You can use other ways according to environment and download tools.

• Command encrypt file:

sops --gcp-kms [key-sops] --encrypt [file-input] > [file-output]

• In order to decrypt file while building, you can use image rayou/sops public on DockerHub (consulting in bengal-api project)

sops -d ./path-to/env-sops-file > env-file

Also, you can add steps download sops and give permission through cloud-builder gcloud (consulting in several projects such as smart-kaigisitsu, azoom-room-revervation, kaigi-naivi, etc.)

Connect to Cloud SQL

• Create a Cloud SQL for MySQL instance if it has not already existed.
• Find the INSTANCE_CONNECTION_NAME for the instance on the Instance details page. It uses the format PROJECT_ID:REGION:INSTANCE_ID, and is used to identify the Cloud SQL instance you are connecting to.
• Enable the Cloud SQL Admin API, if you haven't already done so.
• Use the option --add-cloudsql-instances when deploying to connect to the SQL Cloud.

Save resources

• When deploy to Cloud Run with Staging environment, it is recommended that you should use configuration with the minimum memory of 128 Mi instead of 256 Mi by default to save resources. In case of using Pupperteer library to run Headless browser, you need use configuration with the minimum memory of 512 Mi

• Besides Cloud Build, GCP also provides a tool for developers to build in Local with steps similar to steps in Cloud Build, called Cloud Build Local. You should build in Cloud Build Local to test and debug before building directly in Cloud Build.

• Similarly, before deploy to Cloud Run, you should test in Local. Consult the instruction of Google Cloud in here