Buildkite Pipeline w/ step_template

README.md

Step Templates

As there get to be more Buildkite plugins that are useful, the challenge is they clutter up pipelines and it's hard to make cross-cutting changes to the plugin configuration.

What if there was top-level block for creating templates for steps that was basically just top-level properties that would be copied to the steps?

It's an incremental solution vs a redesign of plugins, but it would allow for much cleaner templates.

The attached example is a very simple pipeline, but I've seen long pipelines with lots and lots of plugins, it's a mess!

pipeline.yml

cache:
  paths:
    - "reservations_cache"
  size: "50g"

step_templates:
  golang:
    plugins:
      - elastic/hermit#v1.0.0
    cache:
      - "/root/go/pkg/mod"
      - "/root/.cache/"

  terraform:
    plugins:
      - elastic/hermit#v1.0.0
      - gcp-workload-identity-federation#v1.3.0:
          audience: "$GCP_CLOUD_AUDIENCE"
          service-account: "$GCP_CLOUD_SERVICE_ACCOUNT"
    cache:
      - "/root/go/pkg/mod"
      - "/root/.cache/"        

  docker:
    plugins:
      - elastic/hermit#v1.0.0
      - gcp-workload-identity-federation#v1.3.0:
          audience: "$GCP_CLOUD_AUDIENCE"
          service-account: "$GCP_CLOUD_SERVICE_ACCOUNT"
      - planetscale/docker-login-gcr#v0.0.1:
          registries: ["us-docker.pkg.dev"]
    cache:
      - "/root/.cache/"   
      
steps:
  - group: "🔬 Test"
    key: "lint-and-test"
    steps:
      - id: "integration-test"
        label: ":go: Integration Test"
        template: golang
        branches:
          - "!main"
      - id: "service-test"
        label: ":docker: Service Test"
        template: golang
        branches:
          - "!main"
        commands: ./run_service_test.sh

  - group: ":terraform: Plan"
    key: "terraform-plan"
    template: terraform # template works on a group
    
    steps:
      - label: ":terraform: Plan dev"
        branches:
          - "!main"
        commands: .buildkite/scripts/tf-plan.sh dev
        
      - label: ":terraform: Plan common"
        branches:
          - "!main"
        commands: .buildkite/scripts/tf-plan.sh common
        
      - label: ":terraform: Plan prod"
        branches:
          - "!main"
        commands: .buildkite/scripts/tf-plan.sh prod
          
  - id: "build-docker-image"
    label: ":docker: Build Docker Image"
    template: docker
    branches:
      - "!main"
    commands: .buildkite/scripts/build-docker-image.sh