dcd.md

DCD

Spinnaker File

A few different concepts I've been kicking around.

Plugin, YAML manifest

I really like the flexibility and terseness of this one. As far as structure of the spec goes, it's really simple and defers almost all of its meat and potatoes to being parsed, validated and acted upon by plugins or handlers.

No templating, no Jinja craziness, no inheritance. Just discreet sets of configuration that Spinnaker can run on.

name: clouddriver
definition:
  # I don't have an immediate usage for this bit, but I feel like naming the
  # different sections should be valuable for logging, error handling and so-on.
  # If someone defines the same kind of plugin more than once, we'll need a way
  # of differentiating them.
  myAppConfig:
    kind: appConfig
    params:
    - name: pagerDuty
      value: [email protected]
    - name: accounts
      value:
      - prod
      - test
  chaos:
    kind: chaosMonkey
    params:
    - name: enabled
      value: true
  pipelines:
    kind: pipelines
    params:
    # This is a sticky situation: I don't really feel keen on importing other
    # files, but it may be the only way to keep big apps' dcd files managable.
    - name: config
      value: my-pipeline-config.yml
  loadBalancers:
    kind: loadBalancers
    # ...
# ...

Terraform plugin

We could try leveraging Terraform entirely. This has an advantage of using a tool that is already well adopted and has a good reputation. One disadvantage is that HCL / Terraform can be really verbose.

If we went this route, I'd love to see Spinnaker setup to support also being a Terraform State Backend, which would allow us to control the TF state files for people's other Providers (AWS, K8S, etc). Since the state files have the entire resource state, we'd be able to use it as a fast cache invalidation, or setup internal triggers to respond to external state changes made via TF.

Based on my prior work contributing to Terraform, I'd suggest we host the plugin totally separately from the upstream so we can iterate faster. I do have reservations about how much logic gets dumped into TF as well, so we'd have a complexity problem of spreading logic between our configuration format and our services.

provider "spinnaker" {
  host = "https://spinnaker.net"
  x509 {
    cert_file = ""
    key_file = ""
  }
}

resource "spinnaker_app" "clouddriver" {
  name = "clouddriver"
  tags = ["spinnaker"]
  
  attributes {
    owner = "[email protected]"
    # Issue: Terraform doesn't support extensions to resources. We'd need to
    # publish a -nflx provider that extends OSS. This should be possible, but
    # we'd need to spike out exactly how.
    pagerDuty = "Spinnaker"
    description = cloud read and write operations
  }
  
  notification {
    type = "slack"
    channel = "#spinnaker"
    when {
      event = "pipeline.complete"
      message = optional
    }
    when {
      event = "pipeline.failed"
      message = optional
    }
  }
  
  features {
    pipelines = true
  }
  # ...
}

resource "spinnaker_templated_pipeline" "bake_and_deploy" {
  application = "${spinnaker_application.clouddriver.name}"
  
  name = "Bake & Deploy to Test"
  template {
    source = "spinnaker://my-managed-pipeline-template"
  }
  stages {
    stage {
      
    }
    # ...
  }
  modules {
    # ...
  }
}

# Alternative to making `spinnaker_application` resource huge...
resource "spinnaker_application_notification" "slack_oncomplete" {
  application = "${spinnaker_application.clouddriver.name}"
  
  type = "slack"
  channel = "#spinnaker"
  when {
    event = "pipeline.complete"
    message = optional
  }
  when {
    event = "pipeline.failed"
    message = optional
  }
}

I was considering the Terraform idea a while back - I have commit privileges to Terraform so getting things merged shouldn't be problem.