Terragrunt config to auto-generate provider and backend config so you can apply library modules directly in infrastructure-live without an adapter module in infrastructure-modules

README.md

Using Terragrunt generate for extra DRY Terraform

Terragrunt config to auto-generate provider and backend config so you can apply library modules directly in infrastructure-live without an adapter module in infrastructure-modules.

dev > .terraform-version

0.22.4

dev > .terragrunt-version

0.12.21

dev > ca-central-1 > dev > data-stores > sqs-worker > terragrunt.hcl

# ------------------------------------------------------------------------------
# ENVIRONMENT VARIABLES
# Define these secrets as environment variables
# ------------------------------------------------------------------------------

# AWS_ACCESS_KEY_ID
# AWS_SECRET_ACCESS_KEY

# ------------------------------------------------------------------------------
# TERRAGRUNT CONFIGURATION
# This is the configuration for Terragrunt, a thin wrapper for Terraform that supports locking and enforces best
# practices: https://github.com/gruntwork-io/terragrunt
# ------------------------------------------------------------------------------

# Terragrunt will copy the Terraform configurations specified by the source parameter, along with any files in the
# working directory, into a temporary folder, and execute your Terraform commands in that folder.
terraform {
  source = "git::[email protected]:gruntwork-io/package-messaging.git//modules/sqs?ref=v0.3.1"
}

# Include all settings from the root terragrunt.hcl file
include {
  path = find_in_parent_folders()
}

#dependency "kms_master_key" {
#  config_path = "../../../_global/kms-master-key"
#}

inputs = {
  name = "worker"

  visibility_timeout_seconds = 60
  message_retention_seconds = 86400
  max_message_size = 131072
  delay_seconds = 10
  receive_wait_time_seconds = 20
  dead_letter_queue = true

#  kms_master_key_id = dependency.kms_master_key.outputs.remote_state.key_id
}

dev > ca-central-1 > dev > env.yaml

# These variables apply to this entire environment. They are automatically pulled in using the extra_arguments
# setting in the root terraform.tfvars file's Terragrunt configuration.
vpc_name: "dev"

dev > ca-central-1 > region.yaml

# These variables apply to this entire AWS region. They are automatically pulled in using the extra_arguments
# setting in the root terraform.tfvars file's Terragrunt configuration.
aws_region: "ca-central-1"

dev > empty.yaml

# This is intentionally empty object. This YAML file is used as a catch all to return an empty map when expected yaml
# vars do not exist. Note that this file can not be completely empty because terragrunt's yamldecode function expects an
# object, and it doesn't treat empty files as an empty object (you get a syntax error).
{}

dev > terragrunt.hcl

# ---------------------------------------------------------------------------------------------------------------------
# TERRAGRUNT CONFIGURATION
# Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules,
# remote state, and locking: https://github.com/gruntwork-io/terragrunt
# ---------------------------------------------------------------------------------------------------------------------

# Configure Terragrunt to automatically store tfstate files in an S3 bucket
remote_state {
  backend = "s3"
  config = {
    encrypt        = true
    bucket         = local.terraform_state_s3_bucket
    key            = "${path_relative_to_include()}/terraform.tfstate"
    region         = local.aws_region
    dynamodb_table = "terraform-locks"
  }
}

# ---------------------------------------------------------------------------------------------------------------------
# LOCAL PARAMETERS (just within this file)
# ---------------------------------------------------------------------------------------------------------------------
locals {
  aws_region     = "ca-central-1"
  aws_account_id = "000000000000"
  account_alias  = "dev"
  terraform_state_s3_bucket = "mycompany-${local.account_alias}-${local.aws_account_id}-terraform-state"
}

# ---------------------------------------------------------------------------------------------------------------------
# GLOBAL PARAMETERS
# These variables apply to all configurations in this subfolder. These are automatically merged into the child
# `terragrunt.hcl` config via the include block.
# ---------------------------------------------------------------------------------------------------------------------

inputs = merge(
  # Configure Terragrunt to use common vars encoded as yaml to help you keep often-repeated variables (e.g., account ID)
  # DRY. We use yamldecode to merge the maps into the inputs, as opposed to using varfiles due to a restriction in
  # Terraform >=0.12 that all vars must be defined as variable blocks in modules. Terragrunt inputs are not affected by
  # this restriction.
  yamldecode(
    file("${get_terragrunt_dir()}/${find_in_parent_folders("region.yaml", "${path_relative_from_include()}/empty.yaml")}"),
  ),
  yamldecode(
    file("${get_terragrunt_dir()}/${find_in_parent_folders("env.yaml", "${path_relative_from_include()}/empty.yaml")}"),
  ),
  # Additional global inputs to pass to all modules called in this directory tree.
  {
    aws_account_id             = local.aws_account_id
    terraform_state_s3_bucket  = local.terraform_state_s3_bucket
    terraform_state_aws_region = local.aws_region
  },
)

# When using this terragrunt config, terragrunt will generate the file "provider.tf" with the aws provider block before
# calling to terraform. Note that this will overwrite the `provider.tf` file if it already exists.
generate "provider" {
  path      = "provider.tf"
  if_exists = "skip" # Allow modules to override provider settings
  contents = <<EOF
provider "aws" {
  # The AWS region in which all resources will be created
  region = "${local.aws_region}"

  # Only these AWS Account IDs may be operated on by this template
  allowed_account_ids = ["${local.aws_account_id}"]

  version = "~> 2.50.0"
}
provider "external" {
  version = "~> 1.2"
}
provider "null" {
  version = "~> 2.1"
}
provider "template" {
  version = "~> 2.1"
}
EOF
}

generate "backend" {
  path      = "backend.tf"
  if_exists = "overwrite"
  contents = <<EOF
terraform {
  backend "s3" {}
}
EOF
}

@AlainODea I'm not sure how else to communicate that YAML format uses colons (:), not equals (=), to assign keys a value.

@AlainODea I'm not sure how else to communicate that YAML format uses colons (:), not equals (=), to assign keys a value.

You are absolutely correct here. Good catch! Sorry for my denseness.

Not sure why Terragrunt's YAML implementation doesn't flag this. I can't find anywhere in the specification that allows the = for key value construction. This is a cursed implementation. I can't unsee this now 😅

The outputs of the dependency block doesn't seem to work for terragrunt run-all plan command. How did you workaround / manage to get it to work?

The outputs of the dependency block doesn't seem to work for terragrunt run-all plan command. How did you workaround / manage to get it to work?

@kosperera probably better to ask Gruntwork that question. I do not use run-all plan myself.

The outputs of the dependency block doesn't seem to work for terragrunt run-all plan command. How did you workaround / manage to get it to work?

@kosperera probably better to ask Gruntwork that question. I do not use run-all plan myself.

Looks like there is already an open conversation about outputs at gruntwork-io/terragrunt#1330

I tried terragrunt plan and the error seems to be the same. Do you have a public repo on this example that works?

The outputs of the dependency block doesn't seem to work for terragrunt run-all plan command. How did you workaround / manage to get it to work?

@kosperera probably better to ask Gruntwork that question. I do not use run-all plan myself.

Looks like there is already an open conversation about outputs at gruntwork-io/terragrunt#1330

I tried terragrunt plan and the error seems to be the same. Do you have a public repo on this example that works?

The kms_master_key dependency is missing. There's no terragrunt.hcl for it. Best bet for now is to exclude it and the kms_master_key_id.

@kosperera I've commented out the kms_master_key dependency and the kms_master_key_id input. Hopefully it will plan for you now.

This still depends on private code you need a Gruntwork subscription to use so it's not likely to be practical to use other than a conceptual example.

@kosperera I've commented out the kms_master_key dependency and the kms_master_key_id input. Hopefully it will plan for you now.

Thanks, @AlainODea. I was looking for a decent example that works with the dependency block. So far terragrunt plan doesn't work as expected outputs for me, so wondered whether it's just my code.