Skip to content

Instantly share code, notes, and snippets.

@nicolasdao

nicolasdao/terraform_guide.md Secret

Last active July 19, 2021 02:57
Show Gist options
  • Save nicolasdao/53387b44bf74f91d379ebdb52cfdc125 to your computer and use it in GitHub Desktop.
Save nicolasdao/53387b44bf74f91d379ebdb52cfdc125 to your computer and use it in GitHub Desktop.
Download ZIP
Terraform guide. Keywords: terraform ioc infrastructure as code
Raw
terraform_guide.md

TERRAFORM GUIDE

Usefull links:

Rule of thumb: To figure out how to configure a resource, simply Google terraform <resource name>

WARNING: The workspace concept is very confusing. For now, please be aware that it means different things whether you're managing your project locally or via Terraform Cloud. The difference between the two contexts is explained in the The Workspace conundrum section.

Table of contents

Overview

Terraform is an infrastructure-as-code tool that supports all the major Cloud providers. Its core concepts are:

  • Infrastructure as cattle, not pets, i.e., provisioning an immutable infrastructure rathen than mutating it (1).
  • Declarative approach over procedural, i.e., declare what you want (end result) via config files (.tf files) rather than coding how to get what you want. With the declarative approach, declaring 5 servers must be provisioned guarantees that 5 servers will run in total. With the procedural, there are risks that 5 additional servers are created, rasing the total number to more than 5.
  • Use the CLI tool locally, or use Terraform Cloud to store states and execute operations. The first one is good to get started, but the second one is the one you'd probably want to collaborate, govern and automate your infrastructure (CI/CD).

(1) Tools that would mutate the infrastructure: Chef, Puppet, Ansible.

Install, setup & prerequisites

brew tap hashicorp/tap
brew install hashicorp/tap/terraform
brew update
brew upgrade hashicorp/tap/terraform

To test the install worked:

terraform -help

To install auto-complete:

  1. Make sure you already have a .bashrc (if you're using bash) or .zshrc (if you're using Zsh).
  2. Run this command:
terraform -install-autocomplete

Getting started

This section helps to get started using the simplest approach, which is to use the Terraform CLI. In this context, the entire development is done on your local machine without using any remote or external systems. In reality, you will probably not do this and switch to using Terraform Cloud to improve collaboration, governance, security and automation. Terraform Cloud is covered in another section. We recommend you get familiar with the core concepts of this section first before jumping into Terraform Cloud.

This section does not cover Git workflows and dev lifecycles as those workflows depend on understanding the difference between Terraform CLI and Terraform Cloud (which is mainly the Workspace conundrum). This topic is covered later in the Git workflows and dev lifecycles section.

Overview

Project and file structure

  • A Terraform project describes the infrastructure in .tf files. They look similar to JSON files, but they're not. The language is referred to as HCL (HashiCorp Configuration Language). All .tf files will be executed regardless of their name.
  • Each .tf file can define the following blocks:
    • provider: That's typically a cloud provider such as AWS, Google and so on (AWS example here).
    • resource: That's the actual infrastructure piece that you wish to provision (AWS S3 example here).
    • You could stop there and with those two concepts, you'd be able to deploy anything. However, as you project becomes more complex, the following other concepts become necessary:
      • variable: A variable is a definition that helps passing arguments to your deployment. A variable does not define a value, but can define a default value when a value is not explicitely passed.
      • locals: A series of local variables (not related to the variable above) that can use some native functions which usually helps managing values with string operations (e.g., string concatenation, upper/lower case, ...).
      • output: Helps to modularize project by returning values so that other modules or projects can use them. Also use for debugging as outputs are printed in the terminal.
      • data: Data allows you to query provisioned Cloud resources to extract their details and use them in any module or project.
    • module: Helps to encapsulate logic in reusable resources.
  • The order in which files and their content is provisioned is inferred automatically by Terraform. You can define a resource in one file and refer to it in another without any problems.
  • You can configure the entire infra in a single file, but that's not recommended. Instead, try to break it down in logical blocks that reflect your intent.
  • The other file types are:
    • .tfvars: This files stores the explicit value of variable definitions. A typical use case would be a dev.tfvars and prod.tfvars.
    • .tfstate: This is where the magic happens. It stores the current infrastructure state. That's how Terraform knows something has changed. There is no constraints on how that file should be named, but the usual convention is to name it terraform.tfstate.
  • To manage environments such as dev and prod, Terraform introduces the concept of Workspaces(in Pulumi, this is equivalent to Stacks). The idea is to define specific configurations stored in .tfvars files that can be associated with each workspace so that deploying a specific environment is only a matter of switching workspaces and then executing a terraform apply.
  • By default, the state of your infra is stored on your local machine inside a terraform.tfstate file. This file is critical to run your project, but it cannot be put under source-control as it may contain sensitive information. To solve this challenge, Terraform can use backends other than the default local backend (which stores the terraform.tfstate file locally). A backend is responsible for storing the .tfstate file. To sync your .tfstate so that concurrent deployment don't create unexpected results, choose a backend such as S3 or Terraform Cloud). For more details, please refer to the backends section.

Basic commands

Please refer to the Commands section for a description of the main commands.

Once you have your files ready, the first command to use is:

terraform init

This command will create the default terraform.tfstate. This command must be run again if you update the backend.

When you're ready to deploy your changes, run:

terraform apply

Don't worry, this command will review your changes first (equivelant to terraform plan). It will prompt you to confirm if the details are correct before applying them. If you do not want to be asked to review over and over again, use terraform apply -auto-approve instead.

To list the details of a provisioned resource:

terraform state show aws_s3_bucket.www

If you want to filter a specific property:

terraform state show aws_s3_bucket.www | grep website_endpoint

Basic - Provisioning an AWS S3 bucket

Login to your Cloud provider

Terraform uses the same login procedure as the specific cloud CLI (e.g., AWS CLI, gcloud CLI). To deploy to the right account, make sure you are logged in to it via your preferred CLI (if NPX is installed on your machine, simply run npx switch-profile).

Create a main.tf file as follow:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

# Configure the default AWS Provider
provider "aws" {
  region = "ap-southeast-2"
}

data "aws_s3_bucket" "selected" {
  bucket = "qwerty4321"
}

output "my_bucket" {
  value = data.aws_s3_bucket.selected
}

Where qwerty4321 should be replaced by a bucket's name that exists in your logged in AWS account (also, make sure the region for your S3 bucket matches the one defined above (i.e., ap-southeast-2), otherwise change it).

then run:

terraform init

then:

terraform plan

If this works, this command should output the bucket's details. This rest of this section covers how this file works.

Providers and resources

Single provider with one resource

A provider is usually the first thing you define in your project. It looks like this:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

# Configure the default AWS Provider
provider "aws" {
  region = "ap-southeast-2"
}

AWS regions available in the annexes under the AWS regions section.

To provision a resource like an S3 bucket, use the resource block:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

# Configure the default AWS Provider
provider "aws" {
  region = "ap-southeast-2"
}

# More S3 resource APIs at https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket
resource "aws_s3_bucket" "b" {
  bucket = "my-tf-test-bucket-43917451"
  acl    = "private"

  tags = {
    Name        = "My bucket"
    Environment = "Dev"
  }
}

Then run:

terraform init

Only if your project had not been initialized yet

and then:

terraform apply

To delete a resource, simply remove it from the .tf file, then run terraform apply.

Multiple providers

As you can see, you have to define one provider per region. If you need to define multiple region, use alias as follow:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

# Configure the default AWS Provider
provider "aws" {
  region = "ap-southeast-2"
}

# Configure the US West AWS Provider
provider "aws" {
  alias  = "west"	
  region = "us-west-2"
}

resource "aws_s3_bucket" "b" {
  bucket = "my-tf-test-bucket-43917451"
  acl    = "private"

  tags = {
    Name        = "My bucket"
    Environment = "Dev"
  }
}

resource "aws_s3_bucket" "b_west" {
  provider = aws.west
  bucket   = "my-tf-test-bucket-43917451-west"
  acl      = "private"

  tags = {
    Name        = "My bucket west"
    Environment = "Dev"
  }
}

Restricting a provider to a specific account ID

Because you may have multiple account connected to your local machine, it helps to explicitely define which account is targetted to avoid disrupting an account in prod. Use the allowed_account_ids property on the provider as follow:

provider "aws" {
  allowed_account_ids = ["123455"]
  region = "ap-southeast-2"
}

Outputs

The output block helps to:

  • Debug by getting explicit data back after a terraform plan or terraform apply.
  • Expose specific data if the project is meant to be a module. In the case, loading that module will expose those outputs.

In our simple case, we may want to get the details of the bucket we've just provisioned:

output "my_buckets" {
  value = aws_s3_bucket.b.id
  description: "My bucket ID"
}

As you can see, an output requires a value property and an optional description property.

The above will output:

my_buckets = "my-tf-test-bucket-43917451"

To output the entire bucket data:

output "my_buckets" {
  value = aws_s3_bucket.b
  description: "My bucket data"
}

Data

Now that we have provisioned something, we can explicitely query its metadata via the data block.

data "aws_s3_bucket" "bb" {
  bucket = aws_s3_bucket.b.bucket
}

output "my_buckets" {
  value = data.aws_s3_bucket.bb
}

Put your project under source control and ignore critical files

Ignoring critical files in the .gitignore is important as those files may contain sensitive pieces of information. At a minimum, ignore those files and folder:

.terraform/
terraform.tfstate
terraform.tfstate.backup
terraform.tfstate.*.backup

The .terraform/ folder is ignored because it includes artefact binaries. The terraform.tfstate may contain sensitive pieces of information. As for terraform.tfstate.backup(1) and terraform.tfstate.*.backup(2), .backup is a defacto convention to backup the .tfstate, so it must also be excluded.

IMPORTANT: Because the terraform.tfstate should not be version-controlled, collaboration between multiple engineers requires a backend other than the default local. Please refer to the backends section for more details.

You may also want to ignore some .tfvars files if they contain sensitive information:

.terraform/
terraform.tfstate
terraform.tfstate.backup
*.tfvars

(1) This file is automatically created when the terraform refresh command is executed. (2) These files are automatically created when the terraform state rm <RESOURCE NAME> command is executed.

Intermediate - Provisioning a static website on S3

Before jumping into provisioning a static website on S3, let's learn how to manage strings with Terraform, as this is a common task in many automation processes.

String manipulation

With Terraform, you usually either use inline strings or load them from a file or remote location.

Inline manipulation

When a property requires a string which contains line breaks, use one of the following 2 options:

Option 1 - Maintain indentation:

    someProp = <<WHATEVER_YOU_WANT
      Some text that says:
        Hello world
    WHATEVER_YOU_WANT

With this option, the string passed to someProp is:

      Some text that says:
        Hello world

Option 2 - No indentation:

    someProp = <<-WHATEVER_YOU_WANT
      Some text that says:
        Hello world
    WHATEVER_YOU_WANT

With this option, the string passed to someProp is:

Some text that says:
  Hello world

File reference

If your string is inside a local file, then use the file function to access it as follow:

# Creates a public bucket to host static website.
# Doc: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket#static-website-hosting
resource "aws_s3_bucket" "website_bucket" {
  bucket = "cloudlesslabs.com"
  acl    = "public-read"
  policy = file("public_website_policy.json")

  website {
    index_document = "index.html"
    error_document = "error.html"
  }

  versioning {
    enabled = true
  }
}

Variables and Locals

Let's go back to provisioning a static website on AWS S3.

locals {
	domain = "cloudlesslabs.com"
}

# Creates a public bucket to host static website.
# Doc: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket#static-website-hosting
resource "aws_s3_bucket" "website_bucket" {
  bucket = local.domain
  acl    = "public-read"
  policy = <<-POLICY
  {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "PublicReadGetObject",
        "Effect": "Allow",
        "Principal": "*",
        "Action": "s3:GetObject",
        "Resource": "arn:aws:s3:::${local.domain}/*"
      }
    ]
  }
  POLICY

  website {
    index_document = "index.html"
    error_document = "error.html"
  }

  versioning {
    enabled = true
  }
}

The sample above shows that using the local.domain local variable, we can maintain that value in a single place.

Difference between variables and locals

Very often, you'll have to refer to the same object (string, number, ...) over and over again. This is error prone as well we tedious to manage when an update is required. To deal with this challenge, Terraform uses locals and variable blocks.

locals is the easiest to use:

locals {
	hello = "world"
}

output "msg" {
	value = "Hello ${local.hello}"
}

Running terraform plan outputs Hello world.

Notice the string interpolation $ in "Hello ${local.hello}". This is the same as in Javascript.

Use locals when you need to manage data in a single place throughout your project and that data does not vary from one environment to another (if the data changes whether you're in dev or prod, use variable instead of locals). locals can be defined in their own .tf file and referred in others.

variable can be used similarly to locals if it defines a default value, but its purpose is different. A variable is a definition. Once defined, values can be injected in your project via the CLI using explicit values or .tfvars files. For example:

Using variable as a replacement for locals:

variable "hello" {
	default = "world"
}

output "msg" {
	value = "Hello ${var.hello}"
}

Running terraform plan outputs Hello world. But running terraform plan -var="hello=Sam" outputs Hello Sam.

To pass multiple values use a command similar to this: terraform plan -var="hello=Sam" -var="var2=something else".

Using the following some.tfvars:

hello = "Little File"

Running terraform plan -var-file="some.tfvars" outputs: Hello Little File.

Additionally, variables support the following 4 optional arguments: type, description, validation and sensitive. More about those arguments at https://www.terraform.io/docs/language/values/variables.html#arguments.

terraform.tfvars and *.auto.tfvars files

Variables stored inside a terraform.tfvars or any file ending with .auto.tfvars is in the root folder are automatically applied to the terraform plan or terraform apply without requiring the -var-file option.

Advanced

This section focuses on improving maintainability and reusability.

Looping and conditional block - aka dynamic block and count meta-argument

The dynamic block and the count meta-argument are designed to primarily create multiple blocks of the same type, but with different values. This section will also show a design pattern to use it to emulate a conditional block, i.e., the ability to toggle a configuration on or off.

Looping block

count meta-argument

In cases where you need to provide the same resource multiple times (e.g., multiple EC2 instances):

resource "aws_instance" "server" {
  count = 4 # create four similar EC2 instances

  ami           = "ami-a1b2c3d4"
  instance_type = "t2.micro"

  tags = {
    Name = "Server ${count.index}"
  }
}
dynamic block

The dynamic block allows to rewrite this:

resource "aws_elastic_beanstalk_environment" "tfenvtest" { 
  ...
  setting {
    name      = "Subnets"
    namespace = "aws:ec2:vpc"
    value     = "subnet-06dcc5d225b48b816"
  }
  setting {
    name      = "VPCId"
    namespace = "aws:ec2:vpc"
    value     = "vpc-00e555dc5f3a3cbc2"
  }
}

as this:

resource "aws_elastic_beanstalk_environment" "tfenvtest" { 
  ...
  dynamic "setting" {
  	for_each = var.settings
  	content {
      name      = setting.value["name"]
      namespace = setting.value["namespace"]
      value     = setting.value["value"]
  	}
  }
}

Where settings is defined in a .tfvars file as follow:

settings = [
 {
   name = "Subnets"
   namespace = "aws:ec2:vpc"
   value = "subnet-00e555dc5f3a3cbc2"
 },
 {
   name = "VPCId"
   namespace = "aws:ec2:vpc"
   value = "vpc-06dcc5d225b48b816"
 },
]

Conditional block

As of July 2021, Terraform does not support native constructs to toggle a configuration on or off, i.e., conditional block. This section details a couple of design patterns which allows to emulate this feature.

Conditional resource with count
locals {
  my_function_name = "awesome-function"
}

resource "aws_lambda_function" "my_lambda_java" {
  filename         = "lambda.jar"
  function_name    = "${local.my_function_name}"
  handler          = "Handler"
  runtime          = "java8"
  count            = var.lambda_type == "java" ? 1 : 0
}

resource "aws_lambda_function" "my_lambda_javascript" {
  filename         = "lambda.jar"
  function_name    = "${local.my_function_name}"
  handler          = "index.handler"
  runtime          = "nodejs10.x"
  count            = var.lambda_type == "javascript" ? 1 : 0
}

WARNING: Because we're using count, aws_lambda_function.my_lambda_java and aws_lambda_function.my_lambda_javascript are now arrays and not single object. To access their properties, you must use aws_lambda_function.my_lambda_java[0] instead of aws_lambda_function.my_lambda_java.

Conditional resource properties with dynamic
variable "maintenance_mode" {
  type    = bool
  default = false
}

resource "aws_lb_listener" "this" {
  load_balancer_arn = aws_lb.this.arn
  port              = 443
  protocol          = "HTTPS"
  ssl_policy        = "ELBSecurityPolicy-TLS-1-2-2017-01"

  dynamic "default_action" {
    for_each = var.maintenance_mode ? [] : [1]
    content {
      type             = "forward"
      target_group_arn = aws_lb_target_group.this.arn
    }
  }

  dynamic "default_action" {
    for_each = var.maintenance_mode ? [1] : []
    content {
      type = "fixed-response"
      fixed_response {
        content_type = "text/html"
        message_body = file("${path.module}/pages/scheduled-maintenance.html")
        status_code  = "503"
      }
    }
  }
}

Modules

IMPORTANT: Modules cannot define a provider block. provider blocks can only be defined in the root project.

A local module is simply a sub-folder that contains:

  • Some .tf files that describe the resources for that module.
  • Optionally a variables.tf (or whatever you wish to call this file) that contains the possible inputs for this module. You would only define variable blocks in that file.
  • Optionally an outputs.tf (or whatever you wish to call this file) that contains the possible outputs for this module. You would only define output blocks in that file.

To refer that module in the rest of your project, use the module block:

module lambda_edge {
	source = "./modules/lambda_edge"
	name   = "Super cool" # name would be a variable defined in your module.
}

Modules can also be deployed in isolation of the rest of the stack:

terraform apply -target="module.lambda_edge"

Workspaces

Terraform workspaces is a confusing concept as its implementation varies based on its execution context. This section focuses on managing workspaces locally with the CLI. To learn how to manage workspaces with Terraform Cloud, or how to migrate a workspace managed by the CLI to Terraform Cloud, please refer to the Terraform Cloud Overview section.

In practice:

  • Create a new workspace: terraform workspace new prod
  • Use the workspace's name in your resources: "The current workspace is ${terraform.workspace}"
  • Deploy a workspace: terraform workspace select prod && terraform apply -var-file=prod.tfvars

WARNING!!!: terraform.workspace is only usefull if you plan to manage your project locally via the CLI. IT WILL NOT WORK AS EXPECTED WITH TERRAFORM CLOUD. Indeed, with the in Terraform Cloud, terraform.workspace will always return default 😠👿🤯.

Workspace overview

Workspaces allow to manage multiple variations of the same infrastructure (e.g., dev, prod). You'll manage the same .tf files, but there will be more than one .tfstate (one for each workspace). Those new .tfstate files are located under a new terraform.tfstate.d folder. You will also typically replace the terraform.tfvars with workspace specific .tfvars (e.g., dev.tfvars, prod.tfvars). Deploying your changes to an environment will change from executing a single command:

terraform apply

to execute 2 commands:

terraform workspace select dev
terraform apply -var-file=dev.tfvars

As of July 2021, it is not possible to explicitely link dev.tfvars to the dev workspace. We have to remember to apply the right tfvars to the right workspace.

To access the current workspace name in your .tf file:

output current_workspace {
	value = "The current workspace is ${terraform.workspace}"
}

The default workspace is "default".

WARNING!!!: terraform.workspace is only usefull if you plan to manage your project locally via the CLI. IT WILL NOT WORK AS EXPECTED WITH TERRAFORM CLOUD. Indeed, with the in Terraform Cloud, terraform.workspace will always return default 😠👿🤯.

Workspace commands

Command Description
terraform workspace list Lists all the workspaces.
terraform workspace new prod Creates a new workspace.
terraform workspace select dev Selects a workspace.
terraform workspace show Shows the current workspace.
terraform workspace delete prod Deletes a workspace.
terraform state push terraform.tfstate Migrate the terraform.tfstate into the current selected workspace.

Migrating the default workspace into another one

This situation happens when the first terraform draft was done is the default workspace. Later on, when that draft seems stable and the idea of having multiple environments arises, you may want to isolate those environments in their own workspaces (e.g., dev, prod). You may not want to destroy the default workspace and reprovision a new dev or prod. This section shows how to migrate the default worspace into a new dev workspace.

  1. Create a new workspace:
terraform workspace new dev
  1. Confirm dev is the current workspace:
terraform workspace show
  1. Add the default state to dev:
terraform state push terraform.tfstate
  1. Clean the deafult state (that's not required, but recommended to avoid human mistakes):
terraform workspace select default
terraform state list | xargs -L 1 terraform state rm
  1. Delete the terraform.tfstate in your root folder (i.e., the default workspace) if you know you won't need it anymore.

Backends

Usefull links:

A backend is an optional configuration (default is local). Terraform's backends are similar to Pulumi's backends. They primarily help storing an infrastructure's project's state. Contrary to Pulumi, some Terraform's backends can replace the CLI and run operations (though some could argue that Pulumi is capable of running operations without its CLI via its Automation API, but IMHO, that's a different use case). When a Terraform's backend can store the state as well as execute operations, it is referred to as an enhanced backend, otherwise, it is referred to as standard (can only store the state). Backends are listed below:

  • local (default - enhanced): This instructs the CLI that the state is stored on the local file system and that it has to run the operations.
  • remote(1) (enhanced): The most common remote backends are Terraform Cloud and Terraform Enterprise. They can be configured to store state and execute operations or to store state only (in which case the CLI executes the operations). When they execute the operations, the output is streamed back to the CLI.
  • S3(1) (standard)
  • Consul(1) (standard)
  • ... (full list of standard backends available at https://www.terraform.io/docs/language/settings/backends/index.html)

(1) As you can see, this is a bit confusing as S3 and Consul store the state remotely, and have nothing to do with the actual remote backend.

Providers

The Workspace conundrum

The workspace concept is used to manage multiple environments (e.g., dev, uat, prod). Unfortunately, this concept is implemented differently whether you're using it in the context of Terraform CLI or Terraform Cloud. This makes upgrading from using the CLI to using Terraform Cloud very confusing.

YOU MUST REMEMBER THE FOLLOWING:

  • Do not rely on the terraform.workspace in your logic if you plan to migrate to Terraform Cloud as this value is set to default regardless of the Terraform Cloud workspace.
  • Your branching strategy will most likely change depending on whether you're using Terraform CLI or Terraform Cloud. When managing a project via the Terraform CLI, you can have a single branch (e.g., master) with multiple workspaces (e.g., dev and prod), whereas with Terraform Cloud each workspace will most likely be assoaciated with a different branch. Let's consider a dev and prod environment. With the CLI, you would typically maintain a single Git master branch with two different workspaces called dev and prod. You would deploy to dev by switching to the dev workspace, and do something similar to deploy to prod. However, with Terraform Cloud, you would instead maintain two Git branches (e.g., master for prod and dev for dev). You would then connect the master and dev branch to Terraform Cloud workspaces respectively called my-project-prod and my-project-dev.

What is the difference between the Terraform CLI workspaces and the Terraform Cloud workspaces?

Terraform Cloud organizes each stack in workspaces that are not to be confused with the CLI workspaces. To clarify this idiotic conundrum, let's assume you started with a local project using your CLI using its default backend (i.e., local). Let's call this project my-website (this is important as this project name concept does not explicitely exist when you're coding locally, but it somewhat will when we're migrating to Terraform Cloud). Let's also imagine that you've configured two workpaces called dev and prod, as explained in the Workspaces section. With that setup, your project looks like a single directory whose state can be switched between dev and prod via the terraform workspace select <ENV> command. Once migrated to Terraform Cloud, this setup will look like 2 separate directories called my-website-dev and my-website-prod. As far as Terraform Cloud is concerned, these are two separate projects (i.e., Terraform Cloud workspaces), whereas Terraform CLI sees a single project with 2 different workspaces (i.e., Terraform CLI workspaces).

WARNING!!!: terraform.workspace is only usefull if you plan to manage your project locally via the CLI. IT WILL NOT WORK AS EXPECTED WITH TERRAFORM CLOUD. Indeed, with the in Terraform Cloud, terraform.workspace will always return default 😠👿🤯.

Commands

Command Description
terraform init Creates an empty .tfstate file ready to be used. Must be executed when a new project is created or when a new backend is configured.
terraform plan Preview the list of changes that could be applied.
terraform apply Applies the new configuration.
terraform apply -auto-approve Applies the new configuration with a confirmation prompt.
terraform apply -target=aws_security_group.my_sg Applies a single resource or module. For more details about this command, please refer to the How to deploy a specific resources instead of the entire project stack? section.
terraform refresh Sycn the terraform.tfstate with the real target infrastructure. For more details, please refer to the Dealing with drift section.
terraform import aws_security_group.rogue_sg sg-123456 Import a non-tracked resource in Terraform. For more details, please refer to the Dealing with drift section.
terraform state list Lists all the resources in the current workspace.
terraform state rm <RESOURCE NAME> Removes a resource name from the state. You can list them with terraform state list.
terraform workspace list Lists all the workspaces.
terraform workspace new prod Creates a new workspace.
terraform workspace select dev Selects a workspace.
terraform workspace show Shows the current workspace.
terraform workspace delete prod Deletes a workspace.
terraform state push terraform.tfstate Migrate the terraform.tfstate into the current selected workspace.

Functions

Official doc at https://www.terraform.io/docs/language/functions/index.html

file

Loads a local file as a string.

resource "aws_s3_bucket" "website_bucket" {
  bucket = "cloudlesslabs.com"
  acl    = "public-read"
  policy = file("public_website_policy.json")

  website {
    index_document = "index.html"
    error_document = "error.html"
  }

  versioning {
    enabled = true
  }
}

Can be mixed with the jsondecode to convert a JSON file:

locals {
  testJson = jsondecode(file("public_website_policy.json"))
}

output "test" {
  value = local.testJson.Version
}

Dealing with drift

Drift is what happens when your real infrastructure is in a different state than your terraform.tfstate. This happens quite often for a various reasons. Terraform offers a few techniques to deal and reason about your infrastructure drift:

  • The refresh command: This command helps adjusting resources that were initially provisioned by Terraform, but were modified outside of it.
  • The import command: This command helps importing into Terraform a resource that was provisoned outside of it.

terraform refresh

terraform refresh is actually run before any terraform plan or terraform apply.

This command helps adjusting resources that were initially provisioned by Terraform, but were modified outside of it. refresh does not change your infrastructure, so it is perfectly fine to execute it. It may update the terraform.tfstate though. Indeed, this command uses that file to query its real resources and check for discrepancies. If there are any, Terraform updates the terraform.tfstate to match the real state. To avoid data loss, a terraform.tfstate.backup is created. That file is a copy of the terraform.tfstate before any changes are applied to it. This file is also very useful to discover what changed:

git diff --no-index terraform.tfstate terraform.tfstate.backup

The --no-index option is required because both the terraform.tfstate and the terraform.tfstate.backup are not supposed to be tracked by Git. Please refer to the Put your project under source control and ignore critical files section for more details about this topic.

terraform import <tf-resource> <real resource ID>

This command helps importing into Terraform a resource that was provisoned outside of it. Imagine the case where a new security group was manually created in the AWS console.

  1. Create that security group in your Terraform project:
resource "aws_security_group" "rogue_sg" {
	...
}
  1. Get the new SG ID from the AWS Console.
  2. Import the new SG in Terraform using the SG ID (e.g., 'sg-123456'):
terraform import aws_security_group.rogue_sg sg-123456

Terraform Cloud

Terraform Cloud Overview

Terraform Cloud is the most common remote backend categorized as an enhanced backend (as opposed to standard), which means it can store a Terraform state as well as run operations (the other enhanced backend is the default local). The problem that Terraform Cloud tackles is collaboration and governance.

The issue with Terraform Cloud is its workspace implementation, which differs from the normal Terraform CLI workspaces. For more details about this topic, please refer to the The Workspace conundrum section.

Migrating your local workspaces to Terraform Cloud

IMPORTANT PREREQUISITE: Replace your terraform.workspace with a variable that explicitely reference your workspace (e.g., var.env). Indeed, in Terraform Cloud, terraform.workspace is always set to default.

  1. Create an account or login at https://www.terraform.io/cloud.
  2. Create or select an organization.
  3. Connect your local CLI to your Terraform Cloud account (1):
    1. Execute this command:
    terraform login
    1. Copy the API token from your browser and paste it in your CLI.
  4. Update the terraform block by addin a new backend block as follow (refer to (2) for the prefix explanation):
terraform {
  ...
  backend "remote" {
    hostname = "app.terraform.io"
    organization = "my-org"

    workspaces {
      # name = "my-app" # Use 'name' instead of 'prefix' if your only using the default workspace. 
      prefix = "my-app-"
    }
  }
}
  1. Run the following command to migrate your workspaces to Terraform Cloud (refer to (3) if you wish to not migrate the default workspace):
terraform init
  1. (Optional - Only needed if you were using .tfvars files). For each new Terraform Cloud Workspace, manually create all the variables (i.e., the ones you may have stored under files such as dev.tfvars, prod.tfvars):
    1. Login to your Terraform Cloud account: https://www.terraform.io/cloud.
    2. Select the correct organization.
    3. Select the workspace.
    4. Click on the Variables tab.
    5. Click the Add variable button under the Terraform Variables section.
  2. For each new Terraform Cloud Workspace, manually create all the environment variables the stack requires (most likely the AWS or GCP credentials (e.g., AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)):
    1. Login to your Terraform Cloud account.
    2. Select the correct organization.
    3. Select the workspace.
    4. Click on the Variables tab.
    5. Click the Add variable button under the Environment Variables section (IMPORTANT: For each sensitive variable, don't forget to tick the sensitive tick box to ensure no sensitive information can leak).
  3. Test with terraform plan (4)

Now that all the states and variables have been migrated to Terrafform Cloud, the following local files and folders can be deleted:

  • terraform.tfstate file (and its backups)
  • terraform.tfstate.d folder
  • *.tfvars

(1) Those credentials will be stored under /Users//.terraform.d/credentials.tfrc.json. (2) prefix is used when your local project is configured with multiple CLI workspaces. Those workspaces will be exported to Terraform in their own Terraform Cloud workspace. For example, if the project defines two workspaces called dev and prod, then the configuration above will create two Terraform Cloud workspaces called respectively my-app-dev and my-app-prod. (3) If you've stopped using the default workspace, and therefore wish to not migrate it to Terraform Cloud, then delete the terraform.tfstate in the root. (4) There is no more needs to pass the -var-file="some.tfvars" option (this would fail) as those variables should have been migrated in step #6.

Interacting with Terraform Cloud via the CLI

Swicthing between workspaces

This is done the same way as if the workspace was still maintained via the CLI:

terraform workspace select prod

It makes sense and at the same time it doesn't. As explained earlier, workspaces mean something different whether you are using the CLI or Terraform Cloud. So how could this work? It works because the backend block has been configured with the prefix property instead of name in step #4 above. That's how the Terraform CLI knows how to map the CLI workspace with the Terraform Cloud workspace.

Synching a local project with a Terraform Cloud workspace

  1. Select a workspace:
terrafrom workspace select dev
  1. Run the usual plan or apply:
terraform apply

Git workflows and dev lifecycles

As mentionned earlier, the Git workflows and dev lifecycles depend on what backend you're using. This section details the main two options:

Using Terraform CLI to deploy your stack

  1. Create a new project.
  2. Put it under source-control and add a .gitignore similar to:
.terraform/
terraform.tfstate
terraform.tfstate.backup
terraform.tfstate.*.backup

Add *.tfvars if those files contain sensitive pieces of information.

  1. Create one workspace per environment:
terraform workspace new dev

Using Terraform Cloud to deploy your stack

  1. Login to your Terraform Cloud account: https://www.terraform.io/cloud
  2. Select or create an organization (e.g., my-org).
  3. Create a new project in your local machine, and set it up with the following backend:
terraform {
  ...
  backend "remote" {
    hostname = "app.terraform.io"
    organization = "my-org"

    workspaces {
      prefix = "my-app-"
    }
  }
}

IMHO, prefix is better than name because you will most likely need workspaces to support multiple environments.

  1. Put it under source-control and add a .gitignore similar to:
.terraform/
terraform.tfstate
terraform.tfstate.backup
terraform.tfstate.*.backup
  1. Commit your project and push it to your VCS (e.g., github.com).
  2. Log back in to Terraform Cloud account: https://www.terraform.io/cloud and select your organization (e.g., my-org).
  3. Click on the New workspace button and select the Version control workflow option.
  4. Connect to your VCS and select the repository you created in step #5.
  5. Name your workspace using the prefix used in step #3 and the environment name you wish to associated to your branch (e.g., my-app-prod is prod is associated with master).
  6. Click on the Create workspace button.
  7. Optionally, configure that workspace to automatically apply changes after a new commit (or PR merged into that branch).
    • Select your new workspace.
    • In the right menu, click on the Auto apply to switch it from Off to On.

With this setup, each time you deploy to that branch, Terraform Cloud will try to apply your changes.

CDK - Cloud Developmemt Kit

CDK is an attempt to compete with Pulumi and replace the native HCL, aka HashiCorp Configuration Language, with a programmatic language. As of June 2021, The Terraform's CDK only support Typeform and Python.

Troubleshooting

The true and false result expressions must have consistent types

This happens when doing something similar to this:

output "dns_challenge" {
  value       = local.is_prod ? aws_acm_certificate.cert[0].domain_validation_options : "No DNS challenges in non-prod environment."
}

The ternary operator is not allowed to return different types.

The hack is to parse the returned values to the same type:

output "dns_challenge" {
  value       = local.is_prod ? jsonencode(aws_acm_certificate.cert[0].domain_validation_options) : jsonencode({})
}

or use null

output "dns_challenge" {
  value       = local.is_prod ? aws_acm_certificate.cert[0].domain_validation_options : null
}

FAQ

How to debug?

export TF_LOG=trace && terraform apply

TF_LOG values can be:

  • TRACE: This is the most verbose. IMHO, that's the one you should be using when things go wrong.
  • DEBUG: This usually flags the unexpected errors.
  • INFO: This is basic info.
  • WARN: This warns about configurations or states that are not severe but could be harmful.
  • ERROR: This flags explicit errors in your configuration or system.

To disable, the debug mode, simply clear the TF_LOG variable:

unset TF_LOG

How to delete a resource?

Simply remove it from the .tf file, then run terraform apply.

How to restrict a provider to a specific account ID?

Please refer to the Restricting a provider to a specific account ID section.

How to load a JSON file to access its properties?

locals {
  testJson = jsondecode(file("public_website_policy.json"))
}

output "test" {
  value = local.testJson.Version
}

How to list the details of a provisioned resource?

terraform state show aws_s3_bucket.www | grep website_endpoint

How to deploy a specific resources instead of the entire project stack?

To deploy a single resource:

terraform apply -target=aws_security_group.my_sg

You can also specify more than one resource:

terraform apply -target=aws_security_group.my_sg -target=aws_security_group.my_2nd_sg

If deploying many SGs is too tedious, you can choose between:

  • Deploying a resource that depends on those SGs, as the -target option respects dependencies:
terraform apply -target=aws_instance.my_server
  • Encapsulate the SGs in a module, and deploy that module only, as the -target also supports modules:
terraform apply -target=module.my_security_groups

How to migrate a workspace to another?

Please refer to the Migrating the default workspace into another one section.

How to remove all resources from the state?

terraform state list | xargs -L 1 terraform state rm

Where are my Terraform Cloud credentials stored on my local machine?

On Mac, they are stored under /Users/<YOUR USER>/.terraform.d/credentials.tfrc.json.

Annexes

Regions

AWS regions

Region Name Region
US East (Ohio) us-east-2
US East (N. Virginia) us-east-1
US West (N. California) us-west-1
US West (Oregon) us-west-2
Africa (Cape Town) af-south-1
Asia Pacific (Hong Kong) ap-east-1
Asia Pacific (Mumbai) ap-south-1
Asia Pacific (Osaka-Local) ap-northeast-3
Asia Pacific (Seoul) ap-northeast-2
Asia Pacific (Singapore) ap-southeast-1
Asia Pacific (Sydney) ap-southeast-2
Asia Pacific (Tokyo) ap-northeast-1
Canada (Central) ca-central-1
China (Beijing) cn-north-1
China (Ningxia) cn-northwest-1
EU (Frankfurt) eu-central-1
EU (Ireland) eu-west-1
EU (London) eu-west-2
EU (Milan) eu-south-1
EU (Paris) eu-west-3
EU (Stockholm) eu-north-1
Middle East (Bahrain) me-south-1
South America (Sao Paulo) sa-east-1
AWS GovCloud (US-East) us-gov-east-1
AWS GovCloud (US-West) us-gov-west-1

Google Cloud regions

Region Name Region Availability zones
asia-east1 Taiwan a,b,c
asia-east2 Hong Kong a,b,c
asia-northeast1 Tokyo a,b,c
asia-northeast2 Osaka a,b,c
asia-northeast3 Seoul a,b,c
asia-south1 Mumbai a,b,c
asia-south2 Delhi a,b,c
asia-southeast1 Singapore a,b,c
asia-southeast2 Jakarta a,b,c
australia-southeast1 Sydney a,b,c
australia-southeast2 Melbourne a,b,c
europe-central2 Poland a,b,c
europe-north1 Hamina, Finland, Europe a,b,c
europe-west1 St. Ghislain, Belgium, Europe a,b,c
europe-west2 London, England, Europe a,b,c
europe-west3 Frankfurt, Germany Europe a,b,c
europe-west4 Eemshaven, Netherlands, Europe a,b,c
europe-west6 Zurich, Switzerland, Europe a,b,c
northamerica-northeast1 Montréal, Québec, North America a,b,c
southamerica-east1 Osasco, São Paulo, Brazil, South America a,b,c
us-central1 Council Bluffs, Iowa, North America a,b,c,f
us-east1 Moncks Corner, South Carolina, North America b,c,d
us-east4 Ashburn, Virginia, North America a,b,c
us-west1 The Dalles, Oregon, North America a,b,c
us-west2 Los Angeles, California, North America a,b,c
us-west3 Salt Lake City, Utah, North America a,b,c
us-west4 Las Vegas, Nevada, North America a,b,c

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment