Terraform environment specific variables

Question

Anyone know if there's a way to populate variables in Terraform based on what the environment/workspace is? Preferably one that

• populates the var namespace (ie not an external data source),
• doesn't require a wrapper
  • like tf(){ terraform --var-file=$(get_tf_env).tfvars
• takes effect by changing a terraform env/workspace, without any additional manual steps (ie steps that aren't triggered by running terraform env )?

Answer 1

You can dynamically load variables in Terraform based on the workspace using terraform.workspace without external scripts or wrappers. Here's how:

Create Workspace-Specific Variable Files: Example files: dev.tfvars, prod.tfvars, etc.

Use the Workspace to Load Variables:

terraform apply -var-file="${terraform.workspace}.tfvars"

Alternatively, use conditional logic in main.tf:

locals { instance_type = terraform.workspace == "prod" ? "t3.large" : "t2.micro" }

This auto-loads variables when switching workspaces (terraform workspace select dev).

Answer 2

My reply is an addition to @intotecho one but a little bit more primitive approach of using terraform.workspace which in most cases is enough. And also similar to the reply of @AlexT but with more verbose explanation to it to help people who is not familiar with terraform workspace approaches.

The main idea is to have your workspace name (which you provide and create yourself, see last paragraph) in the env local variable so then you can re-use it later. And you run all your terraform commands based on workspace you're currently in. See code snipped below:

locals {
  # specify workspace name as `env` local variable:
  env = terraform.workspace

  # some other variables
  octopus_deploy_instance = {
    test  = "test_octopus"
    dev = "dev_octopus"
  }
  # access to the related octopus based on env value (current tf workspace)
  octopus_id = local.octopus_deploy_instance[local.env]

  # and you can create other local vars based on the env from workspace name:
  paltform_prefix = "platform-${local.env}-deployment"
}

So, local.env will contain the current workspace name, and local.octopus_id will contain the corresponding Octopus Deploy instance name based on the workspace. Similarly, local.platform_prefix will provide the platform prefix based on the workspace.

Remember to switch to the correct workspace before applying or any commands.

cd <your terraform root folder>
terraform init
terraform workspace select <your terraform workspace name>
terraform validate
terraform plan <any parameters you would like to use>
terraform apply # if necessary

You always can see list of available (already created) workspaces via terraform workspace list and as well as add new (terraform workspace new) or see more details with terraform workspace --help or even ``terraform workspace new --help`

This all should be relevant for terraform v1.5.6 or higher (at least I tested it with such version)

Answer 3

I use terraform workspace and created a bash script to echo the --var-file argument.

#!/bin/bash

echo --var-file=variables/$(terraform workspace show).tfvars

To run terraform with tfvars applied by workspace

terraform plan $(./var.sh)
terraform apply $(./var.sh)

Answer 4

Handling environmental variables in Terraform Workspaces -Taking Advantage of Workspaces, by Miles Collier 2019 explains clearly how this works. This is just a summary.

In parameters.tf:

locals {
   env = {
      default = {
         instance_type  = "t2.micro"
         ami            = "ami-0ff8a91507f77f867"
         instance_count = 1
      }
      dev = {
         instance_type  = "m5.2xlarge"
         ami            = "ami-0130c3a072f3832ff"
      }
      qa = {
         instance_type  = "m5.2xlarge"
         ami            = "ami-00f0abdef923519b0"
         instance_count = 3
      }
      prod = {
         instance_type  = "c5.4xlarge"
         ami            = "ami-0422d936d535c63b1"
         instance_count = 6
      }
   }
environmentvars = "${contains(keys(local.env), terraform.workspace)}" ? terraform.workspace : "default"
   workspace       = "${merge(local.env["default"], local.env[local.environmentvars])}"
}

To reference a variable, add to locals or pass this to a module:

instance_type = "${local.workspace["instance_type"]}"

This will use the value from the selected workspace, or the default value if either the variable is not defined for that workspace, or no workspace is selected. If not default is defined, it fails gracefully.

Use terraform workspace select dev to select the dev workspace.

Answer 5

Populates the var namespace, doesn't require a wrapper, and takes effect by changing the workspace (Terraform 0.12 code):

variable "ami_id" {
  type = map(string)

  default = {
    stg = "ami-foo28929"
    prd = "ami-bar39b12"
  }
}

resource "aws_instance" "this" {
  ami = var.ami_id[terraform.workspace]
  (...)
}

Answer 6

Terraform workspaces

A workspace is a named container for Terraform state. With multiple workspaces, a single directory of Terraform configuration can be used to manage multiple distinct sets of infrastructure resources.

In the 0.9 line of Terraform releases, this concept was known as "environment". It was renamed in 0.10 based on feedback about confusion caused by the overloading of the word "environment" both within Terraform itself and within organizations that use Terraform.

Referencing the current workspace is useful for changing behavior based on the workspace. For example, for non-default workspaces, it may be useful to spin up smaller cluster sizes. For example:

resource "aws_instance" "example" {
  count = "${terraform.workspace == "default" ? var.default : var.min}"

  # ... other arguments
}

Answer 7

I would recommend taking a "Stacks" based approach for your Terraform Project so that you can configure and manage the "Stacks" and the Remote State per Workspace (aka Environment). This limits the blast radius of your changes from a risk perspective, simplifies workflow, and also provides for a cleaner more maintainable code base.

What will make your day better?

• An objectively simple design that allows you to reason about the platform and its moving parts. (aka Stacks)
• An implementation that provides you with flexibility while limiting risk from changes. (aka Limit the Blast Radius)
• A solution that delivers value today and continues to improve while building momentum over the long haul. (aka Patterns, Workflow)

Here is a quick list of good practices

• Manage "State" separately for "Stacks" across "Workspaces"

• Implement "Stacks" for consistent "Configuration" across "Workspaces"

• Keep it objective and simple with good "Patterns" and "Workflow".

Example Terraform Project using a Stacks Based Approach

/
  /scripts
     <shell scripts>
     <terraform wrapper functions>

  /stacks
    /application_1   # Provisions Application 1 and its dependencies
    /application_2   # Provisions Application 2 and its dependencies
    /application_n   # Provisions Application N and its dependencies
        backend.tf       # Remote State
        data.tf          # Data Sources
        stack.tf         # Stack Variables and Defaults
        aws_resource.tf 
        ...
        ...    
    /network    # Provisions VPC, Subnets, Route Tables, Route53 Zones
    /security   # Provisions Security Groups, Network ACLs, IAM Resources
    /storage    # Provisions Storage Resources like S3, EFS, CDN

  global.tf     # Global Variables
  dev.tfvars    # Development Environment Variables
  tst.tfvars    # Testing Environment Variables
  stg.tfvars    # Staging Environment Variables
  prd.tfvars    # Production Environment Variables
  terraform.sh  # Wrapper Script for Executing Terraform (Workflow)

A few more thoughts

As your implementation grows it is much simpler to incorporate future requirements into existing stacks or as new stacks if they are a shared dependency.

Terraform allows for use of the Remote State as a Data Source. Configuring your own Output Variables per stack makes it much cleaner to configure and use exported resource attributes.

Setting up your project so that you can define variables and reasonable defaults at the stack level allows you to override them at the workspace level as necessary to meet requirements for different environments such as Dev, Test, Production, etc.... while keeping the configuration consistent and remote state managed separately per environment.

These are some of the practices we have developed and deployed on our team to improve our experience working with Terraform to manage our AWS Platform.

Cheers!

Answer 8

There isn't a native way of doing it with Terraform that I know of. If you search around you will see that a lot of people will have different folder structures for entry points into their TF configurations, each different folder can have different values in tfvars file. One options that may get you some of the way there is to use Terraform Workspaces, introduced in 0.10.

I've implemented something similar to what you are suggesting using OctopusDeploy. If you've not used it before, Octopus is good for managing environment specific variables. I have a default tfvars file and a list of corresponding variable values within Octopus, per environment.

I have a basic step that iterates through every variable in tfvars and looks for an Octopus variable with the same name and replaces it if it is found.

I've found this to be a decent way of working as it gives a nice separation between the Terraform tfvars file (what values are needed) and the variable values in Octopus (what the actual values are).

E.g. If I have the a tfvars file containing

instance_size = "Medium"

And I have 2 environments within Octopus, Staging and Production. I can add a variable to Octopus called 'instance_size' and set a different value per environment (e.g. "Big" and "Biggest" respectively).

The step template I've written will find a corrresponding value for "instance_size" so it means that when I run it for staging I would get:

instance_size = "Big"

and for production

instance_size = "Biggest"