Terraform Cloud (i.e. remote backend) TF_VAR_ environment substitutions not working?

Question

Terraform version: 0.12.24

This is really weird because I have used the TF_VAR_ substitution syntax before and it has worked fine.

provider.tf

# Configure the AWS Provider
provider "aws" {
  version = "~> 2.0"
  region  = "ap-southeast-2"
  access_key = var.aws_access_key_id
  secret_key = var.aws_secret_access_key
}

vars.tf

variable "aws_access_key_id" {
  description = "Access Key for AWS IAM User"
}

variable "aws_secret_access_key" {
  description = "Secret Access Key for AWS IAM User"
}

variable "terraform_cloud_token" {
  description = "Token used to log into Terraform Cloud via the CLI"
}

backend.tf for terraform cloud

terraform {
  backend "remote" {
    organization = "xx"

    workspaces {
      name = "xx"
    }
  }
}

Build logs

---------------
TF_VAR_aws_secret_access_key=***
TF_VAR_aws_access_key_id=***
TF_VAR_terraform_cloud_token=***
---------------

It also fails locally when I try to run this in a local Docker Container

Dockerfile

FROM hashicorp/terraform:0.12.24

COPY . /app

COPY .terraformrc $HOME

ENV TF_VAR_aws_secret_access_key 'XX'
ENV TF_VAR_aws_access_key_id 'XX'
ENV TF_VAR_terraform_cloud_token 'XX'

WORKDIR /app

ENTRYPOINT ["/app/.github/actions/terraform-plan/entrypoint.sh"]

entrypoint.sh

#!/bin/sh -l

# move terraform cloud configuration file to user root as expected
# by the backend resource
mv ./.terraformrc ~/

terraform init
terraform plan

output from docker container run

$ docker run -it tf-test
---------------
TF_VAR_aws_secret_access_key=XX
TF_VAR_aws_access_key_id=XX
TF_VAR_terraform_cloud_token=XX
---------------

Initializing the backend...

Successfully configured the backend "remote"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "aws" (hashicorp/aws) 2.56.0...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Running plan in the remote backend. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.

Preparing the remote plan...

To view this run in a browser, visit:
https://app.terraform.io/app/XX/XX/runs/run-XX

Waiting for the plan to start...

Terraform v0.12.24
Configuring remote state backend...
Initializing Terraform configuration...
2020/04/03 01:43:04 [DEBUG] Using modified User-Agent: Terraform/0.12.24 TFC/05d5abc3eb

Error: No value for required variable

  on vars.tf line 1:
   1: variable "aws_access_key_id" {

The root module input variable "aws_access_key_id" is not set, and has no
default value. Use a -var or -var-file command line argument to provide a
value for this variable.


Error: No value for required variable

  on vars.tf line 5:
   5: variable "aws_secret_access_key" {

The root module input variable "aws_secret_access_key" is not set, and has no
default value. Use a -var or -var-file command line argument to provide a
value for this variable.


Error: No value for required variable

  on vars.tf line 9:
   9: variable "terraform_cloud_token" {

The root module input variable "terraform_cloud_token" is not set, and has no
default value. Use a -var or -var-file command line argument to provide a
value for this variable.

Answer 1

Okay... it is confusing because the logs generated in Terraform's VMs are streamed to your own terminal/run logs.

But this is what I found out. There are two options available to you when you use Terraform Cloud.

1. Use Terraform's VMs to run your terraform commands
2. Use your own (or your CI/CD platform's) infrastructure to run those terraform commands.

If you choose the first option (which is annoyingly the default)... you must set your environment variables within the Terraform Cloud Dashboard. This is because all terraform commands for this execution type are run in their VMs and the environment variables in your local environment, for good security reasons, aren't passed through to Terraform.

If you have the remote option selected, once you do this, it will work as expected.