CODEWITHSUNDEEP
variablesenvironment-variablesterraformterraform-provider-azure
Mert Alnuaimi
Mert Alnuaimi

Reputation: 362

Setting Terraform env variables correctly for Azure resources

I have a GitHub Actions workflow that includes this part for Terraform;

  terraform:
    name: 'Terraform'
    needs: build
    runs-on: ubuntu-latest

    # Add env variables for service principal
    env:
      TF_LOG: TRACE
      ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID }}
      ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET }}
      ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID }}
      ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID }}
    steps:
      - name: 'Checkout'
        uses: actions/checkout@v2
      - name: 'Azure CLI'
        uses: azure/login@v1
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}
      - run: |
          az account show
      - name: 'Terraform Setup'
        uses: hashicorp/setup-terraform@v1
        with:
          cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}
      - name: 'Terraform Init'
        run: terraform init 
      - name: 'Terraform Format'
        run: terraform fmt -check 
      - name: 'Terraform Plan'
        run: terraform plan 
      - name: Terraform Apply
        if: github.ref == 'refs/heads/develop' && github.event_name == 'push'
        run: terraform apply -auto-approve

And I have the following .tf files;

main.tf

provider "azurerm" {
  version         = "=2.5.0"
  features {}
}

variables.tf

variable "subscription_id" {
  description = "The Azure subscription ID."
}
variable "client_id" {
  description = "The Azure Service Principal app ID."
}
variable "client_secret" {
  description = "The Azure Service Principal password."
}

variable "tenant_id" {
  description = "The Azure Tenant ID."
}

terraform.tfvars

subscription_id = "2d0bd.."
client_id       = "hl4kj..."
client_secret   = "kj2ee..."
tenant_id       = "f9cc2..."

And I also have the following secrets set in secrets section of the repository;

ARM_CLIENT_ID
ARM_CLIENT_SECRET
ARM_SUBSCRIPTION_ID
ARM_TENANT_ID

When I run the workflow I get the following log and error, terraform plan gets stuck;

/home/runner/work/_temp/cd8bfc2c-354b-41a4-9d10-f1ed7183c439/terraform-bin plan ./terraform
2020/11/05 13:49:02 [INFO] Terraform version: 0.13.5  
2020/11/05 13:49:02 [INFO] Go runtime version: go1.14.7
2020/11/05 13:49:02 [INFO] CLI args: []string***"/home/runner/work/_temp/cd8bfc2c-354b-41a4-9d10-f1ed7183c439/terraform-bin", "plan", "./terraform"***
2020/11/05 13:49:02 [DEBUG] Attempting to open CLI config file: /home/runner/.terraformrc
2020/11/05 13:49:02 Loading CLI configuration from /home/runner/.terraformrc
2020/11/05 13:49:02 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2020/11/05 13:49:02 [DEBUG] ignoring non-existing provider search directory /home/runner/.terraform.d/plugins
2020/11/05 13:49:02 [DEBUG] ignoring non-existing provider search directory /home/runner/.local/share/terraform/plugins
2020/11/05 13:49:02 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2020/11/05 13:49:02 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2020/11/05 13:49:02 [INFO] CLI command args: []string***"plan", "./terraform"***
2020/11/05 13:49:02 [TRACE] Meta.Backend: built configuration for "azurerm" backend with hash value 4172574508
2020/11/05 13:49:02 [TRACE] Preserving existing state lineage "2214372f-9818-d87e-197a-ef8533e8fa6e"
2020/11/05 13:49:02 [TRACE] Preserving existing state lineage "2214372f-9818-d87e-197a-ef8533e8fa6e"
2020/11/05 13:49:02 [TRACE] Meta.Backend: working directory was previously initialized for "azurerm" backend
2020/11/05 13:49:02 [TRACE] Meta.Backend: using already-initialized, unchanged "azurerm" backend configuration
2020/11/05 13:49:02 [TRACE] Meta.Backend: instantiated backend of type *azure.Backend
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
2020/11/05 13:49:02 [TRACE] getproviders.SearchLocalDirectory: .terraform/plugins is a symlink to .terraform/plugins
2020/11/05 13:49:02 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/azuread v0.7.0 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/azuread/0.7.0/linux_amd64
2020/11/05 13:49:02 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/azurerm v2.5.0 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/azurerm/2.5.0/linux_amd64
2020/11/05 13:49:02 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/helm v1.3.2 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/helm/1.3.2/linux_amd64
2020/11/05 13:49:02 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/random v3.0.0 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/random/3.0.0/linux_amd64
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/random/3.0.0/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/random 3.0.0
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/azuread/0.7.0/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/azuread 0.7.0
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/azurerm/2.5.0/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/azurerm 2.5.0
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/helm/1.3.2/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/helm 1.3.2
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
2020/11/05 13:49:02 [DEBUG] checking for provisioner in "."
2020/11/05 13:49:02 [DEBUG] checking for provisioner in "/home/runner/work/_temp/cd8bfc2c-354b-41a4-9d10-f1ed7183c439"
2020/11/05 13:49:02 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
2020/11/05 13:49:02 [TRACE] Meta.Backend: backend *azure.Backend does not support operations, so wrapping it in a local backend
2020/11/05 13:49:02 [INFO] backend/local: starting Plan operation
2020/11/05 13:49:02 [TRACE] backend/local: requesting state manager for workspace "default"
2020/11/05 13:49:02 [TRACE] backend/local: requesting state lock for workspace "default"
2020/11/05 13:49:02 [DEBUG] Azure Backend Request: 
HEAD /tstate/terraform.tfstate HTTP/1.1
Host: tstateidentity15466.blob.core.windows.net
User-Agent: Terraform/0.13.5
X-Ms-Date: Thu, 05 Nov 2020 13:49:02 GMT
X-Ms-Version: 2018-11-09


Acquiring state lock. This may take a few moments...
2020/11/05 13:49:03 [DEBUG] Azure Backend Response for https://tstateproject.blob.core.windows.net/tstate/terraform.tfstate: 
HTTP/1.1 200 OK
Content-Length: 978
Accept-Ranges: bytes
Content-Md5: qi87ZYbc9/fceVy/LIgnjQ==
Content-Type: application/json
Date: Thu, 05 Nov 2020 13:49:02 GMT
Etag: "0x8D881909C5B3223"
Last-Modified: Thu, 05 Nov 2020 13:42:17 GMT
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
X-Ms-Access-Tier: Hot
X-Ms-Access-Tier-Inferred: true
X-Ms-Blob-Type: BlockBlob
X-Ms-Creation-Time: Tue, 20 Oct 2020 11:48:51 GMT
X-Ms-Lease-State: broken
X-Ms-Lease-Status: unlocked
X-Ms-Meta-Terraformlockid: eyJJRCI6ImY3YWI2MTVlLTQ2MjItMDY5Yy00YjFiLWM4ZGNkM2ZiODg4ZiIsIk9wZXJhdGlvbiI6Ik9wZXJhdGlvblR5cGVQbGFuIiwiSW5mbyI61bm5lckBmdi1hejEyMC0yMjQiLCJWZXJzaW9uIjoiMC4xMy41IiwiQ3JlYXRlZCI6IjIwMjAtMTEtMDVUMTM6NDI6MTYuNDkyMjUzOTiJ0c3RhdGUvdGVycmFmb3JtLnRmc3RhdGUi***
X-Ms-Request-Id: b9a10809-d01e-002f-5a-b3a500000
X-Ms-Server-Encrypted: true
X-Ms-Version: 2018-11-09


2020/11/05 13:49:03 [DEBUG] Azure Backend Request: 
PUT /tstate/terraform.tfstate?comp=lease HTTP/1.1
Host: tstateproject.blob.core.windows.net
User-Agent: Terraform/0.13.5
Content-Length: 0
X-Ms-Date: Thu, 05 Nov 2020 13:49:03 GMT
X-Ms-Lease-Action: acquire
X-Ms-Lease-Duration: -1
X-Ms-Proposed-Lease-Id: 99a99396-3a95-215-693d-023e7f07f
X-Ms-Version: 2018-11-09
Accept-Encoding: gzip


2020/11/05 13:49:03 [DEBUG] Azure Backend Response for https://tstateproject.blob.core.windows.net/tstate/terraform.tfstate?comp=lease: 
HTTP/1.1 201 Created
Content-Length: 0
Date: Thu, 05 Nov 2020 13:49:02 GMT
Etag: "0x8D881909C5B3223"
Last-Modified: Thu, 05 Nov 2020 13:42:17 GMT
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
X-Ms-Lease-Id: 99a99396-3a95-2175-693d-01e823f07f
X-Ms-Request-Id: b9a10885-d01e-002f-3c7a-b3a5000000
X-Ms-Version: 2018-11-09


2020/11/05 13:49:03 [DEBUG] Azure Backend Request: 
HEAD /tstate/terraform.tfstate HTTP/1.1
Host: tstateproject.blob.core.windows.net
User-Agent: Terraform/0.13.5
X-Ms-Date: Thu, 05 Nov 2020 13:49:03 GMT
X-Ms-Lease-Id: 99a99396-3a95-2175-693d-0823e7f07f
X-Ms-Version: 2018-11-09


2020/11/05 13:49:03 [DEBUG] Azure Backend Response for https://tstateproject.blob.core.windows.net/tstate/terraform.tfstate: 
HTTP/1.1 200 OK
Content-Length: 978
Accept-Ranges: bytes
Content-Md5: qi87ZYbc9/fceVy/LIgnjQ==
Content-Type: application/json
Date: Thu, 05 Nov 2020 13:49:02 GMT
Etag: "0x8D881909C5B3223"
Last-Modified: Thu, 05 Nov 2020 13:42:17 GMT
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
X-Ms-Access-Tier: Hot
X-Ms-Access-Tier-Inferred: true
X-Ms-Blob-Type: BlockBlob
X-Ms-Creation-Time: Tue, 20 Oct 2020 11:48:51 GMT
X-Ms-Lease-Duration: infinite
X-Ms-Lease-State: leased
X-Ms-Lease-Status: locked
X-Ms-Meta-Terraformlockid: eyJJRCI6ImY3YWI2MTVlLTQ2MjItMY5Yy00YjFiLWM4ZGNkM2ZiODg4ZiIsIk9wZXJhdGlvbiI6Ik9wZXJhdGlvblR5cGVQbGFuIiwiSW5mbyI6IinJ1bm5lckBmdi1hejEyMC0yMjQiLCJWZXJzaW9uIjoiMC4xMy41IiwiQ3JlYXRlZCI6IjIwMjAtMTEtMDVUMTM6NDI6MTYuNDkyMjUzOTk5WiIsIlBhdGgiOiJ0c3RhdGUvdGycmFmb3JtLnRmc3RhdGUi***
X-Ms-Request-Id: b9a108f2-d01e-002f-187a-b3a24000000
X-Ms-Server-Encrypted: true
X-Ms-Version: 2018-11-09


2020/11/05 13:49:03 [DEBUG] Azure Backend Request: 
PUT /tstate/terraform.tfstate?comp=metadata HTTP/1.1
Host: tstateproject.blob.core.windows.net
User-Agent: Terraform/0.13.5
Content-Length: 0
X-Ms-Date: Thu, 05 Nov 2020 13:49:03 GMT
X-Ms-Lease-Id: 99a99396-3a95-2175-693d-01e823e7f07f
X-Ms-Meta-Terraformlockid: eyJJRCI6Ijk5YTk5Mzk2LTNhOTUtMjE3NS02OTNkLTAxZTgyM2U3ZjA3ZiIsIk9wZXJhdGlvbiI6Ik9wZXJhdGlvblR5cGVQbGFuIiwiSW5mbyI6IiIsIlJ1bm5lckBmdi1hejE3NC0yMTciLCJWZXJzaW9uIjoiMC4xMy41IiwiQ3JlYXRlZCI6IjIwMjAtMTEtMDVUMTM6NDk6MDIuNzgzNDQI5WiIsIlBhdGgiOiJ0c3RhdGUvdGVycmFmb3JtLnRmc3RhdGUi***
X-Ms-Version: 2018-11-09
Accept-Encoding: gzip


2020/11/05 13:49:03 [DEBUG] Azure Backend Response for https://tstateproject.blob.core.windows.net/tstate/terraform.tfstate?comp=metadata: 
HTTP/1.1 200 OK
Content-Length: 0
Date: Thu, 05 Nov 2020 13:49:02 GMT
Etag: "0x8D881918E9DEEFF"
Last-Modified: Thu, 05 Nov 2020 13:49:03 GMT
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
X-Ms-Request-Id: b9a10962-d01e-002f-707a-b524000000
X-Ms-Request-Server-Encrypted: true
X-Ms-Version: 2018-11-09


2020/11/05 13:49:03 [TRACE] backend/local: reading remote state for workspace "default"
2020/11/05 13:49:03 [DEBUG] Azure Backend Request: 
GET /tstate/terraform.tfstate HTTP/1.1
Host: tstateproject.blob.core.windows.net
User-Agent: Terraform/0.13.5
X-Ms-Date: Thu, 05 Nov 2020 13:49:03 GMT
X-Ms-Version: 2018-11-09
Accept-Encoding: gzip


2020/11/05 13:49:03 [DEBUG] Azure Backend Response for https://tstateproject.blob.core.windows.net/tstate/terraform.tfstate: 
HTTP/1.1 200 OK
Content-Length: 978
Accept-Ranges: bytes
Content-Md5: qi87ZYbc9/fceVy/LIgnjQ==
Content-Type: application/json
Date: Thu, 05 Nov 2020 13:49:02 GMT
Etag: "0x8D881918E9DEEFF"
Last-Modified: Thu, 05 Nov 2020 13:49:03 GMT
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
X-Ms-Blob-Type: BlockBlob
X-Ms-Creation-Time: Tue, 20 Oct 2020 11:48:51 GMT
X-Ms-Lease-Duration: infinite
X-Ms-Lease-State: leased
X-Ms-Lease-Status: locked
X-Ms-Meta-Terraformlockid: eyJJRCI6Ijk5YTk5Mzk2LTNhtMjE3NS02OTNkLTAxZTgyM2U3ZjA3ZiIsIk9wZXJhdGlvbiI6Ik9wZXJhdGlvblR5cGVQbGFuIiwiSW5mbyI6IiIsIldobyI6InJ1bm5lckBmdi1hejE3NC0yMTciLCJWZXJzaW9uIjoiMC4xMy41IiwiQ3JlYXRlZCI6IjIwMjAtMTEtMDVM6NDk6MDIuNzgzNDQwNjI5WiIsIlBhdGgiOiJ0c3RhdGUvdGVycmFmb3JtLnRmc3RhdGUi***
X-Ms-Request-Id: b9a109dd-d01e-002f-577a-b3a524000000
X-Ms-Server-Encrypted: true
X-Ms-Version: 2018-11-09

***
  "version": 4,
  "terraform_version": "0.13.5",
  "serial": 12,
  "lineage": "7f667e4-4407-c040-32ba-dce44bfda167",
  "outputs": ***,
  "resources": [
    ***
      "mode": "managed",
      "type": "azurerm_resource_group",
      "name": "aks",
      "provider": "provider[\"registry.terraform.io/hashicorp/azurerm\"]",
      "instances": [
        ***
          "schema_version": 0,
          "attributes": ***
            "id": "/subscriptions/***/resourceGroups/sociallme-k8s-rg",
            "location": "westeurope",
            "name": "project-k8s-rg",
            "tags": ***
              "env": "Dev project rg",
              "source": "project"
            ***,
            "timeouts": null
          ***,
          "private": "...AwfX0="
        ***
      ]
    ***
  ]
***

2020/11/05 13:49:03 [TRACE] backend/local: retrieving local state snapshot for workspace "default"
2020/11/05 13:49:03 [TRACE] backend/local: building context for current working directory
2020/11/05 13:49:03 [DEBUG] backend/local: will prompt for input of unset required variables [subscription_id client_id client_secret tenant_id]
2020/11/05 13:49:03 [DEBUG] command: asking for input: "var.client_id"
var.client_id
  The Azure Service Principal app ID.

Error: The operation was canceled.

What am I missing? Thanks!

Upvotes: 2

Views: 7033

Answers (1)

Christian Pearce
Christian Pearce

Reputation: 1026

The run is canceled because it is expecting input for var.client_id.

2020/11/05 13:49:03 [DEBUG] backend/local: will prompt for input of unset required variables [subscription_id client_id client_secret tenant_id]
2020/11/05 13:49:03 [DEBUG] command: asking for input: "var.client_id"
var.client_id
  The Azure Service Principal app ID.

Error: The operation was canceled.

You suggest it is included with terraform.tfvars, but there is not indication it is read in. A lot of default .gitignore files for terraform ignore terraform.tfvars.

Further you are mixing your authentication method with both variable input and environment variables. The practice I follow is to store the secrets in github and use the environment. It is a security risk and considered bad practice to commit your credentials.

To fix your issues you can probably delete these [subscription_id client_id client_secret tenant_id] variables.

Upvotes: 1

Related Questions