Terraform: Error building ARM Config - Authenticating using the Azure CLI is only supported

Question

As the terraform azurerm provider misses support for azure webapp access restrictions (see github issue). We use a null_resource with local-exec to apply a access restriction:

  provisioner "local-exec" {
    command = <<COMMAND
      az webapp config access-restriction add --subscription ${self.triggers.subscription_id} --resource-group ${self.triggers.resource_group} \
        --name ${self.triggers.web_app_name} --rule-name 'allow application gateway' --action Allow --vnet-name ${self.triggers.vnet_name} \
        --subnet ${self.triggers.subnet_name} --priority 100
    COMMAND
  }

Our terraform code is then later run by an azure DevOps Pipeline, which uses a Service Connection (with Service Principal) to authenticate with Azure. The following task is trying to apply the terraform resources:

  - task: TerraformCLI@0
    displayName: "Terraform apply"
    inputs:
      command: 'apply'
      commandOptions: '--var-file="./environments/${{ parameters.environment }}.tfvars"'
      workingDirectory: '$(System.DefaultWorkingDirectory)/${{ parameters.projectFolder }}'
      environmentServiceName: 'shared-${{ parameters.environment }}-001'

which results in the following Error:

Error: Error running command '      az webapp config access-restriction remove --subscription shared-staging-001 --resource-group rg-hub-network-staging \
        --name landing-webapp-hub --rule-name 'allow application gateway'
': exit status 1. Output: Subscription 'shared-staging-001' not recognized.
Command group 'webapp config access-restriction' is in preview. It may be changed/removed in a future release.
Please run 'az login' to setup account.

No we tried to replace the TerraformCLI@0 Task with either a plain bash script or a AzureCLI@2 Task.

We could not get az login to work in a plain bash script due to the missing Infos. The approach described here does not work either.

Running the terraform commands inside a AzureCLI@2 Task looks promissing but causes some strange errors related to the service principal login:

  - task: AzureCLI@2
    displayName: "Terraform init"
    inputs:
      azureSubscription: shared-${{ parameters.environment }}-001
      scriptType: bash
      scriptLocation: inlineScript
      inlineScript: |
        terraform init --backend-config="./environments/${{ parameters.environment }}_backend.tfvars"

This causes the following error:

Initializing modules...
- app-gateway in modules/app-gateway
- dummy1 in modules/BRZ365-AppService
- dummy2 in modules/BRZ365-AppService
- hub-network in modules/hub-network
- landing_zone_app in modules/BRZ365-AppService
- squad-area in modules/squad-area

Initializing the backend...

Error: Error building ARM Config: Authenticating using the Azure CLI is only supported as a User (not a Service Principal).

To authenticate to Azure using a Service Principal, you can use the separate 'Authenticate using a Service Principal'
auth method - instructions for which can be found here: 

Alternatively you can authenticate using the Azure CLI by using a User Account.

Answer 1

The accepted answer by @quadroid is perfect, which is so helpful and save my time a lot. set addSpnToEnvironment and use export commands to set Azure Environment Variables are keys to resolve the issue.

But it doesn't support management group (if you know AWS, it is similar as AWS cross account assume role) , only works with current subscription.

Understand scopes for Azure RBAC

• management group
• subscription
• resource group
• resource

So the error I got is

Error: building account: unable to configure ResourceManagerAccount: subscription ID could not be determined and was not specified

I finally worked it out with management group and I can run terraform/terragrunt command to any subscriptions under this management group.

• use addSpnToEnvironment and set it to true (it adds the service provider credentials to the environment, as described in the documentation)
• set Azure enironment variables as described by terraform.

# these variables are automatically generated by `addSpnToEnvironment` set to `true`
export ARM_CLIENT_ID=$servicePrincipalId
export ARM_CLIENT_SECRET=$servicePrincipalKey
export ARM_TENANT_ID=$tenantId

# this variable need be input by pipeline's parameters or hardcode if you know its subscription id
export ARM_SUBSCRIPTION_ID="20000000-0000-0000-0000-000000000000"
or
export ARM_SUBSCRIPTION_ID="$(subscription_id)"

The final pipeline for your reference

parameters:
- name: subscription_id
  displayName: 'Azure Subscription ID'
  type: string

stages:
- stage: TerraformDeployment
  jobs:
  - job: TerraformDeployment
    displayName: TerraformDeployment
    steps:
      - task: AzureCLI@2
        displayName: "Terraform"
        inputs:
          azureSubscription:  <replace_with_service_account_with_management_group_permission>
          scriptType: bash
          addSpnToEnvironment: true
          scriptLocation: inlineScript
          inlineScript: |
            export ARM_CLIENT_ID=$servicePrincipalId
            export ARM_CLIENT_SECRET=$servicePrincipalKey
            export ARM_TENANT_ID=$tenantId
            # it need be set via Environment variable, or get from parameters
            export ARM_SUBSCRIPTION_ID="$(subscription_id)"

            terraform init .....

         

Answer 2

Update November 2023

I made it work by using the ARM prefix, and changing my azurerm provider to 3.0

Answer 3

I had a similar issue when working with Terraform to setup resources on Azure.

When I run terraform plan I get the error:

│ Error: building AzureRM Client: Authenticating using the Azure CLI is only supported as a User (not a Service Principal).
│ 
│ To authenticate to Azure using a Service Principal, you can use the separate 'Authenticate using a Service Principal'
│ auth method - instructions for which can be found here: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret
│ 
│ Alternatively you can authenticate using the Azure CLI by using a User Account.
│ 
│   with provider["registry.terraform.io/hashicorp/azurerm"],
│   on main.tf line 18, in provider "azurerm":
│   18: provider "azurerm" {

Here's how I solved it.

The problem was that I was authenticating Azure CLI with Terraform the wrong way. I was using the command below to authenticate it with a service principal:

az login --service-principal --username $SERVICE_PRINCIPAL_APP_ID --password $SERVICE_PRINCIPAL_PASSWORD --tenant $SERVICE_PRINCIPAL_TENANT_ID

The issue is that the command above does not authenticate you as a user, but as a service principal, which is not a supported way to authenticate using service principal by Terraform.

[
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "f8256ac9-e164-4ac6-a487-0f6e3dc17532",
    "id": "bec123cd-bead-43ba-90c6-5235cririe903",
    "isDefault": true,
    "managedByTenants": [],
    "name": "my-subscription-1",
    "state": "Enabled",
    "tenantId": "f8556ac9-e159-4ac6-a487-8r9erhwjw",
    "user": {
      "name": "r673reb62-bbf7-40e1-ab10-ry45640484",
      "type": "servicePrincipal"
    }
  }
]

The right way to do this was to simply run the command below:

az login

Then the Azure CLI will open up a browser window where I will log in to my Azure account on the portal and get authenticated as a user. If you have multiple subscriptions, it will select your currently logged-in subscription as your default subscription.

[
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "f8256ac9-e164-4ac6-a487-0f6e3dc17532",
    "id": "bec123cd-bead-43ba-90c6-5235cririe903",
    "isDefault": true,
    "managedByTenants": [],
    "name": "my-subscription-1",
    "state": "Enabled",
    "tenantId": "f8556ac9-e159-4ac6-a487-8r9erhwjw",
    "user": {
      "name": "john.doe@my-domain.com",
      "type": "user"
    }
  },
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "ghfjg6ac9-e164-7466-a487-0f6e3dc17hjr",
    "id": "rtkrt7f3d-51b2-7587-88b7-d29ftheruofw",
    "isDefault": false,
    "managedByTenants": [],
    "name": "my-subscription-2",
    "state": "Enabled",
    "tenantId": "f8556ac9-e159-4ac6-a487-8r9erhwjw",
    "user": {
      "name": "john.doe@my-domain.com",
      "type": "user"
    }
  }
]

OR if you desire to authenticate as service principal, then you will export the following variables using the provided variables names:

export ARM_CLIENT_ID="your-service-principal-appid"
export ARM_CLIENT_SECRET="your-service-principal-password"
export ARM_SUBSCRIPTION_ID="your-current-subscription-id"
export ARM_TENANT_ID="your-tenant-id"

Now, you can run your terraform plan and everything will work fine.

Resources: Configuring the Service Principal in Terraform

Answer 4

I finally got this to work with the AzureCLI approach I described in the first post. I use addSpnToEnvironment (it adds the service provider credentials to the environment, as described in the documentation) and set the required parameters as described by terraform.

      - task: AzureCLI@2
        displayName: "Terraform"
        inputs:
          azureSubscription:  shared-${{ parameters.environment }}-001
          scriptType: bash
          addSpnToEnvironment: true
          scriptLocation: inlineScript
          inlineScript: |
            export ARM_CLIENT_ID=$servicePrincipalId
            export ARM_CLIENT_SECRET=$servicePrincipalKey
            export ARM_TENANT_ID=$tenantId

            terraform init .....

         

Answer 5

I got through this with local-exec.

provisioner "local-exec" {
    command = <<COMMAND
      az login --service-principal --username #{APP_ID}# --password #{SP_PASSWORD}# --tenant #{TENANT_ID}#
      az webapp config access-restriction add --resource-group ${azurerm_resource_group.example.name} --name ${azurerm_app_service.example.name} --rule-name developers --action Allow --ip-address 130.220.0.0/27 --priority 200
    COMMAND

    interpreter = ["PowerShell", "-Command"]
  }

Unfortunately I had to create another service principal for this purpose as I didn't want to reset the one used by Azure DevOps (but you can give it a try and reuse this one).

I used these commands:

az ad sp create-for-rbac --name sp-for-cli

az role assignment create --assignee APP_ID --role Contributor

As next I declared variables APP_ID, SP_PASSWORD and TENANT_ID on my release pipeline with values given by command above.

As last step I added token replace step:

steps:
- task: qetza.replacetokens.replacetokens-task.replacetokens@3
  displayName: 'Replace tokens in main.tf'
  inputs:
    rootDirectory: '$(System.DefaultWorkingDirectory)/terraform/drop'
    targetFiles: main.tf

Now when I run az webapp config access-restriction show --resource-group example-resources --name example-app-service-for-cli I get:

"ipSecurityRestrictions": [
    {
      "action": "Allow",
      "additional_properties": {},
      "description": null,
      "ip_address": "130.220.0.0/27",
      "name": "developers",
      "priority": 200,
      "subnet_mask": null,
      "subnet_traffic_tag": null,
      "tag": "Default",
      "vnet_subnet_resource_id": null,
      "vnet_traffic_tag": null
    },

The whole code you can find here.