Reputation: 231
I have followed the instructions here https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret
So, I created a service principal
az ad sp create-for-rbac -n TerraformSP --role="Contributor" --scopes="/subscriptions/SUBSCRIPTION_ID"
This is the (resumed) terraform script
resource "azurerm_resource_group" "default" {
name = "myaks-rg"
location = var.location
}
resource "azurerm_kubernetes_cluster" "default" {
name = "myaks"
location = azurerm_resource_group.default.location
resource_group_name = azurerm_resource_group.default.name
dns_prefix = var.dns_prefix
identity {
type = "SystemAssigned"
}
role_based_access_control {
enabled = true
}
}
data "azurerm_container_registry" "acr" {
name = "myregistry"
resource_group_name = "another-rg"
}
resource "azurerm_role_assignment" "aks_acr" {
scope = data.azurerm_container_registry.acr.id
role_definition_name = "AcrPull"
principal_id = azurerm_kubernetes_cluster.default.kubelet_identity[0].object_id
skip_service_principal_aad_check = true
}
Then I executed terraform (using the IDs/Secret generated by az ad sp create-for-rbac
command)
az logout
export ARM_CLIENT_ID="00000000-0000-0000-0000-000000000000"
export ARM_CLIENT_SECRET="00000000-0000-0000-0000-000000000000"
export ARM_SUBSCRIPTION_ID="00000000-0000-0000-0000-000000000000"
export ARM_TENANT_ID="00000000-0000-0000-0000-000000000000"
terraform apply
The AKS is created but the Pod cannot pull the images. This is error message in the pod.
Failed to pull image "myregistry.azurecr.io/myimage:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://myregistry.azurecr.io/v2/myimage/manifests/latest: unauthorized: authentication required...
However, It works when I run terraform apply
using my personal account (az login
) and I don't set the environment variables ARM_CLIENT_ID, ARM_CLIENT_SECRET, etc...
I understand that the service principal TerraformSP (role Contributor) can't assign the role AcrPull
to azurerm_kubernetes_cluster.default.kubelet_identity[0].object_id
.
What is the best practice in this case? Should I assign some admin role to TerraformSP? Or, am I missing something in the terraform script?
The goal is to put this terraform script in a pipeline in Azure DevOps.
Upvotes: 1
Views: 925
Reputation: 231
I managed to make Kubernetes to pull images from my registry by re-creating the Service Principal with the role Owner
.
az ad sp create-for-rbac -n TerraformSP --role="Owner" --scopes="/subscriptions/SUBSCRIPTION_ID"
I also noticed that terraform creates successfully the role assignment.
azurerm_role_assignment.aks_acr: Creation complete after 24s [id=/subscriptions/SUBSCRIPTION_ID/resourceGroups/another-rg/providers/Microsoft.ContainerRegistry/registries/myregistry/providers/Microsoft.Authorization/roleAssignments/SOME_ID]
Upvotes: 1