CODEWITHSUNDEEP
azurekubernetesazure-aks
Mehran Hafizi
Mehran Hafizi

Reputation: 2592

Azure Kubernetes Stuck on Failed (Running) After Attempted Node Pool Addition - Quota Exceeded Issue

I'm currently facing a perplexing issue with my Azure Kubernetes Service (AKS). I recently attempted to add a new node pool to my AKS cluster, and encountered a "Quota exceeds" error during the process. Since then, my AKS cluster has been stuck in a state where it shows "Failed (Running)".

The error message I received during the node pool addition was as follows:

(CreateVMSSAgentPoolFailed) Code="OperationNotAllowed" Message="Operation could not be completed as it results in exceeding approved Total Regional Cores quota. Additional details - Deployment Model: Resource Manager, Location: uksouth, Current Limit: 10, Current Usage: 10, Additional Required: 4, (Minimum) New Limit Required: 14. Submit a request for Quota increase at [Quota Approval Link] by specifying parameters listed in the ‘Details’ section for deployment to succeed. Please read more about quota limits at [Quota Documentation Link]"

In an attempt to resolve the issue, I executed the command az resource update --ids as suggested in the error message. However, I received the same error again.

The error message indicates that I need to request a quota increase due to exceeding the approved Total Regional Cores quota for the location "uksouth". The details specify that the current limit is 10, current usage is 10, and additional required cores are 4, with a minimum new limit of 14.

I followed the provided link to request a quota increase, but I'm still stuck with a failed AKS cluster. I'm quite new to AKS and Azure, so I'm unsure how to proceed from here. Has anyone encountered a similar issue, or could provide guidance on how to resolve this? Is there something specific I need to do after requesting the quota increase?

Any assistance or insight would be greatly appreciated. Thank you in advance!

Code: CreateVMSSAgentPoolFailed Message: Code="OperationNotAllowed" Message="Operation could not be completed as it results in exceeding approved Total Regional Cores quota. Additional details - Deployment Model: Resource Manager, Location: uksouth, Current Limit: 10, Current Usage: 10, Additional Required: 4, (Minimum) New Limit Required: 14. Submit a request for Quota increase at [Quota Approval Link] by specifying parameters listed in the ‘Details’ section for deployment to succeed. Please read more about quota limits at [Quota Documentation Link]"

Upvotes: 1

Views: 804

Answers (1)

Vlad Rudenko
Vlad Rudenko

Reputation: 2789

To fix the failed state of the AKS cluster (after the "Cores" quota increase in this case) you need to run az aks upgrade to the same current version:

AKS_STATE=$(az aks show -g $RG_NAME -n $AKS_NAME --query provisioningState -o tsv)

if [ $AKS_STATE == "Failed" ]; then
        
    AKS_CURRENT_VER=$(az aks show -g $RG_NAME -n $AKS_NAME --query kubernetesVersion -o tsv)

    az aks upgrade \
        --resource-group $RG_NAME \
        --name $AKS_NAME \
        --kubernetes-version $AKS_CURRENT_VER \
        --yes \
        --output none
fi

Example of such az aks upgrade output:

Cluster currently in failed state. Proceeding with upgrade to existing version 1.29.2 to attempt resolution of failed cluster state.

Upvotes: 0

Related Questions