CODEWITHSUNDEEP
azure-container-serviceazure-container-instances
Alexander Trauzzi
Alexander Trauzzi

Reputation: 7405

Azure Container Instances stuck in "Creating" state

Whether I have the azure agent plugin for Jenkins make my container, or if I do it manually, it seems like either way it never enters a running state.

enter image description here

az container create \
    --os-type Windows \
    --location eastus \
    --registry-login-server SERVER.azurecr.io \
    --registry-password PASSWORD \
    --registry-username USERNAME \
    --image namespace/image \
    --name jenkins-permanent \
    --resource-group devops-aci \
    --cpu 2 \
    --memory 3.5 \
    --restart-policy Always \
    --command-line "-jnlpUrl http://host:8080/computer/NAME/slave-agent.jnlp -secret SECRET -workDir \"C:\\jenkins\""

I've gone through all the troubleshooting steps that apply, tried a different region, but to no avail.

Here's a current event that I got which seems to be the most progress I've had yet:

          {
            "count": 1,
            "firstTimestamp": "2017-12-07T03:02:56+00:00",
            "lastTimestamp": "2017-12-07T03:02:56+00:00",
            "message": "Failed to pull image \"MYREPO.azurecr.io/my-company/windows-agent:latest\": Error response from da
emon: {\"message\":\"Get https://MYREPO.azurecr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout
exceeded while awaiting headers)\"}",
            "name": "Failed",
            "type": "Warning"
          }

The funny thing is, this event happens before and after one case of the instance working (but unfortunately my entrypoint command was wrong, so it never started).

I really feel like Azure is punting on this and I just have no way to change the order I do anything. It's simply one command.

Upvotes: 2

Views: 5956

Answers (3)

Alexander Trauzzi
Alexander Trauzzi

Reputation: 7405

Figured it out, turns out the backslashes in the command in my executable path were not having their escapes honoured. Either because I was calling az from bash, or because something Azure side isn't handling the escaping correctly, or not escaping them itself.

My solution has been to just use forward slashes in the paths. Windows seems to be handling them correctly, and I prefer to not be bothered with its odd preference for backslashes.

Related to my issue is that the speed of the service makes troubleshooting very difficult. It takes a long time to go round trip with any fixes. So if you're using Azure Container Instances and want better performance, go upvote this feedback item that I've created.

Upvotes: 1

shou3301
shou3301

Reputation: 214

How big is your image? You can always debug with 2 steps.

Run az container show -g devops-aci -n jenkins-permanent. It should contain a list of container events in the container json object. The event message should give you hint what's going on.

Run az container logs -g devops-aci -n jenkins-permanent. It should give you the logs of your container. If it's a problem within your image, you should be able to see some error output.

Upvotes: 0

Adam Smith - Microsoft Azure
Adam Smith - Microsoft Azure

Reputation: 2513

Alexander, here's a lead to actually check what could be causing the delay, or if the deployment has failed in the background, this information would be critical to narrow down what the issue is: https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-troubleshoot-tips#determine-error-code

From the article above check on deployment logs: Enable debug logging: PowerShell In PowerShell, set the DeploymentDebugLogLevel parameter to All, ResponseContent, or RequestContent.

New-AzureRmResourceGroupDeployment -ResourceGroupName examplegroup -TemplateFile c:\Azure\Templates\storage.json -DeploymentDebugLogLevel All

or Azure CLI:

az group deployment operation list --resource-group ExampleGroup --name vmlinux

Check Also Check deployment sequence: Many deployment errors happen when resources are deployed in an unexpected sequence. These errors arise when dependencies are not correctly set. When you are missing a needed dependency, one resource attempts to use a value for another resource but the other does not yet exist.

The above link contains more details. Let me know if this helps.

Upvotes: 3

Related Questions