CODEWITHSUNDEEP
dockerdocker-swarm
zeisi
zeisi

Reputation: 5540

Orphaned Tasks in Docker Swarm after removal of failed node

Last week I had to remove a failed node from my Docker Swarm Cluster, leaving some tasks that ran on that node in desired state "Remove".

Even after deleting the stack and recreating it with the same name, docker stack ps stackname still shows them.

Interestingly enough, after recreating the stack, the tasks are still there, but with no node assigned. screenshot of docker stack ps

Here's what I tried so far to "cleanup" the stack:

Is there a way to remove a specific task?

Here's the output for docker inspect fkgz0oihexzs, the first task in the list:

[
  {
    "ID": "fkgz0oihexzsjqwv4ju0szorh",
    "Version": {
      "Index": 14422171
    },
    "CreatedAt": "2018-11-05T16:15:31.528933998Z",
    "UpdatedAt": "2018-11-05T16:27:07.422368364Z",
    "Labels": {},
    "Spec": {
      "ContainerSpec": {
        "Image": "redacted",
        "Labels": {
          "com.docker.stack.namespace": "redacted"
        },
        "Env": [
          "redacted"
        ],
        "Privileges": {
          "CredentialSpec": null,
          "SELinuxContext": null
        },
        "Isolation": "default"
      },
      "Resources": {},
      "Placement": {
        "Platforms": [
          {
            "Architecture": "amd64",
            "OS": "linux"
          }
        ]
      },
      "Networks": [
        {
          "Target": "3i998stqemnevzgiqw3ndik4f",
          "Aliases": [
            "redacted"
          ]
        }
      ],
      "ForceUpdate": 0
    },
    "ServiceID": "g3vk9tgfibmcigmf67ik7uhj6",
    "Slot": 1,
    "Status": {
      "Timestamp": "2018-11-05T16:15:31.528892467Z",
      "State": "new",
      "Message": "created",
      "PortStatus": {}
    },
    "DesiredState": "remove"
  }
]

Upvotes: 7

Views: 1996

Answers (1)

SmartTom
SmartTom

Reputation: 821

I had the same problem. I resolved it following this instructions :

docker run --rm -v /var/run/docker/swarm/control.sock:/var/run/swarmd.sock dperny/tasknuke <taskid>

Be sure to use the full long task id or it will not work (fkgz0oihexzsjqwv4ju0szorh in your case).

Upvotes: 6

Related Questions