CODEWITHSUNDEEP
docker-swarm
herm
herm

Reputation: 16315

docker swarm how to find out why service can't start

I often have problems because a service that I deploy on docker swarm with multiple nodes won't start and there are not logs generated that I can look at with docker service logs {serviceName}

There are many possible reasons for a service not to start such as

I have trouble finding out why a container won't start. I found the command docker service ps {serviceName} which List the tasks of one or more services and a short error message (if there was an error). However when I try to inspect the task with docker service logs {taskId} (which should show logs of a task) I get Error response from daemon: task 3lkgo8t2sn7k not found.

Can anyone help me get a full error message why a service won't start?

Upvotes: 97

Views: 61763

Answers (2)

nicorellius
nicorellius

Reputation: 4043

Another trick I've been using is from this article. I was quite stuck and this trick got me a bit further down the road:

docker run --network my-network -it --rm --entrypoint bash my-service

The problem is that when I ran it this way, I could start the service and interact with the other services in the swarm (via the external overlay network). But when I tried to start all three services with Docker Compose, the above service failed to start. So I'm still stuck, but slightly closer than before.

I often grep the syslog too, which sometimes reveals some nuggets:

less /var/log/syslog | grep docker
less /var/log/syslog | grep error

And, finally, here is a trick I found that sometimes works:

https://faun.pub/debug-docker-swarm-services-eec20fe3d13e

Upvotes: 0

herm
herm

Reputation: 16315

I found one handy solution to the problem.

docker service ps --no-trunc {serviceName}

which will show errors with downloading images, mounting nfs volumes amongst others.

---------------------- UPDATE

Not all errors can be found in the way described above. Another usefull tool is looking at the docker deamon logs which can be done the follwing way as explained on stackoverflow: 

journalctl -u docker.service | tail -n 50

It depends on your OS. Here are the few locations, with commands for few Operating Systems:

  • Ubuntu (old using upstart ) - /var/log/upstart/docker.log
  • Ubuntu (new using systemd ) - journalctl -u docker.service
  • Boot2Docker - /var/log/docker.log
  • Debian GNU/Linux - /var/log/daemon.log
  • CentOS - /var/log/daemon.log | grep docker
  • CoreOS - journalctl -u docker.service
  • Fedora - journalctl -u docker.service
  • Red Hat Enterprise Linux Server - /var/log/messages | grep docker
  • OpenSuSE - journalctl -u docker.service
  • OSX - ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/log/d‌​ocker.log
  • Windows - Get-EventLog -LogName Application -Source Docker -After (Get-Date).AddMinutes(-5) | Sort-Object Time, as mentioned here.

Upvotes: 224

Related Questions