Docker ContainerID Unavailable

Question

Its being several times that my containers restart automatically with following error.The container is running along with chef on an ec2 instance. Here is the logs says

time="2016-10-19T23:23:36.382529790Z" level=info msg="GET /v1.16/containers/container1/json"
time="2016-10-19T23:23:36.382715521Z" level=error msg="Handler for GET /v1.16/containers/container1/json returned error: no such id: container1"
time="2016-10-19T23:23:36.382746955Z" level=error msg="HTTP Error" err="no such id: container1" statusCode=404
time="2016-10-19T23:23:36.383675061Z" level=info msg="GET /v1.16/containers/container1/json"
time="2016-10-19T23:23:36.383781507Z" level=error msg="Handler for GET /v1.16/containers/container1/json returned error: no such id: container1"
time="2016-10-19T23:23:36.383807920Z" level=error msg="HTTP Error" err="no such id: container1" statusCode=404
time="2016-10-19T23:23:36.384686602Z" level=info msg="GET /v1.16/containers/container1/json"

> Docker Version

docker version
Client:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   a34a1d5/1.9.1
 Built:
 OS/Arch:      linux/amd64

Server:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   a34a1d5/1.9.1
 Built:
 OS/Arch:      linux/amd64

> Docker Info

docker info
Containers: 1
Images: 50
Server Version: 1.9.1
Storage Driver: devicemapper
 Pool Name: docker-202:1-263798-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 107.4 GB
 Backing Filesystem:
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 5.981 GB
 Data Space Total: 107.4 GB
 Data Space Available: 40.37 GB
 Metadata Space Used: 4.989 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.142 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.93-RHEL7 (2015-01-28)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.1.13-19.30.amzn1.x86_64
Operating System: Amazon Linux AMI 2015.09
CPUs: 1
Total Memory: 3.679 GiB
Name: container1
ID: ARKR:MFOG:UO6J:LRZS:336R:TNTE:QTIA:64GD:B6HD:CRU6:G2YE:AWLZ
Username: test
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

Here is the logs from the /var/log/messages file

Oct 19 10:15:04 ip-10-2-130-44 kernel: [14529077.356962] veth98ee000: renamed from eth0
Oct 19 10:15:04 ip-10-2-130-44 kernel: [14529077.368372] docker0: port 1(veth85fc3ec) entered disabled state
Oct 19 10:15:04 ip-10-2-130-44 kernel: [14529077.400306] docker0: port 1(veth85fc3ec) entered disabled state
Oct 19 10:15:04 ip-10-2-130-44 kernel: [14529077.411398] device veth85fc3ec left promiscuous mode
Oct 19 10:15:04 ip-10-2-130-44 kernel: [14529077.413684] docker0: port 1(veth85fc3ec) entered disabled state
Oct 19 10:15:04 ip-10-2-130-44 kernel: [14529077.491597] XFS (dm-1): Unmounting Filesystem
Oct 19 10:15:07 ip-10-2-130-44 ntpd[3332]: Deleting interface #972 veth85fc3ec, fe80::1cd7:99ff:fe84:1495#123, interface stats: received=0, sent=0, droppedd
=0, active_time=29844 secs
Oct 19 10:15:16 ip-10-2-130-44 kernel: [14529088.927071] XFS (dm-1): Mounting V4 Filesystem
Oct 19 10:15:16 ip-10-2-130-44 kernel: [14529088.935124] XFS (dm-1): Ending clean mount

Have anybody encountered this error before, please let me know if any additional information required.

Answer 1

I could able to reproduce this error. If you query a container that doesn't exist on the docker host then it will give you statusCode=404 error.

# curl <dockerMachineIP>:2376/v1.21/containers/<validContainerName>/json will give you all the details about the container.

The same will through you error statusCode=404 if you curl with an invalid container id.

Answer 2

I have seen this exact error and it occurs whenever this an issue with JSON configuration for container. For my case, there was a typo in add-host entries where I added an extra colon, example 'a::b' instead of 'a:b'. It took me an hour to figure it out.