CODEWITHSUNDEEP
ibm-cloud-private
kgibm
kgibm

Reputation: 1028

ICP 2.1 Beta 2 can't create Cloudant DB fast enough

When running the following command from the installation instructions:

sudo docker run --net=host -t -e LICENSE=accept -v $(pwd):/installer/cluster \
            ibmcom/icp-inception:2.1.0-beta-2-ee install

The task to create the Cloudant DB times out:

TASK [master : Ensuring that the Cloudant Database is ready] ***********************************************************************************************************************************************
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (20 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (19 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (18 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (17 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (16 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (15 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (14 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (13 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (12 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (11 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (10 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (9 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (8 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (7 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (6 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (5 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (4 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (3 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (2 retries left).
FAILED - RETRYING: TASK: master : Ensuring that the Cloudant Database is ready (1 retries left).
fatal: [10.20.30.29] => Status code was not [200]: Request failed: <urlopen error [Errno 111] Connection refused>

PLAY RECAP *************************************************************************************************************************************************************************************************
10.20.30.29                : ok=149  changed=57   unreachable=0    failed=1

The container is still using CPU in /usr/bin/python /usr/bin/ansible-playbook -e @cluster/config.yaml playbook/site.yaml so I think it's still installing. How do I increase the number of retries?

Upvotes: 0

Views: 381

Answers (3)

Michael Elder
Michael Elder

Reputation: 246

In some cases, we find that the size of the ICP datastore, which is based on Cloudant, takes awhile to download. Have you considered pre-pulling the image onto your host?

docker pull ibmcom/icp-datastore:{version}

You would need to do this on each master (but not on the worker nodes).

Upvotes: 0

Ali Kaba
Ali Kaba

Reputation: 119

Run the command

docker info | grep -i cgroup

and you should see

Cgroup Driver: systemd

...so matched that in ICP by adding the line

kubelet_extra_args: ["--cgroup-driver=systemd"]

to ICP's config.yaml

Upvotes: 1

SBERENS
SBERENS

Reputation: 68

This usually means cloudant db takes too long to be ready and by that time, 20 times retry limit has been reached. Uninstall ICP 2.1 beta2 and install it usually gets past this issue. In the ICP 2.1 beta3, it will have an increased timeout.

If this does not work, you can check the following:

docker exec -it <cloudant container> bash

now inside the container, to check the database status:

cast cluster status -p $ADMIN_PASSWORD

Also can check:

docker logs <long cloudant container name>

Before installing ICP beta2 again, check whether docker volume ls was cleaned up.

Upvotes: 0

Related Questions