Error: timed out waiting for the condition

Question

On Ubuntu 18.04 with Docker 18 and OpenShift OKD oc v3.11.0 a local cluster will not start successfully and produce a time out error message.

Is it possible to start a local cluster on Ubuntu 18.04 using oc cluster up? Is it supported? How should a cluster be started on Ubuntu 18.04.

myuser:~] $ oc cluster up --public-hostname='ocp.127.0.0.1.nip.io' --routing-suffix='apps.ocp.127.0.0.1.nip.io'
Getting a Docker client ...
Checking if image openshift/origin-control-plane:v3.11 is available ...
Pulling image openshift/origin-control-plane:v3.11
Pulled 1/5 layers, 23% complete
Pulled 2/5 layers, 43% complete
Pulled 3/5 layers, 80% complete
Pulled 4/5 layers, 96% complete
Pulled 5/5 layers, 100% complete
Extracting
Image pull complete
Pulling image openshift/origin-cli:v3.11
Image pull complete
Pulling image openshift/origin-node:v3.11
Pulled 5/6 layers, 88% complete
Pulled 6/6 layers, 100% complete
Extracting
Image pull complete
Creating shared mount directory on the remote host ...
Determining server IP ...
Checking if OpenShift is already running ...
Checking for supported Docker version (=>1.22) ...
Checking if insecured registry is configured properly in Docker ...
Checking if required ports are available ...
Checking if OpenShift client is configured properly ...
Checking if image openshift/origin-control-plane:v3.11 is available ...
Starting OpenShift using openshift/origin-control-plane:v3.11 ...
I0416 08:32:35.747717   22853 config.go:40] Running "create-master-config"
I0416 08:32:37.456151   22853 config.go:46] Running "create-node-config"
I0416 08:32:38.721454   22853 flags.go:30] Running "create-kubelet-flags"
I0416 08:32:39.763094   22853 run_kubelet.go:49] Running "start-kubelet"
I0416 08:32:39.972403   22853 run_self_hosted.go:181] Waiting for the kube-apiserver to be ready ...
I0416 08:33:13.978672   22853 interface.go:26] Installing "kube-proxy" ...
I0416 08:33:13.978684   22853 interface.go:26] Installing "kube-dns" ...
I0416 08:33:13.978689   22853 interface.go:26] Installing "openshift-service-cert-signer-operator" ...
I0416 08:33:13.978694   22853 interface.go:26] Installing "openshift-apiserver" ...
I0416 08:33:13.978704   22853 apply_template.go:81] Installing "openshift-apiserver"
I0416 08:33:13.978752   22853 apply_template.go:81] Installing "kube-dns"
I0416 08:33:13.978758   22853 apply_template.go:81] Installing "kube-proxy"
I0416 08:33:13.978788   22853 apply_template.go:81] Installing "openshift-service-cert-signer-operator"
I0416 08:33:15.418545   22853 interface.go:41] Finished installing "kube-proxy" "kube-dns" "openshift-service-cert-signer-operator" "openshift-apiserver"
Error: timed out waiting for the condition



[myuser:~] 1 $ docker version
Client:
 Version:           18.09.1
 API version:       1.39
 Go version:        go1.10.6
 Git commit:        4c52b90
 Built:             Wed Jan  9 19:35:31 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.1
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.6
  Git commit:       4c52b90
  Built:            Wed Jan  9 19:02:44 2019
  OS/Arch:          linux/amd64
  Experimental:     false
[myuser:~] $ oc version
oc v3.11.0+0cbc58b
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO
[myuser:~] $ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"
[myuser:~] $ 

I think this has to do with Ubuntu 18.04 using dnsmasq by default now. I had some success using script similar to below.

#!/bin/bash
sudo /bin/sh -c 'echo "nameserver 8.8.8.8" > /etc/resolv.conf'
oc cluster up --public-hostname='ocp.127.0.0.1.nip.io' --routing-suffix='apps.ocp.127.0.0.1.nip.io'
oc cluster down
sudo /bin/sh -c 'echo "nameserver 8.8.8.8" > /etc/resolv.conf'
sudo /bin/sh -c 'echo "nameserver 8.8.8.8" > ~/openshift.local.clusterup/kubedns/resolv.conf'
oc cluster up --public-hostname='ocp.127.0.0.1.nip.io' --routing-suffix='apps.ocp.127.0.0.1.nip.io'

So the problem seems to be that /etc/resolv.conf is different under Ubuntu 18.04 and not suitable for oc cluster up.

After you try above workaround you can test if DNS is working correctly using a script similar to below

#!/bin/bash
oc login -u system:admin -n default
podname=$(oc get pods | grep registry | awk '{print $1;}')
oc exec $podname host github.com

Answer 1

Delete the build on openshift and re-build it. It worked for me

Answer 2

After trying to fix all possible issues like timeouts, EOF, errors, panic and other random problems (I had all of them around 300 fails during oc up) I revert my vm to state before installing of below things and I've done it in correct way I guess because now it works like it should (+ I can do oc cluster up/down without stress)

PS. It can be also limit 100 pulls of Docker image from Openshift (try docker run hello-world).

`sudo apt update && sudo apt upgrade`

`sudo apt install curl`
`curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -`


`sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"`

`sudo apt update && sudo apt -y install docker-ce`

`sudo usermod -aG docker XXXX`

`groups XXXX`


`wget https://github.com/openshift/origin/releases/download/v3.11.0/openshift-origin-client-tools-v3.11.0-0cbc58b-linux-64bit.tar.gz`


`cat << EOF | sudo tee /etc/docker/daemon.json
 {
    "insecure-registries" : [ "172.30.0.0/16" ]
 }
 EOF`

`sudo systemctl daemon-reload`
`sudo systemctl restart docker`

`sudo systemctl is-enabled docker`
`sudo systemctl is-active docker`

Regards, Artur

Answer 3

You can solve this issue by stopping all running containers on your machine or stop containers run by openshift if you have other important running containers on your machine and after run again the oc cluster up command:

docker container stop $(docker ps -q)

oc cluster up --skip-registry-check=true