Executing readiness.httpGet: Readiness probe failed: No such host

Question

The following pod definition successfully executes a readinessProbe, which makes a request to the service service-am-i-ready that connects to pods on the same cluster.

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: ready-if-service-ready
  name: ready-if-service-ready
spec:
  containers:
  - image: nginx:1.16.1-alpine
    name: ready-if-service-ready
    resources: {}
    livenessProbe:                                     
      exec:
        command:
        - 'true'
    readinessProbe:
      exec:
        command:
        - sh
        - -c
        - 'wget -T2 -O- http://service-am-i-ready:80'  
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

However, if I change the readinessProbe.exec.command to readinessProbe.httpGet it doesn't work anymore:

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: ready-if-service-ready
  name: ready-if-service-ready
spec:
  containers:
  - image: nginx:1.16.1-alpine
    name: ready-if-service-ready
    resources: {}
    livenessProbe:                                     
      exec:
        command:
        - 'true'
    readinessProbe:
      httpGet:                         # Only changed this method
        host: service-am-i-ready
        path: /
        port: 80
        scheme: HTTP 
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

This is the error message I get running kubectl po describe ready-if-service-ready:

Warning  Unhealty  3m10s  (x139 over 23m)  kubelet  Readiness probe  failed: Get "http://service-am-i-ready:80/": dial tcp: lookup service-am-i-ready: no such host

Running kubectl get po ready-if-service-ready gives:

NAME                    READY       STATUS      RESTARTS        AGE
ready-if-service-ready  0/1         Running     0               27m

Why is the first readinessProbe working, but not the second one? It looks like the second readinessProbe makes a request to the same endpoint as the wget -T2 -O- http://service-am-i-ready:80 command.

Answer 1

@glv's answer is also correct, but let me explain you why it is not working and what is the otherway

The reason is wget uses the DNS resolver configured in the pod, which is set to the Kubernetes DNS service by default.

httpGet probe, which is executed by the kubelet on the node hosting the pod. The kubelet does not use the DNS resolver configured in the pod, but rather the host's DNS resolver configuration

You can modify your readiness probe to use the IP address of the pod providing the service instead of the hostname.

readinessProbe:
  httpGet:
    host: (POD_IP)
    path: /
    port: 80
    scheme: HTTP 

You can dyamically assign IP by this way as well

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: ready-if-service-ready
  name: ready-if-service-ready
spec:
  containers:
  - image: nginx:1.16.1-alpine
    name: ready-if-service-ready
    resources: {}
    env:
    - name: MY_POD_IP
      valueFrom:
        fieldRef:
          fieldPath: status.podIP
    livenessProbe:                                     
      exec:
        command:
        - 'true'
    readinessProbe:
      httpGet:
        host: $(MY_POD_IP)
        path: /
        port: 80
        scheme: HTTP 
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

Answer 2

Try following this example:

    httpGet:
      host:
      scheme: HTTP
      path: /
      httpHeaders:
      - name: Host
        value: service-am-i-ready
      port: 80

Tip: If you need to set the Host header of the HTTP, please do so on httpHeaders, instead of the host parameter.

https://loft.sh/blog/kubernetes-readiness-probes-examples-and-common-pitfalls/#http-probe