Kubernetes: Service routing to pods with multiple containers

Question

I am currently facing an issue in my Kubernetes cluster. While debugging that I had a question in mind which I didn't know the answer to.

I am using AWS EKS, version 1.15 but I don't think my question is related to any specific cloud or kubernetes version

I have a deployment. It has multiple containers. There is a service that exposes this deployment.

Suppose the deployment has 2 containers, C1 and C2. C1 takes 1 second to start but C2 takes 30 seconds to start(crazy!). So, when I start the pod at time t1, what happens is that once C1 starts immediately and the pod goes into running status but only 1/2 containers are ready. The pod C2 finally starts at time t2(t1+30seconds). At time t2, 2/2 containers are ready.

Also assume that C1 takes the incoming request from the service, it does something and then forwards request to C2, C2 does something and then returns it to C1. C1 finally returns to service and the response is served to client.

So, my question is, during the time between t2 and t1, when pod is in running state but only 1/2 container is ready, would the service forward requests to the pods?

Put another way, when does the service forward request to pods? If they are in running state and not matter how many containers are ready? OR if they are in running state and all containers are ready?

My thinking is that service won't forward as it won't make any sense if all the pods are not ready but I don't have any proof/document to justify it.

Answer 1

...when pod is in running state but only 1/2 container is ready, would the service forward requests to the pods?

No.

when does the service forward request to pods? If they are in running state and not matter how many containers are ready? OR if they are in running state and all containers are ready?

My thinking is that service won't forward as it won't make any sense if all the pods are not ready but I don't have any proof/document to justify it.

Here it is :)

Official documentation says that "...the kubelet uses readiness probes to know when a container is ready to start accepting traffic. A Pod is considered ready when all of its containers are ready. One use of this signal is to control which Pods are used as backends for Services. When a Pod is not ready, it is removed from Service load balancers..."

Additionally it says:

"...applications are temporarily unable to serve traffic... an application might depend on external services ... In such cases, you don't want to kill the application, but you don’t want to send it requests either. Kubernetes provides readiness probes to detect and mitigate these situations. A pod with containers reporting that they are not ready does not receive traffic through Kubernetes Services..."

Readiness probe is used to detect the situation when traffic shall not be sent to App.

My thinking is that service won't forward as it won't make any sense if all the pods are not ready

You are absolutely right here.

I hope that helps.

Answer 2

If you pay a glimpse to below mentioned snippet, from a deployment.yaml file -

spec:
  replicas: 4
    strategy:
        type: RollingUpdate
        rollingUpdate:
          maxUnavailable: 25%

It shows that for a canary deployment, the 25% criteria shows that, if you have set 4 dedicated replicas in deployment.yaml then whenever 75% of those have been successfully rolled out then allow traffic to be served by that service.

So basically you are having 3/4 replicas alive and you can serve traffic. This is purely configurable.

Answer 3

In order to make your scenario more understandable, lets call them web and api. those are the components of our service and while web will be ready in seconds, the api component will need more time.

First things first, we need to decide our deployment strategy. If we put web and api in the same deployment, then the service object on top of this deployment will enforce the definitions on both of them. So if you want to expose your web service on port 443, then the api will also be exposed on port 443. Yes, you can label those and set different definitions, but this is far from ideal.

We can say that the service objects in Kubernetes world acts like a load-balancer. So if you put two different components on the same deployment, and define a service object on top of them, then when you call your service from an outside network, you will end up reaching web or api endpoints, randomly.

You can check this image out for visualization: Kubernetes Service Example

In an ideal world, you need to deploy this application in two different deployments, because they can be de-coupled and serves for different purposes. After deploying those, all you need to do is to deploy two different services to expose your deployments. From my understanding, the api is only operating in the internal network, so it can be headless-service.

First things first, let's create a namespace (or project) for the application.

kubectl create ns myapp

And define our deployment So for our web component, let's define the deployment file;

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-deployment
  labels:
    app: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 9376

and the service which exposes our web deployment to the outside network

apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  selector:
    app: web
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9376

You can see that the web-deployment deployment object has three replicas and the web-service service definition will load-balance the incoming requests accordingly.

Now, let's deploy the api

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-deployment
  labels:
    app: api
spec:
  replicas: 5
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
      - name: api
        image: apirepo/api
        ports:
        - containerPort: 3000

and the headless service for the api-deployment

apiVersion: v1
kind: Service
metadata:
  name: api-headless-service
spec:
  clusterIP: None 
  selector:
    app: api
  ports:
    - protocol: TCP
      port: 80
      targetPort: 3000 

and that's all. Now you can scale up or down your web and api deployments based on requests and the service definitions will load-balance those automatically and handle the service discovery.

Answer 4

From the docs here

Ready: the Pod is able to serve requests and should be added to the load balancing pools of all matching Services

So if one pod is ready then that pods IP will be added to endpoints object and service will start sending traffic to that pod. Later if more pods becomes ready then those pods IP also get added to endpoints object and service will start load balancing traffic between all the pods.

To check pod IPs added to a service you can run kubectl describe service servicename and inspect the Endpoints section.

To avoid the scenario of traffic being sent to containers in a pod but containers are not yet ready to accept traffic you can use container probe

When all the containers inside a pod is ready then only service's Endpoints is populated with Pod IP and traffic starts flowing.

Answer 5

If the port inside container is not up, then probably no traffic will be forwarded. However, you can take a tcpdump inside the pod and follow the syn and reset flag to provide that