CODEWITHSUNDEEP
dockerkubernetesdockerfileminikubeskaffold
cheslijones
cheslijones

Reputation: 9194

Can't get response from Express API in k8s-Skaffold from Postman

Trying to do something that should be pretty simple: starting up an Express pod and fetch the localhost:5000/ which should respond with Hello World!.

Not quite sure what I am doing wrong here. Code and configs are below. Suggestions?

index.js

// Import dependencies
const express = require('express');

// Set the ExpressJS application
const app = express();

// Set the listening port
// Web front-end is running on port 3000
const port = 5000;

// Set root route
app.get('/', (req, res) => res.send('Hello World!'));

// Listen on the port
app.listen(port, () => console.log(`Example app listening on port ${port}`));

skaffold.yaml

apiVersion: skaffold/v1beta15
kind: Config
build:
  local:
    push: false
  artifacts:
    - image: sockpuppet/server
      context: server
      docker:
        dockerfile: Dockerfile.dev
      sync:
        manual:
        - src: '**/*.js'
          dest: .
deploy:
  kubectl:
    manifests:
      - k8s/ingress-service.yaml
      - k8s/server-deployment.yaml
      - k8s/server-cluster-ip-service.yaml

ingress-service.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-service
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
    - http:
        paths:
          - path: /?(.*)
            backend:
              serviceName: server-cluster-ip-service
              servicePort: 5000

server-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: server-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      component: server
  template:
    metadata:
      labels:
        component: server
    spec:
      containers:
        - name: server
          image: sockpuppet/server
          ports:
            - containerPort: 5000

server-cluster-ip-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: server-cluster-ip-service
spec:
  type: ClusterIP
  selector:
    component: server
  ports:
    - port: 5000
      targetPort: 5000

Dockerfile.dev

FROM node:12.10-alpine
EXPOSE 5000

WORKDIR "/app"
COPY ./package.json ./
RUN npm install
COPY . .

CMD ["npm", "run", "dev"]

Output from describe

$ kubectl describe ingress ingress-service     
Name:             ingress-service
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host       Path  Backends
  ----       ----  --------
  localhost  
             /   server-cluster-ip-service:5000 (172.17.0.7:5000,172.17.0.8:5000,172.17.0.9:5000)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-service","namespace":"default"},"spec":{"rules":[{"host":"localhost","http":{"paths":[{"backend":{"serviceName":"server-cluster-ip-service","servicePort":5000},"path":"/"}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:
  Type    Reason  Age   From                      Message
  ----    ------  ----  ----                      -------
  Normal  CREATE  16h   nginx-ingress-controller  Ingress default/ingress-service
  Normal  CREATE  21s   nginx-ingress-controller  Ingress default/ingress-service

Output from kubectl get po -l component=server

$ kubectl get po -l component=server
NAME                                READY   STATUS    RESTARTS   AGE
server-deployment-cf6dd5744-2rnh9   1/1     Running   0          11s
server-deployment-cf6dd5744-j9qvn   1/1     Running   0          11s
server-deployment-cf6dd5744-nz4nj   1/1     Running   0          11s

Output from kubectl describe pods server-deployment: Noticed that the Host Port: 0/TCP. Possibly the issue? 

Name:               server-deployment-6b78885779-zttns
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               minikube/10.0.2.15
Start Time:         Tue, 08 Oct 2019 19:54:03 -0700
Labels:             app.kubernetes.io/managed-by=skaffold-v0.39.0
                    component=server
                    pod-template-hash=6b78885779
                    skaffold.dev/builder=local
                    skaffold.dev/cleanup=true
                    skaffold.dev/deployer=kubectl
                    skaffold.dev/docker-api-version=1.39
                    skaffold.dev/run-id=c545df44-a37d-4746-822d-392f42817108
                    skaffold.dev/tag-policy=git-commit
                    skaffold.dev/tail=true
Annotations:        <none>
Status:             Running
IP:                 172.17.0.5
Controlled By:      ReplicaSet/server-deployment-6b78885779
Containers:
  server:
    Container ID:   docker://2d0aba8f5f9c51a81f01acc767e863b7321658f0a3d0839745adb99eb0e3907a
    Image:          sockpuppet/server:668dfe550d93a0ae76eb07e0bab900f3968a7776f4f177c97f61b18a8b1677a7
    Image ID:       docker://sha256:668dfe550d93a0ae76eb07e0bab900f3968a7776f4f177c97f61b18a8b1677a7
    Port:           5000/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 08 Oct 2019 19:54:05 -0700
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-qz5kr (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  default-token-qz5kr:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-qz5kr
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age        From               Message
  ----    ------     ----       ----               -------
  Normal  Scheduled  <unknown>  default-scheduler  Successfully assigned default/server-deployment-6b78885779-zttns to minikube
  Normal  Pulled     7s         kubelet, minikube  Container image "sockpuppet/server:668dfe550d93a0ae76eb07e0bab900f3968a7776f4f177c97f61b18a8b1677a7" already present on machine
  Normal  Created    7s         kubelet, minikube  Created container server
  Normal  Started    6s         kubelet, minikube  Started container server

Upvotes: 2

Views: 1360

Answers (1)

cheslijones
cheslijones

Reputation: 9194

OK, got this sorted out now.

It boils down to the kind of Service being used: ClusterIP.

ClusterIP: Exposes the service on a cluster-internal IP. Choosing this value makes the service only reachable from within the cluster. This is the default ServiceType.

If I am wanting to connect to a Pod or Deployment directly from outside of the cluster (something like Postman, pgAdmin, etc.) and I want to do it using a Service, I should be using NodePort:

NodePort: Exposes the service on each Node’s IP at a static port (the NodePort). A ClusterIP service, to which the NodePort service will route, is automatically created. You’ll be able to contact the NodePort service, from outside the cluster, by requesting <NodeIP>:<NodePort>.

So in my case, if I want to continue using a Service, I'd change my Service manifest to:

apiVersion: v1
kind: Service
metadata:
  name: server-cluster-ip-service
spec:
  type: NodePort
  selector:
    component: server
  ports:
    - port: 5000
      targetPort: 5000
      nodePort: 31515

Making sure to manually set nodePort: <port> otherwise it is kind of random and a pain to use.

Then I'd get the minikube IP with minikube ip and connect to the Pod with 192.168.99.100:31515.

At that point, everything worked as expected.

But that means having separate sets of development (NodePort) and production (ClusterIP) manifests, which is probably totally fine. But I want my manifests to stay as close to the production version (i.e. ClusterIP).

There are a couple ways to get around this:

  1. Using something like Kustomize where you can set a base.yaml and then have overlays for each environment where it just changes the relevant info avoiding manifests that are mostly duplicative.

  2. Using kubectl port-forward. I think this is the route I am going to go. That way I can keep my one set of production manifests, but when I want to QA Postgres with pgAdmin I can do:

    kubectl port-forward services/postgres-cluster-ip-service 5432:5432

    Or for the back-end and Postman:

    kubectl port-forward services/server-cluster-ip-service 5000:5000

I'm playing with doing this through the ingress-service.yaml using nginx-ingress, but don't have that working quite yet. Will update when I do. But for me, port-forward seems the way to go since I can just have one set of production manifests that I don't have to alter.

Skaffold Port-Forwarding

This is even better for my needs. Appending this to the bottom of the skaffold.yaml and is basically the same thing as kubectl port-forward without tying up a terminal or two:

portForward:
  - resourceType: service
    resourceName: server-cluster-ip-service
    port: 5000
    localPort: 5000
  - resourceType: service
    resourceName: postgres-cluster-ip-service
    port: 5432
    localPort: 5432

Then run skaffold dev --port-forward.

Upvotes: 4

Related Questions