istio: ingress with grpc and http

Question

I have a service listening on two ports; one is http, the other is grpc. I would like to set up an ingress that can route to both these port, with the same host.

The load balancer would redirect to the http port if http/1.1 is used, and to the grpc port if h2 is used.

Is there a way to do that with istio ?

I made a hello world demonstrating what I am trying to achieve :

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: hello-world
  namespace: dev
spec:
  replicas: 1
  template:
    metadata:
      annotations:
        alpha.istio.io/sidecar: injected
        pod.beta.kubernetes.io/init-containers: '[{"args":["-p","15001","-u","1337","-i","172.20.0.0/16"],"image":"docker.io/istio/init:0.1","imagePullPolicy":"Always","name":"init","securityContext":{"capabilities":{"add":["NET_ADMIN"]}}}]'
      labels:
        app: hello-world
    spec:
      containers:
      - name: grpc-server
        image: aguilbau/hello-world-grpc:latest
        ports:
        - name: grpc
          containerPort: 50051
      - name: http-server
        image: nginx:1.7.9
        ports:
        - name: http
          containerPort: 80
      - name: istio-proxy
        args:
        - proxy
        - sidecar
        - -v
        - "2"
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        image: docker.io/istio/proxy:0.1
        imagePullPolicy: Always
        resources: {}
        securityContext:
          runAsUser: 1337
---
apiVersion: v1
kind: Service
metadata:
  name: hello-world
  namespace: dev
spec:
  ports:
  - name: grpc
    port: 50051
  - name: http
    port: 80
  selector:
    app: hello-world
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: hello-world-http
  namespace: dev
  annotations:
    kubernetes.io/ingress.class: "istio"
spec:
  rules:
  - host: hello-world
    http:
      paths:
      - backend:
          serviceName: hello-world
          servicePort: 80
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: hello-world-grpc
  namespace: dev
  annotations:
    kubernetes.io/ingress.class: "istio"
spec:
  rules:
  - host: hello-world
    http:
      paths:
      - backend:
          serviceName: hello-world
          servicePort: 50051
---

Answer 1

I'm a bit late to the party, but for those of you stumbling on this post, I think you can do this with very little difficulty. I'm going to assume you have istio installed on a kubernetes cluster and are happy using the default istio-ingressgateway:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: hello-world
  namespace: dev
spec:
  replicas: 1
  template:
    metadata:
      annotations:
        alpha.istio.io/sidecar: injected
        pod.beta.kubernetes.io/init-containers: '[{"args":["-p","15001","-u","1337","-i","172.20.0.0/16"],"image":"docker.io/istio/init:0.1","imagePullPolicy":"Always","name":"init","securityContext":{"capabilities":{"add":["NET_ADMIN"]}}}]'
      labels:
        app: hello-world
    spec:
      containers:
      - name: grpc-server
        image: aguilbau/hello-world-grpc:latest
        ports:
        - name: grpc
          containerPort: 50051
      - name: http-server
        image: nginx:1.7.9
        ports:
        - name: http
          containerPort: 80
      - name: istio-proxy
        args:
        - proxy
        - sidecar
        - -v
        - "2"
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        image: docker.io/istio/proxy:0.1
        imagePullPolicy: Always
        resources: {}
        securityContext:
          runAsUser: 1337

---

apiVersion: v1
kind: Service
metadata:
  name: hello-world
  namespace: dev
spec:
  ports:
  - name: grpc
    port: 50051
  - name: http
    port: 80
  selector:
    app: hello-world

---

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: hello-world-istio-gate
  namespace: dev
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
  - port:
      number: 50051
      name: grpc
      protocol: GRPC
    hosts:
    - "*"

---

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: hello-world-istio-vsvc
  namespace: dev
spec:
  hosts:
  - "*"
  gateways:
  - hello-world-istio-gate
  http:
  - match:
    - port: 80
    route:
    - destination:
        host: hello-world
        port:
          number: 80
  tcp:
  - match:
    - port: 50051
    route:
    - destination:
        host: hello-world
        port:
          number: 50051

The above configuration omits your two Ingresses, and instead includes:

1. Your deployment
2. Your service
3. An istio gateway
4. An istio virtualservice

There is an important extra piece not shown, and I alluded to it earlier when talking about using the default ingressgateway. The following line found in "hello-world-istio-gateway" gives a clue:

istio: ingressgateway

This refers to a pod in the 'istio-system' namespace that is usually installed by default called 'istio-ingressgateway' - and this pod is exposed by a service also called 'istio-ingressgateway.' You will need to open up ports on the 'istio-ingressgateway' service.

As an example, I edited my (default) ingressgateway and added a port opening for HTTP and GRPC. The result is the following (edited for length) yaml code:

dampersand@kubetest1:~/k8s$ kubectl get service istio-ingressgateway -n istio-system -o yaml

apiVersion: v1
kind: Service
metadata:

<omitted for length>

  labels:
    app: istio-ingressgateway
    chart: gateways-1.0.3
    heritage: Tiller
    istio: ingressgateway
    release: istio
  name: istio-ingressgateway
  namespace: istio-system

  <omitted for length>

  ports:
  - name: http2
    nodePort: 31380
    port: 80
    protocol: TCP
    targetPort: 80

  <omitted for length>

  - name: grpc
    nodePort: 30000
    port: 50051
    protocol: TCP
    targetPort: 50051
  selector:
    app: istio-ingressgateway
    istio: ingressgateway
  type: NodePort

The easiest way to make the above change (for testing purposes) is to use:

kubectl edit svc -n istio-system istio-ingressgateway

For production purposes, it's probably better to edit your helm chart or your istio.yaml file or whatever you initially used to set up the ingressgateway.

As a quick aside, note that my test cluster has istio-ingressgateway set up as a NodePort, so what the above yaml file says is that my cluster is port forwarding 31380 -> 80 and 30000 -> 50051. You may (probably) have istio-ingressgateway set up as a LoadBalancer, which will be different... so plan accordingly.

Finally, the following blog post is some REALLY excellent background reading for the tools I've outlined in this post! https://blog.jayway.com/2018/10/22/understanding-istio-ingress-gateway-in-kubernetes/

Answer 2

You may be able to do something like that if you move the grpc-server and http-server containers into different pods with unique labels (i.e., different versions of the service, so to speak) and then using Istio route rules, behind the Ingress, split the traffic. A rule with a match for header Upgrade: h2 could send traffic to the grpc version and a default rule would send the rest of the traffic to http one.