CODEWITHSUNDEEP
kubernetesgrpcload-balancingistioservicemesh
Ivan Voroshilin
Ivan Voroshilin

Reputation: 5416

Kubernetes and Service Mesh load-balancing misalignments

Kubernetes has a support of Pod load-balancing, session affinity through its kube-proxy. Kubernetes’ kube-proxy is essentially an L4 load balancer so we cannot rely on it to load balance L7-transport, e.g. muliple gRPC live connections or load-balancing based on http-headers, cookies, etc.

Service Mesh implementation like e.g. istio can handle these patterns on L7-level including gRPC. But I always thought that Service Mesh is just another layer on top of Kubernetes with additional capabilities(encrypted traffic, blue/green deployments/etc). E.g. My assumption always was that Kubernetes applications should be able to work on both vanilla Kubernetes without Mesh (e.g. for development/testing) or with a Mesh on. Adding this advanced traffic management on L7 breaks this assumption. I won't be able to work on a vanilla Kubernetes anymore, I will be tied to a specific implementation of Istio dataplane(Envoy).

Please let know if my assumption is correct or why not? There's not much information about this type of separation of concerns on this internet.

Upvotes: 2

Views: 425

Answers (1)

Nepomucen
Nepomucen

Reputation: 6687

Let me refer first to the following statement of yours:

My assumption always was that Kubernetes applications should be able to work on both vanilla Kubernetes without Mesh (e.g. for development/testing) or with a Mesh on. Adding this advanced traffic management on L7 breaks this assumption.

I have different view at that, Service Meshes are transparent to the application, so they don't break anything in them, but just add an extra (network, security, monitoring) functions at no cost (ok, the cost it quite complex configuration from Mesh operator perspective). The Service Mesh like Istio doesn't need to occupy all K8S namespaces, so you can still have mixed type of workloads in your cluster (with and w/o proxies). If we speak about Istio, to enable full interoperatbility between them (mixed workloads) you may combine two its features together:

  • Peer authentication set to PERMISSIVE, so that workloads without sidecars (proxy) can accept both mutual TLS and plain text traffic.
  • Manual protocol selection, e.g. if you prefer your app speak raw TCP instead of app protocol determined by Envoy itself (e.g. http) - to avoid extra decorations injected by proxy to intercepted requests.

Alternatively you can write your own custom tcp_proxy EnvoyFilter to use Envoy as a L4 network proxy.

References:

https://istio.io/latest/docs/reference/config/networking/envoy-filter/ https://istio.io/latest/docs/concepts/security/#peer-authentication https://istio.io/latest/docs/ops/configuration/traffic-management/protocol-selection/

Upvotes: 1

Related Questions