CODEWITHSUNDEEP
azureistioazure-container-instancesistio-sidecar
AndyMoose
AndyMoose

Reputation: 425

Is there a way to run Istio sidecar within virtual-node on azure?

I have an AKS cluster on Azure with virtual-nodes enabled (virtual-kubelet/azure-aci v1.3.2) and it works OK (a little flakey, but does mostly work). My problem is that as soon as I enable istio side car injection on any deployment I want to run on a virtual-node, the istio-proxy sidecar won't start, preventing the entire pod from starting up. I believe the problem is that v1:status.podIP is not yet supported in virtual-kubelet/azure-aci, and the istio sidecar spec makes use of it.

Has anyone had this problem? I can't find anything much about istio + virtual kubelet on the googlez.

I'm working around the problem for the time being by using PERMISSIVE mtls mode on the one dependency it relies on being able to make http calls to, and turning off side car injection via an annotation on the deployment. This is not ideal.

Thanks

Upvotes: 1

Views: 398

Answers (1)

iarunpaul
iarunpaul

Reputation: 117

Edit: The Istio CNI plugin can be a workaround. Haven't tried it but the documentation says:

For application pods in the Istio service mesh, all traffic to/from the pods needs to go through the sidecar proxies (istio-proxy containers). This istio-cni Container Network Interface (CNI) plugin will set up the pods' networking to fulfill this requirement in place of the current Istio injected pod initContainers istio-init approach.

Reference: https://github.com/istio/istio/tree/master/cni

Istio uses an init container to get the istio container started. But Init containers are not yet supported in Virtual Nodes. That explains why the Istio is not getting started in your virtual nodes.

The microsoft documentation says:

Virtual Nodes functionality is heavily dependent on ACI's feature set. In addition to the quotas and limits for Azure Container Instances, the following scenarios are not yet supported with Virtual nodes:

  • Using service principal to pull ACR images. Workaround is to use Kubernetes secrets
  • Virtual Network Limitations including VNet peering, Kubernetes network policies, and outbound traffic to the internet with network security groups.
  • Init containers
  • Host aliases
  • Arguments for exec in ACI
  • DaemonSets will not deploy pods to the virtual nodes
  • Virtual nodes support scheduling Linux pods. You can manually install the open source Virtual Kubelet ACI provider to schedule Windows Server containers to ACI.
  • Virtual nodes require AKS clusters with Azure CNI networking.
  • Using api server authorized ip ranges for AKS.
  • Volume mounting Azure Files share support General-purpose V1. Follow the instructions for mounting a volume with Azure Files share
  • Using IPv6 is not supported.

Upvotes: 1

Related Questions