istio sidecar not injected

Question

I'm working on a project deploying Azure Kubernetes Service.

I deployed AKS with internal load-balancer and Istio-based service mesh add-on enabled. https://learn.microsoft.com/en-us/azure/aks/istio-deploy-addon https://learn.microsoft.com/en-us/azure/aks/istio-deploy-ingress#enable-internal-ingress-gateway

I tried to deploy test apps and istio resources to test deployment, but realized Envoy sidecars were not injected.

kubectl get pod

NAME                                  READY   STATUS    RESTARTS   AGE
httpbin-deployment-65d859bd68-m8qbx   1/1     Running   0          9m49s
order-service-76d7f5b8f5-77js4        1/1     Running   0          9m49s
product-service-7566c548bd-hsjrq      1/1     Running   0          9m49s
rabbitmq-6ddd848578-qxghc             1/1     Running   0          9m48s
store-front-7cc6c7bb67-9fzzn          1/1     Running   0          9m48s

I checked istio-injection was enabled.

kubectl get namespace -L istio-injection

NAME                STATUS   AGE     ISTIO-INJECTION
aks-istio-egress    Active   5h16m
aks-istio-ingress   Active   5h16m
aks-istio-system    Active   5h16m
calico-system       Active   5h22m
default             Active   5h23m   enabled
gatekeeper-system   Active   5h22m
kube-node-lease     Active   5h23m
kube-public         Active   5h23m
kube-system         Active   5h23m
tigera-operator     Active   5h22m

Is there any cause about this problem?

Here are the yaml files I'm using.

app.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin-deployment
spec:
  selector:
    matchLabels:
      app: httpbin
  replicas: 1
  template:
    metadata:
      annotations:
        sidecar.istio.io/inject: "True"
      labels:
        app: httpbin
    spec:
      containers:
      - name: httpbin
        image: kennethreitz/httpbin:latest
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  selector:
    app: httpbin
  ports:
    - protocol: TCP
      port: 5001
      targetPort: 80

aks-store-sample.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: rabbitmq
spec:
  replicas: 1
  selector:
    matchLabels:
      app: rabbitmq
  template:
    metadata:
      labels:
        app: rabbitmq
    spec:
      nodeSelector:
        "kubernetes.io/os": linux
      containers:
      - name: rabbitmq
        image: mcr.microsoft.com/mirror/docker/library/rabbitmq:3.10-management-alpine
        ports:
        - containerPort: 5672
          name: rabbitmq-amqp
        - containerPort: 15672
          name: rabbitmq-http
        env:
        - name: RABBITMQ_DEFAULT_USER
          value: "username"
        - name: RABBITMQ_DEFAULT_PASS
          value: "password"
        resources:
          requests:
            cpu: 10m
            memory: 128Mi
          limits:
            cpu: 250m
            memory: 256Mi
        volumeMounts:
        - name: rabbitmq-enabled-plugins
          mountPath: /etc/rabbitmq/enabled_plugins
          subPath: enabled_plugins
      volumes:
      - name: rabbitmq-enabled-plugins
        configMap:
          name: rabbitmq-enabled-plugins
          items:
          - key: rabbitmq_enabled_plugins
            path: enabled_plugins
---
apiVersion: v1
data:
  rabbitmq_enabled_plugins: |
    [rabbitmq_management,rabbitmq_prometheus,rabbitmq_amqp1_0].
kind: ConfigMap
metadata:
  name: rabbitmq-enabled-plugins
---
apiVersion: v1
kind: Service
metadata:
  name: rabbitmq
spec:
  selector:
    app: rabbitmq
  ports:
    - name: rabbitmq-amqp
      port: 5672
      targetPort: 5672
    - name: rabbitmq-http
      port: 15672
      targetPort: 15672
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-service
spec:
  replicas: 1
  selector:
    matchLabels:
      app: order-service
  template:
    metadata:
      labels:
        app: order-service
    spec:
      nodeSelector:
        "kubernetes.io/os": linux
      containers:
      - name: order-service
        image: ghcr.io/azure-samples/aks-store-demo/order-service:latest
        ports:
        - containerPort: 3000
        env:
        - name: ORDER_QUEUE_HOSTNAME
          value: "rabbitmq"
        - name: ORDER_QUEUE_PORT
          value: "5672"
        - name: ORDER_QUEUE_USERNAME
          value: "username"
        - name: ORDER_QUEUE_PASSWORD
          value: "password"
        - name: ORDER_QUEUE_NAME
          value: "orders"
        - name: FASTIFY_ADDRESS
          value: "0.0.0.0"
        resources:
          requests:
            cpu: 1m
            memory: 50Mi
          limits:
            cpu: 75m
            memory: 128Mi
      initContainers:
      - name: wait-for-rabbitmq
        image: busybox
        command: ['sh', '-c', 'until nc -zv rabbitmq 5672; do echo waiting for rabbitmq; sleep 2; done;']
        resources:
          requests:
            cpu: 1m
            memory: 50Mi
          limits:
            cpu: 75m
            memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
  name: order-service
spec:
  type: ClusterIP
  ports:
  - name: http
    port: 3000
    targetPort: 3000
  selector:
    app: order-service
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: product-service
spec:
  replicas: 1
  selector:
    matchLabels:
      app: product-service
  template:
    metadata:
      labels:
        app: product-service
    spec:
      nodeSelector:
        "kubernetes.io/os": linux
      containers:
      - name: product-service
        image: ghcr.io/azure-samples/aks-store-demo/product-service:latest
        ports:
        - containerPort: 3002
        resources:
          requests:
            cpu: 1m
            memory: 1Mi
          limits:
            cpu: 1m
            memory: 7Mi
---
apiVersion: v1
kind: Service
metadata:
  name: product-service
spec:
  type: ClusterIP
  ports:
  - name: http
    port: 3002
    targetPort: 3002
  selector:
    app: product-service
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: store-front
spec:
  replicas: 1
  selector:
    matchLabels:
      app: store-front
  template:
    metadata:
      labels:
        app: store-front
    spec:
      nodeSelector:
        "kubernetes.io/os": linux
      containers:
      - name: store-front
        image: ghcr.io/azure-samples/aks-store-demo/store-front:latest
        ports:
        - containerPort: 8080
          name: store-front
        env:
        - name: VUE_APP_ORDER_SERVICE_URL
          value: "http://order-service:3000/"
        - name: VUE_APP_PRODUCT_SERVICE_URL
          value: "http://product-service:3002/"
        resources:
          requests:
            cpu: 1m
            memory: 200Mi
          limits:
            cpu: 1000m
            memory: 512Mi
---
apiVersion: v1
kind: Service
metadata:
  name: store-front
spec:
  type: ClusterIP
  ports:
  - port: 5000
    targetPort: 8080
  selector:
    app: store-front

gateway.yaml

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: app-gateway
spec:
  selector:
    istio: aks-istio-ingressgateway-internal
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
      - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: store-front
spec:
  hosts:
  - "*"
  gateways:
  - app-gateway
  http:
  - match:
    - uri:
        prefix: /sample
    route:
    - destination:
        host: store-front
        port:
          number: 5000
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: web-service
spec:
  hosts:
  - "*"
  gateways:
  - app-gateway
  http:
  - match:
    - uri:
        prefix: /app
    route:
    - destination:
        host: web-service
        port:
          number: 5001

Answer 1

Istio 1.21 is not officially supported on Kubernetes versions above 1.26. This mismatch likely causes the issue with the Istio sidecar injection. Try using latest istio version.

upgrade Istio to a version compatible with Kubernetes 1.29 or 28 whichever you are using

You can download the latest istio version from here-

curl -L https://istio.io/downloadIstio | sh -
cd istio-<latest-version> # replace <latest-version> with the actual directory name
export PATH=$PWD/bin:$PATH

install the istio latest version you downloaded

istioctl install --set profile=demo -y

Label the namespace for Istio Injection

Reapply the deployment configurations to recreate the pods with the correct annotations