Airflow kubernetesExecutor kubernetes_environment_variables and kubernetes_secrets not passed to generated pod

Question

I'm deploying Apache Airflow 2.9.2 with the Kubernetes Executor in an AKS cluster.

KubernetesExecutor will create a pod for each of the tasks to be executed.

I see that this pod is defined by a pod template. You can use the default pod template or you can create a pod template from the official helm chart.

I have created this template in my helm chart:

podTemplate: |-
      apiVersion: v1
      kind: Pod
      metadata:
        name: airflow-task-pod
      spec:
        serviceAccountName: airflow-worker
        containers:
          - name: base
            image: {{ .Values.images.airflow.repository }}:{{ .Values.images.airflow.tag }}
            imagePullPolicy: {{ .Values.images.airflow.pullPolicy }}
            resources:
              requests:
                cpu: 0.5
                memory: "1Gi"
              limits:
                cpu: 2
                memory: "3Gi"
            env:
              - name: AIRFLOW__CORE__EXECUTOR
                value: LocalExecutor
              {{- include "custom_airflow_environment" . | indent 6 }}
              - name: AIRFLOW_CONN_AIRFLOW_DB
                valueFrom:
                  secretKeyRef:
                    name: airflow-metadata-connection
                    key: connection
              - name: AIRFLOW__DATABASE__SQL_ALCHEMY_CONN
                valueFrom:
                  secretKeyRef:
                    name: airflow-metadata-connection
                    key: connection
              - name: AIRFLOW__CORE__FERNET_KEY
                valueFrom:
                  secretKeyRef:
                    name: airflow-fernet-key
                    key: fernet-key
              
        tolerations:
          - key: "kubernetes.azure.com/scalesetpriority"
            operator: "Equal"
            value: "spot"
            effect: "NoSchedule"
        affinity:
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
                - matchExpressions:
                    - key: "kubernetes.azure.com/scalesetpriority"
                      operator: In
                      values:
                        - "spot"

This is working correctly, since I can pass to the pod the env variables that I have defined in the extraEnv section and the secrets that I have created in the secret section.

{{- include "custom_airflow_environment" . | indent 6 }}

This will execute the function defined in the _helpers.yaml from the airflow helm chart.

What I find strange is that when the helm chart detects that it is KubernetesExecutor, in the deployments it executes the same helper function, which creates env variables with the prefix AIRFLOW__KUBERNETES_ENVIRONMENT_VARIABLES__ and secrets with the prefix AIRFLOW__KUBERNETES_SECRETS__.

As you can see in the image, I load this parameters in the configurations

I was expecting that this secrets and env variables are passed to the Pod created by KubernetesExecutor, but it is not the case.

I see this PR from 2019, where there was documentation related to the kubernetes_environment_variables and kubernetes, but I cannot see it any more in the current master branch in the airflow github repository.

In the airflow repository I see that the pod that is generated by the pod_generator.py file, and it is creating only one env variable, in this line of code.

So I wonder, if this is really something that was used before and now it is not used anymore? should it be removed from the helm chart or am I configuring it incorrectly?