CODEWITHSUNDEEP
python-3.xkubernetesairflow-schedulerairflow
Naveen Kamagani
Naveen Kamagani

Reputation: 11

Airflow Kubernetes executor , Tasks in queued state and pod says invalid image and uses local executor

strong textI have installed airflow in docker and using kubernetes executor unable to trigger dags. Dag runs are in queued state. KubernetesExecutor is creating pod but it says invalid image. If i describe the pod it uses local executor instead of kubernetes executor. Please help

Attaching log file for reference

**kubectl describe pod tablescreationschematablescreation-ecabd38a66664a33b6645a72ef056edc
Name:         swedschematablescreationschematablescreation-ecabd38a66664a33b6645a72ef056edc
Namespace:    default
Priority:     0
Node:         10.73.96.181
Start Time:   Mon, 11 May 2020 18:22:15 +0530
Labels:       airflow-worker=5888feda-6aee-49c8-a94b-39cbe5758062
              airflow_version=1.10.10
              dag_id=Swed-schema-tables-creation
              execution_date=2020-05-11T12_52_09.829627_plus_00_00
              kubernetes_executor=True
              task_id=Schema_Tables_Creation
              try_number=1
Annotations:  <none>
Status:       Pending
IP:           172.17.0.46
IPs:
  IP:  172.17.0.46
Containers:
  base:
    Container ID:
    Image:         :
    Image ID:
    Port:          <none>
    Host Port:     <none>
    Command:
      airflow
      run
      Swed-schema-tables-creation
      Schema_Tables_Creation
      2020-05-11T12:52:09.829627+00:00
      --local
      --pool
      default_pool
      -sd
      /root/airflow/dags/User_Creation_dag.py
    State:          Waiting
      Reason:       InvalidImageName
    Ready:          False
    Restart Count:  0
    Environment:
      AIRFLOW__CORE__EXECUTOR:          LocalExecutor
      AIRFLOW__CORE__SQL_ALCHEMY_CONN:  postgresql://airflowkube:[email protected]:5434/airflowkube
    Mounts:
      /root/airflow/dags from airflow-dags (ro)
      /root/airflow/logs from airflow-logs (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-64cxg (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  airflow-dags:
    Type:          HostPath (bare host directory volume)
    Path:          /data/Naveen/Airflow/dags
    HostPathType:
  airflow-logs:
    Type:          HostPath (bare host directory volume)
    Path:          /data/Naveen/Airflow/Logs
    HostPathType:
  default-token-64cxg:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-64cxg
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason         Age                     From                           Message
  ----     ------         ----                    ----                           -------
  Normal   Scheduled      <unknown>               default-scheduler              Successfully assigned default/swedschematablescreationschematablescreation-ecabd38a66664a33b6645a72ef056edc to evblfnclnullnull1538
  Warning  Failed         2m15s (x12 over 4m28s)  kubelet, evblfnclnullnull1538  Error: InvalidImageName
  Warning  InspectFailed  2m (x13 over 4m28s)     kubelet, evblfnclnullnull1538  Failed to apply default image tag ":": couldn't parse image reference ":": invalid reference format**

Upvotes: 0

Views: 3033

Answers (1)

Sander Ardinois
Sander Ardinois

Reputation: 96

It looks like there is no image defined for the workers. You can set this in the airflow.cfg. Be sure to set these worker_container_repository and worker_container_tag. These are null by default which results in the behaviour you're experiencing. A working example:

 AIRFLOW__KUBERNETES__WORKER_CONTAINER_REPOSITORY: "apache/airflow"
 AIRFLOW__KUBERNETES__WORKER_CONTAINER_TAG: "1.10.10-python3.6"
 AIRFLOW__KUBERNETES__RUN_AS_USER: "50000"

The airflow docs have some more detailed info on the available properties: https://airflow.apache.org/docs/stable/configurations-ref.html#kubernetes

Upvotes: 4

Related Questions