CODEWITHSUNDEEP
airflow
Rishabh Jain
Rishabh Jain

Reputation: 23

Can this warning be avoided in apache airflow 2.0?

I am using airflow v2.0 on windows 10 WSL (Ubuntu 20.04). The warning message is :

/home/jainri/.local/lib/python3.8/site-packages/airflow/models/dag.py:1342: PendingDeprecationWarning: The requested task could not be added to the DAG because a task with task_id create_tag_template_field_result is already in the DAG. Starting in Airflow 2.0, trying to overwrite a task will raise an exception.
  warnings.warn(
Done.

Due to this warning, the dags showing in web UI are also some example dags included with apache airflow. I have setup **AIRFLOW_HOME** and it also picks up dags from there. But the list of example dags also displayed. I have posted the image of WEB UI also.

WebUI

This is the dag below that I am trying to run:


import datetime
import logging

from airflow import DAG
from airflow.operators.python_operator import PythonOperator

#
# TODO: Define a function for the python operator to call
#

def greet():
    logging.info("Hello Rishabh!!")

dag = DAG(
    'lesson1.demo1',
    start_date = datetime.datetime.now()
    end_date 
)

#
# TODO: Define the task below using PythonOperator
#

greet_task = PythonOperator(
    task_id='greet_task',
    python_callable=greet,
    dag=dag
)

Also, the main issue is like the list of dags showing in webUI is some example dags. That shows up a huge list along with my own dags. Which makes it cumbersome to look for my own dags.

Upvotes: 1

Views: 1706

Answers (1)

kaxil
kaxil

Reputation: 18844

I found the issue, the error you are seeing is because of airflow/example_dags/example_complex.py (one of the example_dags) that is shipped with Airflow.

Disable loading of example_dags by setting AIRFLOW__CORE__LOAD_EXAMPLES=False as an environment variable or set [core] load_examples = False in airflow.cfg (docs).

Upvotes: 1

Related Questions