CODEWITHSUNDEEP
airflowairflow-2.x
Ayan Bhattacharya
Ayan Bhattacharya

Reputation: 3

Not able to find my DAG in airflow WEB UI even though the dag is in correct folder

I have been trying past 2 days to resolve this. There is a DAG python script which I created and saved it in the dags folder in airflow which is being referred to in the "airflow.cfg" file. The other dags are getting updated except for one dag. I tried to restart scheduler and also tried to reset the airflow db using airflow db reset and then tried airflow db init once again but still the same issue exists.

Upvotes: 0

Views: 985

Answers (1)

TJaniF
TJaniF

Reputation: 1046

Some ideas on what you could check:

  • Do all of your DAGs have a unique dag_id? (I lost a few hours to this once, if two dags have the same name, the scheduler will randomly pick one to display with every dag_dir_list_interval)
  • If you are using a the @dag decorator: are you calling the DAG below its definition? Like so:
from airflow.decorators import dag, task
from pendulum import datetime

@dag(
    dag_id="unique_name",
    start_date=datetime(2022,12,10),
    schedule=None,
    catchup=False
)
def my_dag():

    @task
    def say_hi():
        return "hi"

    say_hi()

# without this line the DAG will not show up in the UI
my_dag()
  • What is the output of airflow run dags list and airflow run dags list-import-errors ?
  • If you have a lot of DAGs in your environment you might want to increase the dagbag_import_timeout.
  • Does your DAG work if thrown into a new Airflow instance (the easiest way to check is by spinning up a project with the Astro CLI and putting the dag into the dags folder created by astro dev init)

Disclaimer: I work at Astronomer, who develops the Astro CLI as an OS project.

Upvotes: 1

Related Questions