CODEWITHSUNDEEP
airflowgoogle-cloud-composer
Indrajeet Gour
Indrajeet Gour

Reputation: 4500

DAG seems to be missing from google Airflow UI

We are having google Airflow environment set for our needs.

I have read a lot on the stackoverflow but everyone is saying to restart your webserver, which I think I can not do, as it is managed by google.

All the time when we deploy new DAG into environment, its always like DAG is missing.

What is happening is it - it take few hours before everything work fine after deployment, but until that time its hard for me to understand what is wrong and how to fix that.

Could you please help me get rid of this issue permanently.

Please let me know if any more information required here. Thanks in advance.

Upvotes: 0

Views: 404

Answers (1)

Hussein Awala
Hussein Awala

Reputation: 5110

Every dag_dir_list_interval, the DagFileProcessorManager process list the scripts in the dags folder, then if the script is new or processed since more than the min_file_process_interval, it creates a DagFileProcessorProcess process for the file to process it and generate the dags.

You can check what do you have as values for these variables, and reduce them to add the dag ASAP to the Metadata.

Upvotes: 1

Related Questions