CODEWITHSUNDEEP
continuous-integrationairflow
Prashant Tiwari
Prashant Tiwari

Reputation: 99

Airflow CI Setup for Dags

If I want to push airflow DAG to git repo and then have jenkins job executed using a webhook, what do I do from there on for airflow to pick up the new DAG ?

Is there any utility/operator to fill the dagbag from a git repo ? If not, can someone please share how they setup filling dagbags with new DAGs?

Upvotes: 2

Views: 1822

Answers (1)

listik
listik

Reputation: 229

In my particular case, I have been using supervisord to keep our Airflow scheduler and webserver up.

Last command on my Jenkins job is a supervisord restart airflow_scheduler;supervisord restart airflow_webserver that refreshes both Airflow processes.

Since Airflow is keeping current state of processes in it's database, nothing should break because of resetting the scheduler or the webserver.

Not sure if this is the best Airflow-y thing to do, but I has worked for me so far.

Upvotes: 1

Related Questions