Best practice to run 1 time setup tasks

Question

I often run into a scenario like this in my dags:

• Setup a long-term resource (like a SQL table) or skip if it already exists
• Do some work, create temporary resources
• Store results in the long-term resource
• Clean up temporary resources

Is there an airflow way or a best practice to handle 1-off tasks that would only run once?

Answer 1

You can create a DAG that will run only one time as:

from airflow.models import DAG

dag = DAG(
    dag_id='my_one_time_dag',
    schedule_interval='@once'
)