CODEWITHSUNDEEP
pythonairflow
amustafa
amustafa

Reputation: 858

Triggering an Airflow DAG from terminal not working

I'm trying to use airflow to define a specific workflow that I want to manually trigger from the command line.

I create the DAG and add a bunch of tasks.

dag = airflow.DAG(
    "DAG_NAME",
    start_date=datetime(2015, 1, 1),
    schedule_interval=None,
    default_args=args)

I then run in the terminal

airflow trigger_dag DAG_NAME

and nothing happens. The scheduler is running in another thread. Any direction is much appreciated. Thank You

Upvotes: 24

Views: 35235

Answers (6)

Taimoor Abbasi
Taimoor Abbasi

Reputation: 91

Now you can use the following command to trigger the dag from the cli.

airflow dags trigger -e 2024-06-02T01:10:00+00:00 <dag_id>

I was struggling with the time format but figured it out eventually so I thought I'll post it here if someone else is facing the same issue. Also if you are not sure about the dag_id you can use the airflow dags list command to get the dag_id

PS: Also make sure your DAG start date is small than the date that you're trying to trigger the dag for. For example, if your dag start date is 10th of June 2024, you won't be able to trigger dag for any date prior to 10th of june 2024.

Upvotes: 0

Data_explorer
Data_explorer

Reputation: 11

airflow trigger_dag -e <execution_date> <dag_id>

Upvotes: 0

Eden Wang
Eden Wang

Reputation: 71

You should 'unpause' the drag you what to trigger. use airflow unpause xxx_drag and then airflow trigger_dag xxx_drag and it should work.

Upvotes: 1

Anmol Karki
Anmol Karki

Reputation: 361

first make sure your database connection string on the airflow is working, weather it be on postgres, sqlite(by default) or any other database. Then run the command

 airflow initdb

This command should not be showing any connection errors

Secondly make sure your webserver is running on a separate thread

airflow webserver

Then run your schdeuler on a different thread 

airflow scheduler

Finally trigger your dag on a different thread after the scheduler is running

airflow trigger_dag dag_id

Also make sure the dag name and task are present in the dag and task list

airflow list_dags
airflow list_tasks dag_id

And if the dag is switched off in your UI then toggle it on.

Upvotes: 6

neolytics
neolytics

Reputation: 381

I just encountered the same issue.

Assuming you are able to see your dag in airflow list_dags or via the web server then:

Not only did I have to turn on the dag in the web UI, but I also had to ensure that airflow scheduler was running as a separate process.

Once I had the scheduler running I was able to successfully execute my dag using airflow trigger_dag <dag_id>

My dag configuration is not significantly different from yours. I also have schedule_interval=None

Upvotes: 38

sidd607
sidd607

Reputation: 1409

You may have disabled the workflow. To enable the workflow manually. Open up the airflow web server by 

$ airflow webserver -p 8080

Go to http://localhost:8080 . You should see the list of all available dags with a toggle button on/off. By default everything is set to off. Search for your dag and toggle your workflow. Now try triggering the workflow from terminal. It should work now.

Upvotes: 10

Related Questions