CODEWITHSUNDEEP
airflowairflow-scheduler
chaooder
chaooder

Reputation: 1506

Airflow: Trigger DAG via UI with Parameters/Config

I see that one can trigger_dag with parameters/config key-value pairs using the airflow command line:

For Apache Airflow, How can I pass the parameters when manually trigger DAG via CLI?

In my case, I would like to centralize all operations for airflow via the airflow UI (preferably no CLI access should be granted), which has a button for triggering the dags:

enter image description here

How can I pass the external parameters/configs via the UI?

Upvotes: 17

Views: 51154

Answers (4)

Alexander Martins
Alexander Martins

Reputation: 383

Add this line to your airflow.cfg file:

[webserver] ... show_trigger_form_if_no_params = True

Then restart the webserver.

Upvotes: 2

Naveen Bussari
Naveen Bussari

Reputation: 1

You can pass a configuration in dictionary format while triggering a dag using dag_run.conf parameter. This configuration enables us to use a dynamic value for a variable for each run.

In the dag, the code to access a config passed via dag_run.conf is as follows:

dag_run_task = BashOperator(
    task_id= 'dag_run_conf_task',
    bash_command= "echo  The sample value given via dag_run config is  {{dag_run.conf['key1']}} ",
)

Pass the config while triggering the dag.

triggering dag with custom config

passing conf in the UI

Final restult

final log

Make sure to core.dag_run_conf_overrides_params is set to True, so passing any configuration here will override task params which can be accessed via {{ params }}.

Upvotes: 0

Ramji
Ramji

Reputation: 138

With latest versions [we use 2.2.2] of Airflow we only pass config json

When we click on the Trigger/Play button available against the DAG, we get a drop down with 2 options

  1. Trigger DAG
  2. Trigger DAG w/ config

enter image description here Clicking on the second option you can pass the conf in json format and that can be used in the script and its usage can be viewed while you create the configuration JSON as below.

To access configuration in your DAG use {{ dag_run.conf }}. As core.dag_run_conf_overrides_params is set to False, so passing any configuration here won't override task params.

enter image description here

Upvotes: 10

Daniel Huang
Daniel Huang

Reputation: 6538

You're in luck, assuming you're on a recent version of Airflow or can upgrade. Support for triggering a DAG run with a config blob was added in Airflow 1.10.8 under AIRFLOW-5843 in #5843. It's not through the same Trigger DAG icon you've pointed to, but it's through creating a DAG Run from Browse->DAG Runs->Create.

Screenshot of the new form that supports conf copied below from the pull request that added it.

Add DAG Run

The form will also validate the JSON of the conf which may actually be a step beyond the CLI.

Upvotes: 10

Related Questions