CODEWITHSUNDEEP
pythonparameter-passingairflow
Adt
Adt

Reputation: 333

In airflow can end user pass parameters to keys which are associated with some specific dag

i have searched many links but didn't find any solution to the problem i have. I have seen option to pass key/var into airflow UI ,but it is really confusing for end user to work as which key is associated with which dag. Is there any way to implement functionality like :

While running an airflow job, end user will be asked for values to some parameters and after entering those details airflow will run the job.

Upvotes: 10

Views: 3322

Answers (1)

s7anley
s7anley

Reputation: 2498

Unfortunately, it's not possible to wait for user input let say in Airflow UI. DAG's are programmatically authored which means defined as a code and they should not be dynamic since they are imported in web server, scheduler and workers in same time and has to be same.

There are two workarounds I came up with, and we use first in production for a while.

1) Create a small wrapper around Variables. For each DAG then load Variables and compose arguments which are then passed into Operators via default_arguments.

2) Add Slack operator which can be programmatically configured to wait for user input. Afterwards, propagate that information via XCOM into next Operator.

Upvotes: 7

Related Questions