CODEWITHSUNDEEP
airflow
Stone cold Cold
Stone cold Cold

Reputation: 152

Changing Dependencies of the task using UI in apache airflow

Is there a way to set the tasks dependency from the UI itself instead of defining it in a DAG file. I know we can set dependencies like task1 >> task2 >> task3. But, can I do the same using the localhost webserver UI.

Upvotes: 5

Views: 513

Answers (1)

y2k-shubham
y2k-shubham

Reputation: 11607

[Q] Does Airflow natively support UI-based DAG editing?

[A] No

[Q] Can the same 'effect' be achieved?

[A] Yes

If you cook-up a DAG-builder code that (say) parses a JSON / YAML stored in Airflow Variable (which contain information of what DAGs, operators to create and how to link them together) to generate DAGs, then by editing those Variables (from UI itself), you'll be able to modify the structure of your DAG

Some good references are

[Q] Should this be done (recommended practise)?

[A] As long as the DAGs that your are generating in this way are immutable (their structure doesn't keep on changing every now and then), it is fine. And while it might be enticing to have dynamically modifying structure (sounds very cool / powerful), in my opinion it should be avoided since that would compromise system's sanity (system will become difficult to reason about, debugg-ability will be hurt)

Further reads

Upvotes: 2

Related Questions