CODEWITHSUNDEEP
google-cloud-platformairflowgoogle-cloud-composerterraform-provider-gcp
Raunak Jhawar
Raunak Jhawar

Reputation: 1651

Change the default GCS location for DAGs for Cloud Composer

Is there a way to override the default path for Airflow DAGs. While creating a new environment and executing one, the composer environment is creating a multi-regional bucket in the project.

Are there ways to override the DAG location to another GCS location and also change the storage class to regional? I have found the core-dags_folder is the property used to specify the DAG location on the environment, but I do not understand why this is a blocked configuration and cannot be updated to any other GCS location by the developer

Upvotes: 2

Views: 1724

Answers (1)

hexacyanide
hexacyanide

Reputation: 91769

At the time, no, the default path for Airflow DAGs in Cloud Composer cannot be changed.

This is likely by design (which is why core-dags_folder is a blocked configuration value), because one or more managed components (like the Airflow web server) depends on these paths to coordinate DAG sync/etc. If you're curious, you can read more about the data synchronization here.

If you have a strong use case for a single-region storage class or an alternate DAG location, I would recommend filing feedback.

Upvotes: 2

Related Questions