CODEWITHSUNDEEP
gcloudgoogle-cloud-composer
Yoli
Yoli

Reputation: 65

When creating a new cloud composer env, is it possible to set the bucket to preexisting one?

So I already have an empty storage bucket created for this and I don't want composer to create its own bucket for the dags - I'd like to use the one already created.

It's not ideal to have it just create a random bucket and then go 

gcloud composer environments run test-environment  --location europe-west1 variables -- --set gcs_bucket gs://my-bucket

I've dug around the docs but it seems you cannot go around it creating a brand new bucket every time?

Upvotes: 2

Views: 1691

Answers (2)

ch_mike
ch_mike

Reputation: 1576

Currently, it is not possible.

In the environment’s configuration in Cloud Composer API, the dagGcsPrefix parameter is output only, you cannot set it. Documentation also mentions a Cloud Storage bucket is always created along with the Composer environment, the name of the bucket is based on the environment’s region, name and a random Id.

You may want to “Star” this Feature Request for the mentioned functionality, to receive notifications whenever an update on this regard is published. You can also review or subscribe to the Cloud Composer release notes to be updated about recently added features.

Upvotes: 7

Feng Lu
Feng Lu

Reputation: 691

You are right, this is currently not supported in Composer.

Upvotes: 2

Related Questions