CODEWITHSUNDEEP
google-cloud-platformairflowgcloudgoogle-cloud-composer
ACHO-Chen
ACHO-Chen

Reputation: 11

For GCP Composer, How to delete a DAG with "gcloud" command in GCP Compute Engine

I want to run a "gcloud" command:

gcloud composer environments run --location us-central1 my-composer-instance delete_dag -- my_dag_id

Inside a Compute Instance. What's the best way to do it?

So far, running the "delete_dag" command will return this error message:

ERROR: (gcloud.composer.environments.run) Invalid choice: 'delete_dag'. Did you mean 'list_dags'?

Then if I run the "list_dags" command, it will ask me to install kubectl with:

$ gcloud components install kubectl

But when I install kubectl:

ERROR: (gcloud.components.install) You cannot perform this action because this Cloud SDK installation is managed by an external package manager. Please consider using a separate installation of the Cloud SDK created through the default mechanism described at: https://cloud.google.com/sdk/

Upvotes: 1

Views: 1252

Answers (2)

ACHO-Chen
ACHO-Chen

Reputation: 11

Use apt install kubectl instead of gcloud components install kubectl solved my problem.

Upvotes: 0

Mahboob
Mahboob

Reputation: 1955

You need to installing Cloud SDK in to the VM instance to execute the command. To delete a DAG you need to remove the Python .py file for the DAG from the environment's dags folder in Cloud Storage. To removing a DAG you can follow the doc.

gcloud composer environments storage dags delete \
    --environment ENVIRONMENT_NAME \
    --location LOCATION \
    DAG_NAME.py

Or (Reference)

gcloud composer environments run --location LOCATION ENVIRONMENT_NAME delete_dag --DAG_NAME

where:
ENVIRONMENT_NAME is the name of the environment.
LOCATION is the Compute Engine region where the environment is located.
DAG_NAME is the name of the DAG to delete.

Upvotes: 1

Related Questions