CODEWITHSUNDEEP
airflowgoogle-cloud-composer
reddikih
reddikih

Reputation: 11

Can't use plugin module with Cloud Composer

I'm trying to use the Cloud Composer to run my workflow. I wanted to use "GoogleCloudStorageToGoogleCloudStorageOperator" operator which is available from Apache Airflow v1.10, but not being supported in current cloud composer (it supports only Apache Airflow v1.9 for now (2019/01/16)).

Following the guidance of the Google's blog post, I added the operator to a cloud composer environment by myself, and it worked well until a few days ago.

However, for now, when I tried to create a new cloud composer env and to deploy the same DAG that worked well previously, I got a following error message on the Airflow Web UI. And DAG is failed.

Broken DAG: [/home/airflow/gcs/dags/xxx.py] Relationships can only be set between Operators; received GoogleCloudStorageToGoogleCloudStorageOperator

I couldn't understand why this error occurred even if I used the same code and followed the same procedure to deploy my DAG to the cloud composer.

I appreciate for those who give me any advice to solve this problem.

Upvotes: 1

Views: 825

Answers (2)

VictorGGl
VictorGGl

Reputation: 1916

This was due to a bug in Composer 1.4.2 which was already fixed.

Airflow error importing DAG using plugin - Relationships can only be set between Operators

Upvotes: 0

ryw
ryw

Reputation: 9655

Try out the DAG on Astronomer Cloud (http://astronomer.io/cloud), free 30 day trial.

Disclosure: I work at Astronomer.

Upvotes: -5

Related Questions