CODEWITHSUNDEEP
airflowgoogle-cloud-composer
aName
aName

Reputation: 3053

composer doesn't use imported variable

I import a json file that define variables to be used by composer.
I used the gcloud beta composer environments storage data import command, I can see that the file is imported correctly to the <composer_bkt>/data/variables, however, when I accessed to airflow webUI, I find that there is no variable declared !

Upvotes: 1

Views: 686

Answers (2)

TC Arlen
TC Arlen

Reputation: 1482

Thank you for the answer @itroulli. It appears that my composer version (v2.2.5) failed on that command, but instead a command of this form worked:

gcloud composer environments run <ENVIRONMENT_NAME> --location=<LOCATION> variables -- import /home/airflow/gcs/data/variables/variables.json

I'll leave this for anyone else that comes across this problem

Upvotes: 0

itroulli
itroulli

Reputation: 2094

Moving the file to to <COMPOSER_BCKT>/data/variables is not enough by itself to import the variables to Airflow. Apart from that you need to run the Airflow CLI command:

airflow variables --i <JSON_FILE>

To do that in Composer you have to run the following command as described here:

gcloud composer environments run <ENVIRONMENT_NAME> --location=<LOCATION> variables -- --i /home/airflow/gcs/data/variables/variables.json

Upvotes: 3

Related Questions