CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-sqlgoogle-cloud-composer
Tomxiao
Tomxiao

Reputation: 1

How can I create external data source in BigQuery and connect to composer cloud sql?

We have a composer environment in GCP and we want to export the airflow database in the tenant project to BigQuery. Is there a way to do so? Composer

Upvotes: 0

Views: 282

Answers (1)

guillaume blaquiere
guillaume blaquiere

Reputation: 75910

I searched in all the location of Composer (on GKE with the secret and the cloud SQL proxy), in the Airflow db connexion, I got all the required element to connect my external source to BigQuery and I got this error:

enter image description here

So, you can't.....

EDIT 1:

Firstly, you can use the automatically configured connexion

Then, use the MySqlOperator to request the database and do what you want. Sometime it's easier to use the MySqlHook.

enter image description here

Upvotes: 1

Related Questions