CODEWITHSUNDEEP
google-cloud-platformairflowgoogle-cloud-composerk8s-serviceaccount
Hutch
Hutch

Reputation: 501

Can Cloud Composer run Dags with different Service Accounts in the same Composer environment?

I have several Dags setup to run the KubernetesPodOperator in Cloud Composer / Airflow, they run under the Composer Environment Service Account, I want to know if there is away of running a Dag with a different Service Account to the one set in the Composer environment ?

I have looked thought the documentation but cannot see how this might be done

When you create an environment, you specify a service account. Your environment's cluster uses this service account to run pods with different environment components, such as Airflow workers and schedulers. https://cloud.google.com/composer/docs/how-to/access-control

https://cloud.google.com/composer/docs/composer-2/authentication

I think you might be able to do this with impersonate service account but there are no examples of how to do this https://github.com/apache/airflow/issues/12061

Upvotes: 2

Views: 3429

Answers (1)

Jarek Potiuk
Jarek Potiuk

Reputation: 20097

If you use GCP operators to interface with Google services, you can do it through different connections.

You can define any number of connections you want that should be 'GCP' connections, and use those connections in your DAGS/tasks. This way you can even have different service account for different tasks in the same DAG.

See https://airflow.apache.org/docs/apache-airflow-providers-google/stable/connections/gcp.html

Upvotes: 4

Related Questions