Airflow Permission denied while getting Drive credentials

Question

I am trying to run a bigquery query on Airflow with MWAA.

This query uses a table that is based on a Google Sheet. When I run it, I have the following error:

google.api_core.exceptions.Forbidden: 403 Access Denied: BigQuery BigQuery: Permission denied while getting Drive credentials.

I already have a working Google cloud connection on Airflow with an admin service account.

Also:

• This service account has access to the google sheet
• I added https://www.googleapis.com/auth/drive in the scopes of the Airflow connection
• I re-generated a JSON file

Am I doing something wrong? Any idea what I can do to fix this problem?

Thanks a lot

Answer 1

I fixed my issue by creating a NEW Airflow connection. It's a new google cloud connection, with the exact same values as the default google_cloud_default values. Now it works perfectly.

Hope it can help !