CODEWITHSUNDEEP
google-cloud-platformgoogle-bigquerygoogle-cloud-rungoogle-iamgoogle-cloud-iam
DarioB
DarioB

Reputation: 1609

Google Cloud Run permissions to query bigquery

I have a small python app running in google cloud run with docker. The application is triggered by http requests, executes a query in big query and return the result. Unfortunately I get the following permission error:

Reason: 403 POST https://bigquery.googleapis.com/bigquery/v2/projects/XXXX/jobs: Access Denied: Project XXXX: User does not have bigquery.jobs.create permission in project XXXX.\n\n(job ID: XXXX-XX-XX-XX-XXXX)\n\n

I understand I need to give access from cloud run to big query. How do I do it? to which user? how can i find out?

Upvotes: 2

Views: 6154

Answers (3)

Pradeep Bhadani
Pradeep Bhadani

Reputation: 4721

One of the issues could be that Service Account which your Cloud Run job is using does not have permissions on BigQuery.

You can update the service account permission and add roles/bigquery.user role to create a job. Also, based on your application requirement add relevant roles. You can see details about different BigQuery roles here.

A good rule is provide only required permissions to a service account.

I hope this helps.

Upvotes: 2

winwiz1
winwiz1

Reputation: 3164

The application is triggered by http requests, executes a query in big query and return the result.

From the security standpoint the permissions required are identical to those used by the custom website from this solution. I'm the author. The website is also triggered by http requests, executes a query in BQ and returns the result. And granting the permission to create jobs (via bigquery.jobUser role) is not enough.

You can grant the required permissions to the service account in different ways (e.g. a more sweeping permission and a more restricted one), the details are here at the Step 6.

Generally speaking, the more restricted and the more granular the permissions are the better for security.

I'm adding extra clarifications and also pasting specific instructions related to Google's tools usage.

  1. To add the permission to create and run jobs (the BQ error message says this permission is lacking) execute the command: 

    gcloud projects add-iam-policy-binding <project-name> --member=serviceAccount:<sa-name>@<project-name>.iam.gserviceaccount.com --role roles/bigquery.jobUser

    The command can be executed in Cloud Shell, open it using the "Activate Cloud Shell" icon in BigQuery Web UI or from other Google Console page. Replace the placeholders:

    <sa-name> - replace with service account name used by Cloud Run,
    <project-name> - replace with the project name.

    The command adds the role bigquery.jobUser to the service account. Do not add other permissions/roles to solve the inability to create/run jobs because excessive permissions are bad for security.

  2. Another permission is required to read BQ data. There are two options to add it:

    • Grant the bigquery.dataViewer role to the service account:
    gcloud projects add-iam-policy-binding <project-name> --member=serviceAccount:<sa-name>@<project-name>.iam.gserviceaccount.com --role roles/bigquery.dataViewer

    Then proceed to the next step. Not recommended unless you are using a throw-away project. The drawback of this approach is granting permissions to view all project datasets.

    • Take more granular approach (recommended) by allowing the service account to query one dataset only. This is the approach described below.

    Execute the commands replacing <ds-name> with the dataset name (used by your query):

    bq show --format=prettyjson <ds-name>  >/tmp/mydataset.json  
vi /tmp/mydataset.json

    Using vi, append the following item to the existing access array and replace the placeholders before saving the file:

    ,
{  
"role": "READER",  
"userByEmail": "[<sa-name>@<project-name>.iam.gserviceaccount.com](mailto:<sa-name>@<project-name>.iam.gserviceaccount.com)"  
}

    Execute the command to effect the changes for the dataset:

    bq update --source /tmp/mydataset.json <ds-name>

Upvotes: 2

John Hanley
John Hanley

Reputation: 81356

You need to add BiqQuery permissions via IAM Roles to the service account assigned to Cloud Run.

To allow Cloud Run to create Big Query jobs (bigquery.jobs.create) you need one of the following roles:

  • roles/bigquery.user
  • roles/bigquery.jobUser

The service account for Cloud Run is displayed in the Google Cloud Console in the Cloud Run section for your service. Most likely this is Compute Engine default service account.

To add a BiqQuery role, you can use the Google Cloud Console. Go to IAM, find the service account. Add roles to the service account.

Documentation:

Upvotes: 8

Related Questions