CODEWITHSUNDEEP
google-bigquerygoogle-cloud-storage
Breandán
Breandán

Reputation: 1883

Permissions Error Exporting a table from Google BigQuery to Google Cloud Storage

Getting the following error as of 2017-07-08 when exporting a table to Cloud Storage.

Access Denied: BigQuery BigQuery: Permission denied while writing data

Working fine before that date, haven't changed anything in the project, big query or cloud storage.

JobId: alert-basis-89415:bquijob_6adfd86e_15d2de5ce1b

Upvotes: 14

Views: 8873

Answers (6)

gavinest
gavinest

Reputation: 348

I had the same error. All my IAM grants looked correct.

It turns out it was failing because the SA has permission to create files (object.Creator) but not to overwrite/replace files in the bucket. Because a file by the same name already existed in the bucket it failed until I removed the file (or you can give it permission to overwrite the file).

The error is confusing/misleading though.

Access Denied: BigQuery BigQuery: Permission denied while writing data."

Using bq cli to show the job details was helpful in debugging this issue.

bq show --format=prettyjson --job=true <<job-id>>

Upvotes: 0

Kyle Pennell
Kyle Pennell

Reputation: 6097

Make sure the service account has the right permissions to do things on that bucket. It's confusing because even if the bucket is public, that doesn't allow write access from just plain old anyone.

Upvotes: 0

Yogesh Devi
Yogesh Devi

Reputation: 689

I was getting a similar error. It was because I was using a bucket in region different from the BQ dataset, and a incorrect bucket URL. After I ensured following two

  • I double checked and fixed specified bucket URL is correct
  • I created bucket in same region that the data resides

I was good

Upvotes: 2

user2609980
user2609980

Reputation: 10484

Look up the service account's e-mail address, and add the account to the bucket ACL as a writer and file reader: 

gsutil acl ch -u [email protected]:W gs://bucket
gsutil acl ch -u [email protected]:R gs://bucket/1.csv
gsutil acl ch -u [email protected]:R gs://bucket/2.csv

Upvotes: 0

Breand&#225;n
Breand&#225;n

Reputation: 1883

It was a problem on Google's end. I found a topic in Google groups (can't seem to find it now) where you could supply your details and they applied a hotfix for those accounts before rolling it out proper.

Upvotes: -3

Victor Mota
Victor Mota

Reputation: 1269

It appears the user running the job doesn't have storage.objects.create permission to the GCS bucket - perhaps something changed with bucket permissions? Is this still recurring, can you explicitly add that user to the bucket ACLs?

Upvotes: 1

Related Questions