CODEWITHSUNDEEP
javagoogle-bigquerydataflowgoogle-compute-engine
이용운
이용운

Reputation: 21

"Access Denied" when trying to access BigQuery on GCE

I developed and built a jar file is processing read from two BigQuery tables and then write it another BigQuery table. (There are three GCP projects and each project has own BigQuery table.) The jar file is a result building and packaging by Maven. I used Dataflow SDK dependency.

After building a jar file, I tested it on my Mac and I checked it works well. And then, I uploaded a jar file to GCE and tested it again. I thought it would work well but it occurs some error like this :

com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden { "code" : 403, "errors" : [ { "domain" : "global", "message" : "Access Denied: Dataset ${PROJECT_ID}:${DATASET_ID}: The user ${MY_ACCOUNT_ID} does not have bigquery.tables.list permission for dataset ${PROJECT_ID}:${DATASET_ID}.", "reason" : "accessDenied .... }

When I created GCE instance, I set service account properly. I am sure it`s not about this.

I do not understand why same jar runs differently, makes error or not?

The account on testing is member or owner of three projects. Before testing, I use this commands on my Mac and GCE :

  1. gcloud auth login
  2. gcloud config set project rgpkorea-datalake
  3. gcloud auth application-default login

In Java project, I created credential object and build BigQuery service with credential object.

GoogleCredential credential = GoogleCredential.getApplicationDefault(HTTP_TRANSPORT, JSON_FACTORY);
Bigquery bigquery = new Bigquery.Builder(HTTP_TRANSPORT, JSON_FACTORY, credential)
                .setApplicationName("BdtAndroidGaSessionMigrator" + CalendarUtil.getYesterdayAsDateFormat()).build();
List<Tables> tableList = bigquery.tables().list("${PROJECT_ID}", "${DATASET_ID}").setMaxResults(9999999L).execute().getTables();

For security, I replaced project id and dataset id as EL expression.

As you know as, "gcloud init" command is supported on Google-Cloud-SDK. I also tried "gcloud init" command, but result is not different. In my guess, above commands work differently depends on environments such as GCP instance or local machine.

How do I resolve "Access denied" issue? please help this.

P.S. Forgive my poor English. I doubt my English presents what I have problem to you exactly.

Upvotes: 0

Views: 2842

Answers (2)

Raviteja S
Raviteja S

Reputation: 1

I got it resolved by providing the the Service Account "Storage Admin" permission. More finer access control should be explored.

Upvotes: 0

이용운
이용운

Reputation: 21

I resolve this problem myself. For someone who encounter same problem like me.

To correct this, you may check permission is set on each project. Permission is set each account even though it`s service account. I mean you should check accounts both GCP login account and service account is added as a member to each project. In my case, I added a GCP login account but not service account.

I hope this help you

Upvotes: 2

Related Questions