CODEWITHSUNDEEP
hadoopgoogle-cloud-platformgoogle-cloud-storagegoogle-cloud-dataproc
Mitya XMitya
Mitya XMitya

Reputation: 1139

Does not have storage.objects.get access

I can not solve GCS bucket permission issue when submitting job to Dataproc.

Here is what I'm doing:

  1. Created a project
  2. Created a bucket xmitya-test
  3. Created a cluster:
gcloud dataproc clusters create cascade --bucket=xmitya-test \
    --master-boot-disk-size=80G --master-boot-disk-type=pd-standard \
    --num-master-local-ssds=0 --num-masters=1 \
    --num-workers=2 --num-worker-local-ssds=0 \
    --worker-boot-disk-size=80G --worker-boot-disk-type=pd-standard \
    --master-machine-type=n1-standard-2 \
    --worker-machine-type=n1-standard-2 \
    --zone=us-west1-a --image-version=1.3 \
    --properties 'hadoop-env:HADOOP_CLASSPATH=${HADOOP_CLASSPATH}:/etc/tez/conf:/usr/lib/tez/*:/usr/lib/tez/lib/*'
  1. Uploaded job jar: /apps/wordcount.jar and library /apps/lib/commons-collections-3.2.2.jar
  2. Then submit a job with jar in classpath:
gcloud dataproc jobs submit hadoop --cluster=cascade \
    --jar=gs:/apps/wordcount.jar \
    --jars=gs://apps/lib/commons-collections-3.2.2.jar --bucket=xmitya-test \
    -- gs:/input/url+page.200.txt gs:/output/wc.out local

Then I'm getting forbidden error accessing the library file:

java.io.IOException: Error accessing: bucket: apps, object: lib/commons-collections-3.2.2.jar
    at com.google.cloud.hadoop.repackaged.gcs.com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl.wrapException(GoogleCloudStorageImpl.java:1957)
    at com.google.cloud.hadoop.repackaged.gcs.com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl.getObject(GoogleCloudStorageImpl.java:1983)
    at com.google.cloud.hadoop.repackaged.gcs.com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl.getItemInfo(GoogleCloudStorageImpl.java:1870)
    at com.google.cloud.hadoop.repackaged.gcs.com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem.getFileInfo(GoogleCloudStorageFileSystem.java:1156)
    at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.getFileStatus(GoogleHadoopFileSystemBase.java:1058)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:363)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:314)
    at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:2375)
    at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:2344)
    at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.copyToLocalFile(GoogleHadoopFileSystemBase.java:1793)
    at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:2320)
    at com.google.cloud.hadoop.services.agent.util.HadoopUtil.download(HadoopUtil.java:70)
    at com.google.cloud.hadoop.services.agent.job.AbstractJobHandler.downloadResources(AbstractJobHandler.java:448)
    at com.google.cloud.hadoop.services.agent.job.AbstractJobHandler$StartDriver.call(AbstractJobHandler.java:579)
    at com.google.cloud.hadoop.services.agent.job.AbstractJobHandler$StartDriver.call(AbstractJobHandler.java:568)
    at com.google.cloud.hadoop.services.repackaged.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
    at com.google.cloud.hadoop.services.repackaged.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
    at com.google.cloud.hadoop.services.repackaged.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
{
  "code" : 403,
  "errors" : [ {
    "domain" : "global",
    "message" : "[email protected] does not have storage.objects.get access to apps/lib/commons-collections-3.2.2.jar.",
    "reason" : "forbidden"
  } ],
  "message" : "[email protected] does not have storage.objects.get access to apps/lib/commons-collections-3.2.2.jar."
}
    at com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
    at com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
    at com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
    at com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:401)
    at com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1097)
    at com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:499)
    at com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
    at com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:549)
    at com.google.cloud.hadoop.repackaged.gcs.com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl.getObject(GoogleCloudStorageImpl.java:1978)
    ... 23 more

Tried set read permission from browser to [email protected] user and set public permissions to all files: gsutil defacl ch -u AllUsers:R gs://xmitya-test and gsutil acl ch -d allUsers:R gs://xmitya-test/** - no effect.

What could be the reason? Thanks!

Upvotes: 0

Views: 4044

Answers (1)

Igor Dvorzhak
Igor Dvorzhak

Reputation: 4457

It's complaining about access to apps, input and output buckets, that you specified in parameters of job submission command:

gcloud dataproc jobs submit hadoop --cluster=cascade --jar=gs:/apps/wordcount.jar --jars=gs://apps/lib/commons-collections-3.2.2.jar --bucket=xmitya-test gs:/input/url+page.200.txt gs:/output/wc.out local

To fix this issue you need to grant access to these buckets or if these are folders inside xmitya-test bucket then you need to specify it explicitly in the path: gs://xmitya-test/apps/wordcount.jar.

Upvotes: 1

Related Questions