CODEWITHSUNDEEP
google-cloud-platformvirtual-machinegoogle-compute-enginegsutil
Pushpender Sharma
Pushpender Sharma

Reputation: 1

How can we copy big folder from google cloud storage to compute engine (windows server 2019) hosted on google cloud platform using gsutil command?

I have saved BI tool setup files in a folder on google cloud storage . we have windows VM created on GCP where i want to move this folder containing all the setup files ( around 60 gb) from google cloud storage by using gsutil command but it is throwing error I am using below command gsutil cp -r gs://bucket-name/folder-name C:\Users\user-name\

getting error as AccessDeniedException: 403 sa-d-edw-ce-cognosserver@prj-edw-d-edw-7f58.iam.gserviceaccount.com does not have storage.objects.list access to the Google Cloud Storage bucket.

can someone please help me to understand where I am making mistake ?

Upvotes: 0

Views: 236

Answers (1)

John Hanley
John Hanley

Reputation: 81444

There are two likely problems:

  1. The CLI is using an identity that does not possess the required permissions.

  2. The Compute Engine instance has restricted the permissions via scopes or has disabled scopes preventing all API access.

To modify IAM permissions/roles requires permissions as well on your account. Otherwise, you will need to contact an administrator for the ORG or project.

The CLI gsutil is using an identity (either a user or service account). That identity does not have an IAM role attached that contains the IAM permission storage.objects.list.

There are a number of IAM roles that have that permission. If you only need to list and read Cloud Storage objects, use the role Storage Legacy Bucket Reader aka roles/storage.legacyBucketReader. The following link provides details on the available roles:

IAM roles for Cloud Storage

Your Google Compute Engine Windows VM instance has a service account attached to it. The Google Cloud CLI tools can use that service account or the credentials from gcloud auth login. There are a few more methods.

To complicate this a bit more, each Compute Engine has scopes assigned which limit a service accounts permissions. The default scopes allow Cloud Storage object read. In the Google Cloud Console GUI lookup or modify the assigned scopes. The following command will output details on the VM which will include the key serviceAccounts.scope.

gcloud compute instances describe INSTANCE_NAME --project PROJECT_ID --zone ZONE

Figure out which identity your VM is using

gcloud auth list

Add an IAM role to that identity

Windows command syntax.

For a service account:

gcloud projects add-iam-policy-binding PROJECT_ID ^
--member="serviceAccount:REPLACE_WITH_SERVICE_ACCOUNT_EMAIL_ADDRESS" ^
--role="roles/storage.legacyBucketReader"

For a user account:

gcloud projects add-iam-policy-binding PROJECT_ID ^
--member="user:REPLACE_WITH_USER_EMAIL_ADDRESS" ^
--role="roles/storage.legacyBucketReader"

Upvotes: 0

Related Questions