GSUtil service account failure

Question

I'm running debian on GCE and I've configured gsutil with gsutil config -e.

When I try to do gsutil -D ls I get the following error:

gsutil version 4.3 checksum f6a4b1f8400e2d1d145c9ec5e9ea8d90 (OK) boto version 2.29.1 python version 2.7.3 (default, Mar 13 2014, 11:03:55) [GCC 4.7.2] config path: /etc/boto.cfg gsutil path: /usr/local/share/google/gsutil/gsutil compiled crcmod: False installed via package manager: False editable install: False Command being run: /usr/local/bin/gsutil -D ls config_file_list: ['/etc/boto.cfg', '/var/lib/postgresql/.boto'] config: [('debug', '0'), ('working_dir', '/mnt/pyami'), ('https_validate_certificates', 'True'), ('debug', '0'), ('working_dir', '/mnt/pyami'), ('default_project_id', 'e-vard'), ('default_api_version', '2'), ('content_language', 'en')] Calling method storage.buckets.list with StorageBucketsListRequest: <StorageBucketsListRequest maxResults: 100 project: 'e-vard' projection: ProjectionValueValuesEnum(full, 0)> Making http GET to https://www.googleapis.com/storage/v1/b?projection=full&prettyPrint=True&fields=nextPageToken%2Citems%2Fid&maxResults=100&project=e-vard&alt=json Headers: {'accept': 'application/json', 'accept-encoding': 'gzip, deflate', 'content-length': '0', 'user-agent': 'apitools gsutil/4.3 (linux2)'} Body: (none) Attempting refresh to obtain initial access_token {'iss': '1234567890@developer.gserviceaccount.com', 'scope': 'https://www.googleapis.com/auth/devstorage.full_control', 'aud': 'https://accounts.google.com/o/oauth2/token', 'exp': 1405436827L, 'iat': 1405433227L} Refreshing access_token connect: (accounts.google.com, 443) send: 'POST /o/oauth2/token HTTP/1.1\r\nHost: accounts.google.com\r\nContent-Length: 726\r\ncontent-type: application/x-www-form-urlencoded\r\naccept-encoding: gzip, deflate\r\nuser-agent: Python-httplib2/0.7.7 (gzip)\r\n\r\ngrant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=ASSERTIONREMOVED' reply: 'HTTP/1.1 400 Bad Request\r\n' header: Content-Type: application/json header: Cache-Control: no-cache, no-store, max-age=0, must-revalidate header: Pragma: no-cache header: Expires: Fri, 01 Jan 1990 00:00:00 GMT header: Date: Tue, 15 Jul 2014 14:07:07 GMT header: Content-Encoding: gzip header: X-Content-Type-Options: nosniff header: X-Frame-Options: SAMEORIGIN header: X-XSS-Protection: 1; mode=block header: Server: GSE header: Alternate-Protocol: 443:quic header: Transfer-Encoding: chunked Failed to retrieve access token: { "error" : "invalid_grant" } DEBUG: Exception stack trace: Traceback (most recent call last): File "/usr/local/share/google/gsutil/gslib/__main__.py", line 419, in _RunNamedCommandAndHandleExceptions debug_level, parallel_operations) File "/usr/local/share/google/gsutil/gslib/command_runner.py", line 194, in RunNamedCommand return_code = command_inst.RunCommand() File "/usr/local/share/google/gsutil/gslib/commands/ls.py", line 378, in RunCommand bucket_fields=bucket_fields): File "/usr/local/share/google/gsutil/gslib/wildcard_iterator.py", line 425, in IterBuckets for blr in self._ExpandBucketWildcards(bucket_fields=bucket_fields): File "/usr/local/share/google/gsutil/gslib/wildcard_iterator.py", line 339, in _ExpandBucketWildcards provider=self.wildcard_url.scheme): File "/usr/local/share/google/gsutil/gslib/gcs_json_api.py", line 380, in ListBuckets global_params=global_params) File "/usr/local/share/google/gsutil/gslib/third_party/storage_apitools/storage_v1_client.py", line 351, in List config, request, global_params=global_params) File "/usr/local/share/google/gsutil/gslib/third_party/storage_apitools/base_api.py", line 587, in _RunMethod http, http_request, retries=self.__client.num_retries) File "/usr/local/share/google/gsutil/gslib/third_party/storage_apitools/http_wrapper.py", line 152, in MakeRequest redirections=redirections, connection_type=connection_type) File "/usr/local/share/google/gsutil/third_party/google-api-python-client/oauth2client/util.py", line 132, in positional_wrapper return wrapped(*args, **kwargs) File "/usr/local/share/google/gsutil/third_party/google-api-python-client/oauth2client/client.py", line 475, in new_request self._refresh(request_orig) File "/usr/local/share/google/gsutil/third_party/google-api-python-client/oauth2client/client.py", line 663, in _refresh self._do_refresh_request(http_request) File "/usr/local/share/google/gsutil/third_party/google-api-python-client/oauth2client/client.py", line 710, in _do_refresh_request raise AccessTokenRefreshError(error_msg) AccessTokenRefreshError: invalid_grant

What have I done wrong?

Answer 1

To use a service account with a GCE internal VM, you should check the Enable Compute Engine service account in the UI and give Storage a scope of your choosing. This will expose GCE service credentials in your instance. However, you can't run it with tools that operate over HTTP because you have no external IP to communicate from.

That means if you want to use Google Cloud Storage with a GCE VM, you need an external IP.

Answer 2

Because gsutil makes HTTP calls to interact with Google Cloud Storage, it can't operate on a GCE instance with no external IP.