CODEWITHSUNDEEP
command-linegoogle-cloud-endpointsgoogle-cloud-platformcommand-line-toolgsutil
jazz
jazz

Reputation: 5519

All of my gsutil commands timeout

so I've successfully installed the newest google-cloud-sdk on my mac so all my gcloud and gsutil command line tools are up to date.

However, whenever I try a gsutil command, it times out. For example, when I run:

gsutil mb gs://cloud-storage-analysis

it starts to run, printing:

Creating gs://cloud-storage-analysis/...

But then it never stops. I stop it using Control-C and it prints out this.

Caught signal 2 - exiting
Traceback (most recent call last):
  File "/Users/jazz/google-cloud-sdk/bin/bootstrapping/gsutil.py", line 71, in <module>
    main()
  File "/Users/jazz/google-cloud-sdk/bin/bootstrapping/gsutil.py", line 54, in main
    'platform/gsutil', 'gsutil', *args)
  File "/Users/jazz/google-cloud-sdk/bin/bootstrapping/bootstrapping.py", line 45, in ExecutePythonTool
    execution_utils.ArgsForPythonTool(_FullPath(tool_dir, exec_name), *args))
  File "/Users/jazz/google-cloud-sdk/bin/bootstrapping/bootstrapping.py", line 86, in _ExecuteTool
    execution_utils.Exec(args + sys.argv[1:], env=_GetToolEnv())
  File "/Users/jazz/google-cloud-sdk/bin/bootstrapping/../../lib/googlecloudsdk/core/util/execution_utils.py", line 146, in Exec
    ret_val = p.wait()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1357, in wait
    pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 478, in _eintr_retry_call
    return func(*args)
KeyboardInterrupt

The one time I let it run for a long time, it actually timed out and stopped. This happens for every gsutil command I try. My bq commands work (but they're really slow).

I don't know what's wrong.

Thanks for any help.

Edit:

So my problem hasn't entirely gone away, but gsutil does sometimes work. It is very intermittent and seemingly random when it works and when it doesn't. It seems like refreshing the shell and/or quitting and reopening Terminal helps, but not every time. I'd still like to get to the bottom of it.

So as Misha suggested, I ran gsutil -D ls as a test. It got to here and then stopped for a while (maybe 2-3 minutes): (some info has been [removed])

gsutil version: 4.13
checksum: [key] (OK)
boto version: 2.38.0
python version: 2.7.5 (default, Mar  9 2014, 22:15:05) [GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)]
OS: Darwin 13.4.0
multiprocessing available: True
using cloud sdk: True
config path: [path-to-home/.boto
gsutil path: [path-to-home]/google-cloud-sdk/platform/gsutil/gsutil
compiled crcmod: True
installed via package manager: False
editable install: False
Command being run: [path-to-home]/google-cloud-sdk/platform/gsutil/gsutil -o GSUtil:default_project_id=storagelogstest -D ls
config_file_list: ['[path-to-home]/.config/gcloud/legacy_credentials/[email]@gmail.com/.boto', '/.boto']
config: [('debug', '0'), ('working_dir', '/mnt/pyami'), ('https_validate_certificates', 'True'), ('debug', '0'), ('working_dir', '/mnt/pyami'), ('content_language', 'en'), ('default_api_version', '2'), ('default_project_id', 'storagelogstest')]
DEBUG 0626 10:43:29.972406 oauth2_client.py] GetAccessToken: checking cache for key 9886bbc6e7e67cf6d2b8b3a707046f2a326dcceb
DEBUG 0626 10:43:29.972712 oauth2_client.py] FileSystemTokenCache.GetToken: key=9886bbc6e7e67cf6d2b8b3a707046f2a326dcceb not present (cache_file=/var/folders/25/zs7lm5jd7dg5jljd4qdxjpnc0000gq/T/oauth2_client-tokencache.503.9886bbc6e7e67cf6d2b8b3a707046f2a326dcceb)
DEBUG 0626 10:43:29.972882 oauth2_client.py] GetAccessToken: token from cache: None
DEBUG 0626 10:43:29.973030 oauth2_client.py] GetAccessToken: fetching fresh access token...
INFO 0626 10:43:29.973551 client.py] Refreshing access_token

Then, it outputted this:

connect fail: (accounts.google.com, 443)
connect: (accounts.google.com, 443)
send: 'POST /o/oauth2/token HTTP/1.1\r\nHost: accounts.google.com\r\nContent-Length: 195\r\ncontent-type: application/x-www-form-urlencoded\r\naccept-encoding: gzip, deflate\r\nuser-agent: Python-httplib2/0.7.7 (gzip)\r\n\r\nclient_secret=ZmssLNjJy2998hD4CTg2ejr2&grant_type=refresh_token&refresh_token=1%2FUl4EXn6N5jPCjFVy6-U5HwIKNApkGmYEEQPZO654NxHBactUREZofsF9C7PrpE-j&client_id=32555940559.apps.googleusercontent.com'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: application/json; charset=utf-8
header: Cache-Control: no-cache, no-store, max-age=0, must-revalidate
header: Pragma: no-cache
header: Expires: Fri, 01 Jan 1990 00:00:00 GMT
header: Date: Fri, 26 Jun 2015 17:44:45 GMT
header: Content-Disposition: attachment; filename="json.txt"; filename*=UTF-8''json.txt
header: Content-Encoding: gzip
header: P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
header: X-Content-Type-Options: nosniff
header: X-Frame-Options: SAMEORIGIN
header: X-XSS-Protection: 1; mode=block
header: Server: GSE
header: Set-Cookie: NID=68=PfDga1cpnMr8ho-0tlBrWNhgLzQsThRzV31vn8QD1cV45H8C4-ydGoMI0ITI0lPJHvKhN_uPSisTQwIzM2LEFKqjXZlgsJ-9l0HiflLdl1UGMevAQ2GFxFqa369vQxZG;Domain=.google.com;Path=/;Expires=Sat, 26-Dec-2015 17:44:45 GMT;HttpOnly
header: Alternate-Protocol: 443:quic,p=1
header: Transfer-Encoding: chunked
DEBUG 0626 10:44:45.808030 oauth2_client.py] GetAccessToken: fresh access token: AccessToken(token=ya29.ngEY7SR5AbdDZ2pWMtBJzAnJGYGUgXB6hKcAwE8I274ieyLmEpuD1WypFJ8jAZN9LS5zCZ3ldGL4MA, expiry=2015-06-26 18:44:45.806928Z)
DEBUG 0626 10:44:45.808321 oauth2_client.py] FileSystemTokenCache.PutToken: key=9886bbc6e7e67cf6d2b8b3a707046f2a326dcceb, cache_file=/var/folders/25/zs7lm5jd7dg5jljd4qdxjpnc0000gq/T/oauth2_client-tokencache.503.9886bbc6e7e67cf6d2b8b3a707046f2a326dcceb
INFO 0626 10:44:45.813867 base_api.py] Calling method storage.buckets.list with StorageBucketsListRequest: <StorageBucketsListRequest
 maxResults: 1000
 project: 'storagelogstest'
 projection: ProjectionValueValuesEnum(full, 0)>
INFO 0626 10:44:45.814872 base_api.py] Making http GET to https://www.googleapis.com/storage/v1/b?project=storagelogstest&fields=nextPageToken%2Citems%2Fid&alt=json&projection=full&maxResults=1000
INFO 0626 10:44:45.815298 base_api.py] Headers: {'accept': 'application/json',
 'accept-encoding': 'gzip, deflate',
 'content-length': '0',
 'user-agent': 'apitools gsutil/4.13 (darwin) Cloud SDK Command Line Tool 0.9.66'}
INFO 0626 10:44:45.815390 base_api.py] Body: (none)
DEBUG 0626 10:45:45.846443 http_wrapper.py] Caught socket error, retrying: timed out

It tried to reconnect a few times but it timed out each time and then I stopped it.

I have not tried this on another machine, but I have talked to a coworker who has the same problem (frequently it won't work but some times it does).

Upvotes: 2

Views: 3530

Answers (1)

jazz
jazz

Reputation: 5519

So, if you have this problem, most likely you'll be able to get gsutil to work if you basically just try it a lot. Try restarting the shell (exec -l $SHELL) and quitting/reopening the command line and keep trying, it eventually worked for me. This is not a permanent fix, it still times out about 2/3 the time for me. But you'll at least be able to run your commands.

Hopefully Google can address this problem

Upvotes: 1

Related Questions