CODEWITHSUNDEEP
google-cloud-storage
Evgeny Vorobyev
Evgeny Vorobyev

Reputation: 111

Proxy setting in gsutil tool

I use gsutil tool for download archives from Google Storage. I use next CMD command:

python c:\gsutil\gsutil cp gs://pubsite_prod_rev_XXXXXXXXXXXXX/YYYYY/*.zip C:\Tmp\gs

Everything works fine, but if I try to run that command from corporate proxy, I receive error:

Caught socket error, retrying: [Errno 10051] A socket operation was attempted to an unreachable network

I tried several times to set the proxy settings in .boto file, but all to no avail. Someone faced with such a problem? Thanks!

Upvotes: 1

Views: 7347

Answers (3)

alnedey
alnedey

Reputation: 11

A change was just merged into github yesterday that fixes some of the proxy support. Please try it out, or specifically, overwrite this file with your current copy:

https://github.com/GoogleCloudPlatform/gsutil/blob/master/gslib/util.py

Upvotes: 1

alnedey
alnedey

Reputation: 11

I believe I am having the same problem with the proxy settings being ignored under Linux (Ubuntu 12.04.4 LTS) and gsutils 4.2 (downloaded today).

I've been watching tcpdump on the host to confirm that gsutils is attempting to directly route to Google IPs instead of to my proxy server.

It seems that on the first execution of a simple command like "gsutil -d ls" it will use my proxy settings specified .boto for the first POST and then switch back to attempting to route directly to Google instead of my proxy server.

Then if I CTRL-C and re-run the exact same command, the proxy setting is no longer used at all. This difference in behaviour baffles me. If I wait long enough, I think it will work for the initial request again so this suggests some form on caching taking place. I'm not 100% of this behaviour yet because I haven't been able to predict when it occurs.

I also noticed that it always first tries to connect to 169.254.169.254 on port 80 regardless of proxy settings. A grep shows that it's hardcoded into oauth2_client.py, test_utils.py, layer1.py, and utils.py (under different subdirectories of the gsutils root).

I've tried setting the http_proxy environment variable but it appears that there is code that unsets this.

Upvotes: 0

Mike Schwartz
Mike Schwartz

Reputation: 12155

Please see the section "I'm connecting through a proxy server, what do I need to do?" at https://developers.google.com/storage/docs/faq#troubleshooting

Basically, you need to configure the proxy settings in your .boto file, and you need to ensure that your proxy allows traffic to accounts.google.com as well as to *.storage.googleapis.com.

Upvotes: 2

Related Questions