CODEWITHSUNDEEP
apache-sparkhadoop-yarngoogle-cloud-dataproc
J.Fratzke
J.Fratzke

Reputation: 1465

Proxying Resource Manager in Google Dataproc

I've followed Google instructions on this.

gcloud compute ssh  --zone=us-central1-b --ssh-flag="-D 8088" --ssh-flag="-N" --ssh-flag="-n" spark-test-m

followed by

/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --proxy-server="socks5://localhost:1080" --host-resolver-rules="MAP * 0.0.0.0 , EXCLUDE localhost" --user-data-dir=/tmp/

Navigate to https://spark-test-m:8088 and I get nothing.

Upvotes: 0

Views: 275

Answers (2)

Angus Davis
Angus Davis

Reputation: 2683

The first SSH flag should be "-D 1080" per http://cloud.google.com/dataproc/cluster-web-interfaces

Upvotes: 1

Dennis Huo
Dennis Huo

Reputation: 10707

There appear to be two problems:

  1. Since you passed --ssh-flag="-D 8088", your local proxy port is actually 8088 instead of 1080, where you specified --proxy-server="socks5://localhost:1080" in your flags to Chrome.

  2. It looks like you're trying to use an HTTPS url instead of HTTP.

Alternatively, sometimes your browser may have been configured to ignore the user-profile proxy settings in favor of some system ones. I'd first try to fix the proxy port, and then try both http://spark-test-m:8088 and also maybe http://spark-test-m:50070 to see if it's specific to the proxy connection or if maybe for some reason just the ResourceManager failed to come up.

Upvotes: 0

Related Questions