Can't connect to spark web UI inside open shift container

Question

I'm running my spark application in open shift container. The application runs for almost 2-4 hours. I do get the message of sparkUI started at http://hostname:4040. But when I click on it ,I'm getting webpage not found even though the application is still running .

Answer 1

You should try with port forwarding: https://docs.openshift.com/enterprise/3.0/dev_guide/port_forwarding.html

oc port-forward -p mypod 4040:4040

then you can use the same URL, localhost or whatever is the hostname.

Answer 2

Maybe this will help.

While running your application, you can specify the port you want the web ui to run on.

For example,

spark2-shell --conf spark.ui.port=4040