UnreachableBrowserException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure

Question

While executing my suite dynamically on grid using docker-compose, I am getting the following exception stacktrace:

org.openqa.selenium.remote.UnreachableBrowserException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
03:52:47 Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
03:52:47 System info: host: 'ip-10-33-0-63', ip: '10.33.0.63', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-1069-aws', java.version: '1.8.0_151'
03:52:47 Driver info: driver.version: RemoteWebDriver
03:52:47    at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:573)
03:52:47    at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:213)
03:52:47    at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:131)
03:52:47    at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:144)
03:52:47    at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
03:52:47    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
03:52:47    at java.lang.reflect.Method.invoke(Method.java:498)
03:52:47    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:131)
03:52:47    at org.testng.internal.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:61)
03:52:47    at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:511)
03:52:47    at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:241)
03:52:47    at org.testng.internal.Invoker.invokeMethod(Invoker.java:581)
03:52:47    at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:792)
03:52:47    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1103)
03:52:47    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:140)
03:52:47    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:122)
03:52:47    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
03:52:47    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
03:52:47    at java.lang.Thread.run(Thread.java:748)
03:52:47 Caused by: java.net.ConnectException: Failed to connect to /10.33.0.220:4444
03:52:47    at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:247)
03:52:47    at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:165)
03:52:47    at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:257)
03:52:47    at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)
03:52:47    at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)
03:52:47    at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
03:52:47    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
03:52:47    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
03:52:47    at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
03:52:47    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
03:52:47    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
03:52:47    at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
03:52:47    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
03:52:47    at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126)
03:52:47    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
03:52:47    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
03:52:47    at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
03:52:47    at okhttp3.RealCall.execute(RealCall.java:77)
03:52:47    at org.openqa.selenium.remote.internal.OkHttpClient.execute(OkHttpClient.java:103)
03:52:47    at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:105)
03:52:47    at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:74)
03:52:47    at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:136)
03:52:47    at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:552)
03:52:47 Caused by: java.net.ConnectException: Connection refused (Connection refused)
03:52:47    at java.net.PlainSocketImpl.socketConnect(Native Method)
03:52:47    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
03:52:47    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
03:52:47    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
03:52:47    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
03:52:47    at java.net.Socket.connect(Socket.java:589)
03:52:47    at okhttp3.internal.platform.Platform.connectSocket(Platform.java:129)
03:52:47    at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:245)

This is occurring while executing:

return (new RemoteWebDriver(new URL(url), new FirefoxOptions()));

This is also happening for chrome. I recently upgraded the selenium to the latest version, i.e 3.141.59.

The docker-compose.yml is:

version: "3"
services:
  selenium-hub:
    image: selenium/hub:3.141.59-bismuth
    container_name: selenium-hub
    ports:
      - "4444:4444"
    environment:
      - GRID_BROWSER_TIMEOUT=30

  chrome:
    image: selenium/node-chrome:3.141.59-bismuth
    depends_on:
      - selenium-hub
    environment:
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444

  firefox:
    image: selenium/node-firefox:3.141.59-bismuth
    depends_on:
      - selenium-hub
    environment:
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444
    volumes:
      - "/dev/shm:/dev/shm"

This is happening both in Jenkins as well as when I execute it in my system from cmd. I am not sure if the issue is related to selenium or networks. If there are other links to a solution, please share here. Thanks for any help in advance.

UPDATE EDIT:

Startup:

1) Build project (using maven-assembly-plugin) by mvn clean install -DskipTests=true

2) Start Grid: sudo docker-compose up -d --scale chrome=5 --scale firefox=3

3) Run executable jar.

4) Remove Grid: sudo docker-compose down

Update:

Adding Hub Config:

Config for the hub :
browserTimeout : 30
debug : false
jettyMaxThreads : -1
host : 192.168.80.2
port : 4444
role : hub
timeout : 1800
cleanUpCycle : 5000
capabilityMatcher : org.openqa.grid.internal.utils.DefaultCapabilityMatcher
newSessionWaitTimeout : -1
throwOnCapabilityNotPresent : true
registry : org.openqa.grid.internal.DefaultGridRegistry

The final configuration comes from:
the default :
browserTimeout : 0
debug : false
host : 0.0.0.0
port : 4444
role : hub
timeout : 1800
cleanUpCycle : 5000
capabilityMatcher : org.openqa.grid.internal.utils.DefaultCapabilityMatcher
newSessionWaitTimeout : -1
throwOnCapabilityNotPresent : true
registry : org.openqa.grid.internal.DefaultGridRegistry
updated with command line options:
-role hub -hubConfig /opt/selenium/config.json
and configuration loaded from /opt/selenium/config.json:
{
"host": "0.0.0.0",
"port": 4444,
"role": "hub",
"maxSession": 5,
"newSessionWaitTimeout": -1,
"capabilityMatcher": "org.openqa.grid.internal.utils.DefaultCapabilityMatcher",
"throwOnCapabilityNotPresent": true,
"jettyMaxThreads": -1,
"cleanUpCycle": 5000,
"browserTimeout": 30,
"timeout": 1800,
"debug": false
}

Adding node config:

browserTimeout: 30
debug: false
jettyMaxThreads: -1
host: 192.168.80.6
port: 5555
role: node
timeout: 1800
cleanUpCycle: 5000
maxSession: 1
capabilities: Capabilities {applicationName: , browserName: chrome, maxInstances: 1, platform: LINUX, platformName: LINUX, seleniumProtocol: WebDriver, server:CONFIG_UUID: d949d886-61f4-4a3a-93f9-c2a..., version: 70.0.3538.110}
downPollingLimit: 2
hub: http://selenium-hub:4444/grid/register
id: http://192.168.80.6:5555
nodePolling: 5000
nodeStatusCheckTimeout: 5000
proxy: org.openqa.grid.selenium.proxy.DefaultRemoteProxy
register: true
registerCycle: 5000
remoteHost: http://192.168.80.6:5555
unregisterIfStillDownAfter: 60000

Answer 1

I solved the same issue by following this article's last comment. Docker Container cannot reach localhost port 4444. Why though?

In your docker-compose.yaml file under services where your hub is what is the hub called. Use that name to formulate the url for remotewebdriver.

Just changed 'localhost:4444/wd/hub' to 'seleniumhub:4444/wd/hub'

Answer 2

This error message...

org.openqa.selenium.remote.UnreachableBrowserException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
03:52:47 Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
03:52:47 System info: host: 'ip-10-33-0-63', ip: '10.33.0.63', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-1069-aws', java.version: '1.8.0_151'
03:52:47 Driver info: driver.version: RemoteWebDriver

...implies that the WebDriver was unable to initiate/spawn a new RemoteWebDriver session.

I don't see any significant issues in your configuration or in the line of code. However your main issue seems to be the incompatibility between the version of the binaries you are using as follows:

• Your JDK version is 1.8.0_151 which is pretty ancient.

Solution

• Upgrade JDK to recent levels JDK 8u191.
• Take a System Reboot.
• Execute your Test.