Cannot bind docker chrome node in seleniumgrid to proxy container

Question

Problem

I am launching seleniumgrid with mobbrowserproxy in a docker bridged network. The remote driver proxy settings are configured correctly but i cannot connect to the proxy server from the chrome driver. I have looked at all the issues and i cannot find a solution.

Below is the error which shows the driver desired capabilities/options.

Stacktrace

selenium-hub    | 15:40:24.222 INFO [TestSlot.getNewSession] - Trying to create a new session on test slot {server:CONFIG_UUID=ef95a1c8-8c30-472b-9e5e-323b8a8c9045, seleniumProtocol=WebDriver, browserName=chrome, maxInstances=1, platformName=LINUX, version=74.0.3729.169, applicationName=, platform=LINUX}
chrome_1        | 15:40:24.243 INFO [ActiveSessionFactory.apply] - Capabilities are: {
chrome_1        |   "browserName": "chrome",
chrome_1        |   "goog:chromeOptions": {
chrome_1        |     "args": [
chrome_1        |       "--disable-gpu",
chrome_1        |       "--headless",
chrome_1        |       "--no-sandbox",
chrome_1        |       "--whitelisted-ips",
chrome_1        |       "--disable-dev-shm-usage",
chrome_1        |       "--allow-insecure-localhost",
chrome_1        |       "--disable-web-security",
chrome_1        |       "--ignore-certificate-errors",
chrome_1        |       "--allow-running-insecure-content"
chrome_1        |     ],
chrome_1        |     "extensions": [
chrome_1        |     ]
chrome_1        |   },
chrome_1        |   "proxy": {
chrome_1        |     "httpProxy": "0.0.0.0:9099",
chrome_1        |     "proxyType": "manual",
chrome_1        |     "sslProxy": "0.0.0.0:9099"
chrome_1        |   },
chrome_1        |   "version": ""
chrome_1        | }
chrome_1        | 15:40:24.243 INFO [ActiveSessionFactory.lambda$apply$11] - Matched factory org.openqa.selenium.grid.session.remote.ServicedSession$Factory (provider: org.openqa.selenium.chrome.ChromeDriverService)
chrome_1        | Starting ChromeDriver 74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729@{#29}) on port 28080
chrome_1        | Only local connections are allowed.
chrome_1        | Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
chrome_1        | [1589038824.252][SEVERE]: bind() failed: Cannot assign requested address (99)
chrome_1        | 15:40:24.418 INFO [ProtocolHandshake.createSession] - Detected dialect: OSS
chrome_1        | 15:40:24.444 INFO [RemoteSession$Factory.lambda$performHandshake$0] - Started new session 4c42082804845cf9571e6a843a63feca (org.openqa.selenium.chrome.ChromeDriverService)
chrome_1        | 15:40:34.798 INFO [ActiveSessions$1.onStop] - Removing session 4c42082804845cf9571e6a843a63feca (org.openqa.selenium.chrome.ChromeDriverService)

I believe the key point of this stacktrace is right here

chrome_1        | Only local connections are allowed.
chrome_1        | Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
chrome_1        | [1589038824.252][SEVERE]: bind() failed: Cannot assign requested address (99)

To Reproduce

To reproduce this error. If you simple launch the containers and login to chromenode and use the chromedriver cli, you should see the same error in the terminal. Again the target of this issue is connecting the proxy container via chromedriver proxy settings.

version: "3"

services:
  selenium-hub:
    image: selenium/hub:3.141.59-20200409
    container_name: selenium-hub
    ports:
      - "4444:4444"
    networks: 
      - caowebtests

  chrome:
    image: selenium/node-chrome:3.141.59-oxygen
    depends_on:
      - selenium-hub
    environment:
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444
    networks: 
      - caowebtests
    expose: 
      - 9515

  firefox:
    image: selenium/node-firefox:3.141.59-20200409
    depends_on:
      - selenium-hub
    environment:
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444
    networks: 
      - caowebtests

  opera:
    image: selenium/node-opera:3.141.59-20200409
    depends_on:
      - selenium-hub
    environment:
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444
    networks: 
      - caowebtests

  proxy:
    image: spothero/browsermob-proxy:1.0.0
    depends_on:
      - selenium-hub
    networks: 
      - caowebtests
    ports:
      - "9090:9090"
    links:
      - selenium-hub
      - firefox
      - chrome
      - opera

  robottests:
    container_name: robottests
    command: /bin/sleep infinity
    depends_on:
      - selenium-hub
    build: .
    volumes:
      - ./reports:/cao_ui_tests/reports
    networks: 
      - caowebtests


networks:
  caowebtests:
    driver: bridge

As you can see in the trace above. the desired capabilities and options are all there. What I am trying to find out is why the chrome driver is getting this error with the proxy settings above.

My test is failing when looking for the first element on the login page. I am not sure if you need the test scripts and the ability to execute the test scripts to try to reproduce this or if this is something that can be pointed out. I can add this repo to github if needed.

Test Script Below

*** Settings ***
Library    BrowserMobProxy
Library    SeleniumLibrary
Library    RequestsLibrary
Library    Collections

Resource   resources${/}base.robot
Resource   resources${/}common.robot
Resource   resources${/}auth.robot

Suite Setup  Setup Test Suite
Test Teardown  Close All Browsers
Suite Teardown  Test Suite Teardown


*** Variables ***
${BMP_HOST}  0.0.0.0
${BMP_PORT}  9090
${SELENIUM}  http://0.0.0.0:4444/wd/hub
${SHOT_NUM}  0
@{TIMINGS}


*** Test Cases ***
Login User
  Wait Until Page Loads
  Wait Until Page Contains Element  ${UI['login']}  timeout=10
  Submit Credentials  %{TEST_USER}  %{TEST_PASS}






*** Keywords ***
Setup Test Suite
  Load UI Repository  ${REPO_PATH}
  Connect To Remote Server  ${BMP_HOST}  ${BMP_PORT}
  Set Selenium Implicit Wait  0.2 seconds
  Set Selenium Timeout  30 seconds
  ${prefs}=  Evaluate  sys.modules['selenium.webdriver'].ChromeOptions()  sys, selenium.webdriver
  &{caps}=  Set Capabilities
  Call Method  ${prefs}  add_argument  --disable-gpu 
  Call Method  ${prefs}  add_argument  --headless 
  Call Method  ${prefs}  add_argument  --no-sandbox
  Call Method  ${prefs}  add_argument  --whitelisted-ips
  Call Method  ${prefs}  add_argument  --disable-dev-shm-usage
  Call Method  ${prefs}  add_argument  --allow-insecure-localhost
  Call Method  ${prefs}  add_argument  --disable-web-security
  Call Method  ${prefs}  add_argument  --ignore-certificate-errors
  Call Method  ${prefs}  add_argument  --allow-running-insecure-content
  Create Webdriver  Remote  command_executor=${SELENIUM}  desired_capabilities=${caps}  
 options=${prefs}
  New Har  LoginPage
  Go To  https://cardatonce.eftsource.com


Test Suite Teardown
  Get Har  file.har
  Close Proxy


Set Capabilities
  [Documentation]  Set the options for the selenium Driver
  ${port}=  Create Proxy

  &{proxy}=  Create Dictionary
  ...  proxyType  MANUAL
  ...  sslProxy  ${BMP_HOST}:${port}
  ...  httpProxy  ${BMP_HOST}:${port}

  &{caps}=  Create Dictionary  browserName=chrome  platform=ANY  proxy=&{proxy}
  Log  Selenium capabilities: ${caps}
  [return]  ${caps}

Create Proxy
  [Documentation]  Get a BMP port for our test
  Create Session  bmp  http://${BMP_HOST}:${BMP_PORT}
  ${resp}=  Get Request  bmp  /proxy
  Should Be Equal As Strings  ${resp.status_code}  200
  Log  BMP Sessions: ${resp.text} [${resp.status_code}]
  &{headers}=  Create Dictionary  Content-Type=application/x-www-form-urlencoded
  &{data}=  Create Dictionary  trustAllServers=True
  ${resp}=  Post Request  bmp  /proxy  data=${data}  headers=${headers}
  Should Be Equal As Strings  ${resp.status_code}  200
  Log  ${resp.text} [${resp.status_code}]
  ${port}=  Get From Dictionary  ${resp.json()}  port
  Log  New BMP port: ${port} [${resp.status_code}]
  Set Global Variable  ${port}
  [return]  ${port}


Close Proxy
  ${resp}=  Delete Request  bmp  /proxy/${port}
  Should Be Equal As Strings  ${resp.status_code}  200
  Log  Closed proxy at ${port} [${resp.status_code}]


New Har
  [Documentation]  Name and initialize a Har
  [arguments]  ${pagename}
  &{data}=  Create Dictionary  initialPageRef=${pagename}
  ${resp}=  Put Request  bmp  /proxy/${port}/har  params=${data}
  #Should Be Equal As Strings  ${resp.status_code}  204
  Log  New Har (${pagename}) [${resp.status_code}]

Environment

This is on a MacOs Mojave.

Suggestions

I think the most important part of this script is the variables connecting to the hub and the mob browser proxy containers

${BMP_HOST}  0.0.0.0
${BMP_PORT}  9090
${SELENIUM}  http://0.0.0.0:4444/wd/hub

This script just shows you how I am setting the preferences that you already see here in the stacktrace on the chrome node.

chrome_1        | Only local connections are allowed.
chrome_1        | Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
chrome_1        | [1589038824.252][SEVERE]: bind() failed: Cannot assign requested address (99)

References

These are the issues i have already found online One Two - at the bottom of this issue it was resolved with adding, and as you can see above. I have already tried this.

--disable-dev-shm-usage'

Three this guy solved this problem by adding ipv6 to his docker config. But i am using a bridged network in docker compose so i dont see why this option would matter....

Previous attempts

I am not sure if this is an issue with chromedriver or docker. To go through a couple of things i have already tried... 1. I tried logging in to the chrome container and using the chromedriver cli...I also get the same error comment on issue 2 I did try to use the image that included chromedriver 74 as the next comment suggested next comment on issue 2