CODEWITHSUNDEEP
pythonseleniumselenium-webdriverhttpclientwebdriverwait
mrd2242
mrd2242

Reputation: 95

http.client.RemoteDisconnected: Remote end closed connection without response error using driver.quit() of Selenium Python

I'm using:

Here is my code:

from selenium import webdriver

driver = webdriver.Chrome("D:/webdriver/chromedriver.exe")
driver.get("https://github.com")
driver.quit()

When executing driver.quit(), the exception raise:

Traceback (most recent call last):
  File "C:/Users/taiping/Desktop/data_test/selenium_test.py", line 5, in <module>
    driver.quit()
  File "D:\python3.8\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 158, in quit
    self.service.stop()
  File "D:\python3.8\lib\site-packages\selenium\webdriver\common\service.py", line 151, in stop
    self.send_remote_shutdown_command()
  File "D:\python3.8\lib\site-packages\selenium\webdriver\common\service.py", line 127, in send_remote_shutdown_command
    url_request.urlopen("%s/shutdown" % self.service_url)
  File "D:\python3.8\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "D:\python3.8\lib\urllib\request.py", line 525, in open
    response = self._open(req, data)
  File "D:\python3.8\lib\urllib\request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "D:\python3.8\lib\urllib\request.py", line 502, in _call_chain
    result = func(*args)
  File "D:\python3.8\lib\urllib\request.py", line 1379, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "D:\python3.8\lib\urllib\request.py", line 1354, in do_open
    r = h.getresponse()
  File "D:\python3.8\lib\http\client.py", line 1332, in getresponse
    response.begin()
  File "D:\python3.8\lib\http\client.py", line 303, in begin
    version, status, reason = self._read_status()
  File "D:\python3.8\lib\http\client.py", line 272, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

But this code has no error on my Macbook. What's the problem?

Updated: 2020-08-05

I open the debugger and found that every HTTPConnection object have been set the system wide http proxy. But I did not set any options explicitly in my code. And the driver.quit method will send http://localhost:59717/shutdown to chrome to perform quit. So I guess the shutdown url is actually sent to the proxy server, not the local browser.

And I try to use fiddler to check the request informations. There is another problem that I can not decode the https requests because of some certificates configs. So I change the argument of driver.get() to an internal web url of my company. The result is : If I close fiddler, RemoteDisconnected error raise again. And if I open fiddler, all works.

What happened? I know fiddler set the proxy to 127.0.0.1:8888, so I think there could be something wrong with the proxy settings. But I can not fix it. I guess the chrome use the system proxy so the github home page could open correctly, but when send shutdown url to chrome, the request object should not use the system proxy, but it does.

Am I right? And how to fix this problem?

Upvotes: 2

Views: 9669

Answers (4)

Dustin taylor
Dustin taylor

Reputation: 1

Check to see if your os has been set to eufi and bios have been flashed to make you think that you are on your system but in all reality your proxy is run from cm and clone of your pc run in background transparent check bios version and see if they have been overwritten also check router switches and any possible server onnnetwork

Upvotes: 0

Chanil Byun
Chanil Byun

Reputation: 1

my OS is linux I resolve this issue add no_poroxy In my case add add no_proxy for localhost and intranet ips no_proxy="127.0.0.1, localhost, 10...* ...

Upvotes: 0

mrd2242
mrd2242

Reputation: 95

I have opened an issue to selenium repo

My solution is to send a request which doesn't has a proxy. Here is my code:

from selenium import webdriver
import http.client

driver = webdriver.Chrome("D:/webdriver/chromedriver.exe")
driver.get("https://github.com")
# driver.quit()
# use below to quit and clean.
conn = http.client.HTTPConnection(driver.service.service_url.split("//")[1])
conn.request("GET", "/shutdown")
conn.close()
del driver

This works well. No exceptions and chromedriver.exe closed. But I think this is not the best solution.

Upvotes: 1

undetected Selenium
undetected Selenium

Reputation: 193108

This error message...

    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

...implies that there was no response from the remote end and the connection to the remote end was closed.

Syntactically, there are no errors in your code block. However GitHub Home Page contains AJAX elements.

This usecase

Most possibly when the webpage attains document.readyState equals "complete" the AJAX calls are still active which implies the connection to the remote end was still not established completely. In such a scenario when you invoke driver.quit() the above error is raised.

Repro

When I executed your usecase I got a similar error on my localhost as follows:

  • Code Block:

    from selenium import webdriver

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get("https://github.com")
driver.quit()

  • Console Output:

    DevTools listening on ws://127.0.0.1:61488/devtools/browser/62c450cf-7ed5-4205-bb9c-d7e50cd13173
[2780:12796:0804/183946.157:ERROR:broker_win.cc(55)] Error reading broker pipe: The pipe has been ended. (0x6D)

Solution

For a cleaner execution of driver.quit() you need to induce WebDriverWait for some element only for the Javascript and AJAX to complete and then invoke driver.quit() as follows:

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get("https://github.com")
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//h1[text()='Built for developers']")))
driver.quit()

References

You can find a couple of detailed discussions in:

Upvotes: 1

Related Questions