CODEWITHSUNDEEP
pythonseleniumparsingselenium-webdriverselenium-chromedriver
etcTryAgain
etcTryAgain

Reputation: 1

Problems with chromedriver on Selenium with python: page never loads

I originally wanted to get a headless browser running to dynamically parse a web page. I've been having a series of problems with this, even after adding in waits to allow website to load. After commenting out the headless option, I used an xming server as a display for the web page to see what it actually looks like after the browser is created.
Here's a screenshot of what I see: a new tab in chrome, and the browser seems to never get to the .get(). I also get a popup saying "Chrome Automation Extension has crashed..."
I've been searching a lot, some suggest not running as admin/root, so I didn't use sudo before running it. That didn't change message.

Below is my code and the terminal output:

options = webdriver.ChromeOptions()
# options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--lang=en_US')
options.add_argument('--user-data-dir')
options.add_argument("--disable-gpu")
# options.add_argument("--remote-debugging-port=9222")

browser = webdriver.Chrome(executable_path='/usr/bin/chromedriver', options=options)
browser.get("http://www.google.com")

Ubuntu terminal:

Traceback (most recent call last):
File "practice3.py", line 40, in
browser = webdriver.Chrome(executable_path='/usr/bin/chromedriver', options=options)
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in init
RemoteWebDriver.init(
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in init
self.start_session(capabilities, browser_profile)
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: DevToolsActivePort file doesn't exist
(Driver info: chromedriver=2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706),platform=Linux 4.4.0-18362-Microsoft x86_64)

Also:
Ubuntu 20.04 LTS
Selenium version 3.141.0
Chrome version 84.0.4147.105

~$ /usr/bin/chromedriver
Starting ChromeDriver 2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706) on port 9515
Only local connections are allowed.

Upvotes: 0

Views: 1828

Answers (1)

Eduardo C
Eduardo C

Reputation: 11

I've seen lots of chrome crashes when there was version mismatch between chromedriver and chrome. Since you're using Chrome version 84.0.4147.105, you should be using nothing but ChromeDriver 84.0.4147.30 Second thing, are you launching the script from an IDE ? if positive that is also another common root cause. Finally I'd stop using XMimg since it adds another variable to the equation. Instead, troubleshoot taking screenshots directly from selenium code regardless you are headless or not.

Upvotes: 1

Related Questions