Reputation: 11
Selenium, Twitter Web Crawling Using Python It runs on the local server without any problems This error occurs when running on a remote server. I'm not sure why this problem is happening. How can we solve this problem?
File "twitter.py", line 435, in <module>
twitter_walk()
File "twitter.py", line 31, in twitter_walk
driver = webdriver.Firefox(executable_path="./geckodriver",options=options)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
keep_alive=True)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status signal
ubuntu 18.04.5
selenium 3.141.0
Mozilla Firefox 86.0
geckodriver 0.29.0
Upvotes: 1
Views: 4475
Reputation: 2557
This is likely because you are attempting to run the browser in non-headless mode, but your remote server doesn't have a visual display.
To circumvent this, try running Selenium in headless mode by adding the --headless
argument to the browser.
from selenium import webdriver
from selenium.webdriver import FirefoxOptions
opts = FirefoxOptions()
opts.add_argument('--headless')
browser = webdriver.Firefox(options=opts)
browser.get('https://example.com')
This answer has more information.
Upvotes: 1