CODEWITHSUNDEEP
pythonseleniumphantomjsaws-lambda
Aymon Fournier
Aymon Fournier

Reputation: 4433

httplib.BadStatusLine: '' with Selenium and PhantomJS

I'm having this error loading a URL, and getting a strange error I have no idea how to fix. My situation demands that I must use PhantomJS, as I do not think I can use the Firefox drivers on AWS Lambda, and in my scraping I come across a button the Chromedriver cannot click.

If I switch PhantomJS out for Chrome or Firefox, the url resolves fine.

Using selenium==3.4.1 and PhantomJS 2.1.1

user_agent = (
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) Gecko/20100101 Firefox/53.0")

dcap = dict(DesiredCapabilities.PHANTOMJS)
dcap["phantomjs.page.settings.userAgent"] = user_agent
dcap["phantomjs.page.settings.javascriptEnabled"] = True

browser = webdriver.PhantomJS(service_log_path=os.path.devnull, service_args=[
    '--ignore-ssl-errors=true'], desired_capabilities=dcap)

browser.set_window_size(1120, 550)
browser.get('https://drizly.com/session/new')



File "main.py", line 257, in <module>
    lambda_handler(None, None)
  File "main.py", line 103, in lambda_handler
    browser.get('https://drizly.com/session/new')
  File "/Users/aymon/Envs/drizly/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 264, in get
    self.execute(Command.GET, {'url': url})
  File "/Users/aymon/Envs/drizly/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 250, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/Users/aymon/Envs/drizly/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 464, in execute
    return self._request(command_info[0], url, body=data)
  File "/Users/aymon/Envs/drizly/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 526, in _request
    resp = opener.open(request, timeout=self._timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1227, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1200, in do_open
    r = h.getresponse(buffering=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1132, in getresponse
    response.begin()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 453, in begin
    version, status, reason = self._read_status()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 417, in _read_status
    raise BadStatusLine(line)
httplib.BadStatusLine: ''

Upvotes: 2

Views: 689

Answers (1)

keepAlive
keepAlive

Reputation: 6655

BadStatusLine is a subclass of HTTPException, which is raised if a server responds with a HTTP status code that we don’t understand. You may want to catch it, as follows

#...
browser.set_window_size(1120, 550)
try:
    browser.get('https://drizly.com/session/new')
except httplib.BadStatusLine as bsl:
    print('[!!!] {message}'.format(bsl.message))
#...

Note that a good practice is that an error should never pass silently. Hence this use of print.

Upvotes: 0

Related Questions