CODEWITHSUNDEEP
pythonseleniumphantomjscelery
Ali
Ali

Reputation: 961

PhantomJS randomly does not exit (using Selenium with Python)

I'm using selenium-python with PhantomJS. The code is pretty much like this:

from selenium.webdriver import PhantomJS
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
driver = PhantomJS()
wait = WebDriverWait(driver, 10)
driver.get(url)
while True:
    // scrap the page
    try:
        driver.find_elements_by_css_selector('.next')[0].click()
    except: break
    wait.until(expected_conditions.visibility_of_element_located((By.CSS_SELECTOR, '.loading')))
    wait.until(expected_conditions.invisibility_of_element_located((By.CSS_SELECTOR, '.loading')))
driver.quit()

I use a celery task which runs this code periodically. The problem is that from time to time there are some stale phantomjs processes. When I look into celery logs the task is completed successfully without any errors but the phantomjs process is still running.

Some extra info:

Can someone suggest a way to debug and find out who's fault is this?

Upvotes: 5

Views: 891

Answers (1)

WKPlus
WKPlus

Reputation: 7255

I have not use celery before, are you sure celery will display all error message for you? For your code, I think if an exception occurs when running wait.until(...), driver.quit() will not be executed.

Upvotes: 2

Related Questions