CODEWITHSUNDEEP
pythonseleniumselenium-webdriverweb-scrapingwebdriver
Ranjan Pandey
Ranjan Pandey

Reputation: 85

Element not clickable - error work around not working as expected using Python and Selenium webdriver

While trying to scrape review data from tripadvisor's new website, am trying to click on the more button to expand the reviews before picking them up. The code works fine on the first page, but throws error on the subsequent pages. Here's the part of code that I am using:

for num in range(page_count):
    try:
        if num != 0:
            try:
                nxt = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.CSS_SELECTOR, "a.nav.next.arrowNav.taLnk")))
                #nxt = driver.find_element_by_css_selector("a.nav.next.rndBtn.ui_button.primary.taLnk")
                nxt.click()
                driver.implicitly_wait(5)
            except NoSuchElementException:
                print("this is a NEE1")
                driver.refresh()
                nxt = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.CSS_SELECTOR, "a.nav.next.arrowNav.taLnk")))
                #actions = ActionChains(driver).move_to_element(nxt).click().perform()
                driver.implicitly_wait(5)
            except WebDriverException:
                print("this is a WDE1")
                #nxt = driver.find_element_by_css_selector("a.nav.next.rndBtn.ui_button.primary.taLnk")
                nxt = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.CSS_SELECTOR, "a.nav.next.arrowNav.taLnk")))
                actions = ActionChains(driver).move_to_element(nxt)
                driver.implicitly_wait(2)
                actions.click()
                actions.perform()
        try:
            more = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "span.taLnk.ulBlueLinks")))
            #WebDriverWait(driver, 20).until(EC.invisibility_of_element_located(By.CLASS_NAME, 'loadingWhiteBox'))
            #more = driver.find_element_by_css_selector("span.taLnk.ulBlueLinks")
            #more.location_once_scrolled_into_view
            #driver.execute_script("arguments[0].scrollIntoView();", more)
            more.click()
            time.sleep(1)
        except TimeoutException:
            print("There is no 'more' button on page %d" % (num+1))
        except WebDriverException:
            print("this is a WDE2")
            more = driver.find_element_by_css_selector("span.taLnk.ulBlueLinks")
            more = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "span.taLnk.ulBlueLinks")))
            #more.location_once_scrolled_into_view
            #driver.execute_script("arguments[0].scrollIntoView();", more)
            #WebDriverWait(driver, 20).until(EC.invisibility_of_element_located(By.CLASS_NAME, "loadingWhiteBox"))
#            actions = ActionChains(driver).move_to_element(more)
#            driver.implicitly_wait(2)
#            actions.click()
#            actions.perform()
            time.sleep(1)

        review_result = WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.CLASS_NAME, 'entry')))
        review_date = driver.find_element_by_css_selector("span.ratingDate.relativeDate").get_attribute('title')

        with open('input.txt', 'a') as fid:
            for date,review in zip(review_date,review_result):
                         fid.write(unidecode(review_date))
                         fid.write(sep1)
                         fid.write(unidecode(review.text))
                         fid.write(sep)
                         fid.write(line_break)

Below is the error:

> WebDriverException: unknown error: Element <span class="taLnk
> ulBlueLinks"
> onclick="ta.prwidgets.call('handlers.clickCollapse',event,this);">...</span>
> is not clickable at point (292, 615). Other element would receive the
> click: <div class="loadingWhiteBox"></div>

I looked up to a number of posts related to same issues and found out few work around for the error and tried them all. The only thing that seemed to be working was using ActionChains, which is currently uncommented in the code. It doesn't throws any error but it wouldn't actually click on the more button and expand the reviews. Moreover, the code processes till a certain page and then get stuck on the same page till the loop ends picking same reviews again and again. I am unable to find any solution for this issue. Please help.

The element is attached to the tag

span.taLnk.ulBlueLinks

Upvotes: 0

Views: 244

Answers (2)

Prateeksha Chauhan
Prateeksha Chauhan

Reputation: 68

This occurs due to page refresh or changes when driver attempt to perform action on the element. The location of the element has changed after the driver has stored its reference and now some other element is present at the desired location

We may use Javascript to scroll to that particular element and the perform the action:

WebElement element = driver.findElement(By.id("id_of_element"));
((JavascriptExecutor) driver).executeScript("arguments[0].scrollIntoView(true);", element);

Upvotes: 1

Murthi
Murthi

Reputation: 5347

It looks like loading white box web element is overlapping the more button. From your code, I have seen the code to wait for invisibility of white box. Please put this code before clicking on more button as given below. 

WebDriverWait(driver, 20).until(EC.invisibility_of_element_located(By.CLASS_NAME, 'loadingWhiteBox')) 
more =driver.find_element_by_css_selector("span.taLnk.ulBlueLinks")
more.click()

Please try to increase the timeout if you get timeout exception.

Upvotes: 1

Related Questions