CODEWITHSUNDEEP
pythonseleniumselenium-webdriver
Efan Mutembo
Efan Mutembo

Reputation: 15

How can I avoid ElementClickInterceptedException error when webscraping a dynamic webpage in selenium python

I'm scraping tables from a webpage and using selenium to click and open a new page to scrape tables consecutivly. I get this error every now and then. The error pops up at diferent times wile i ru the code but always origionates from the same line of code. Sometime I will be able to loop through 12 times and others only 6.



for i in range(0,len(season_tags)):

    """ 
    Step 1: Get to the correct Page

    """
    #Click to open the season dropdown
    driver.find_elements(By.CLASS_NAME,'current')[1].click()
    wait = WebDriverWait(driver,10)# wait for dropdown to apear
    wait.until(EC.presence_of_element_located((By.CLASS_NAME,'current')))
    
    
    #Click to chose a season
    season_tags[i].click()

    """ 
    Step 2: Scrape the data from the page
            - Format the data into a table
    """
    #Wait for table to appear
    driver.implicitly_wait(5)

    # Find table data and split it up
    table_data = driver.find_element(By.CLASS_NAME,'tableBodyContainer').text.split('\n')

    #Extract the data
    team_names = table_data[1::3]
    num_data = table_data[2::3]

    # Put the data into a dataframe
    data = pd.DataFrame([i.split(' ')for i in num_data])
    data['Teams'] = team_names
    print(season_tags[i].text)

    #Concatenate the table data
    frames.append(data)


lementClickInterceptedException          Traceback (most recent call last)
Input In [203], in <cell line: 4>()
     14 driver.implicitly_wait(5)
     16 #Click to chose a season
---> 17 season_tags[i].click()
     19 """ 
     20 Step 2: Scrape the data from the page
     21         - Format the data into a table
     22 """
     23 #Wait for table to appear

File ~\Anaconda\lib\site-packages\selenium\webdriver\remote\webelement.py:88, in WebElement.click(self)
     86 def click(self) -> None:
     87     """Clicks the element."""
---> 88     self._execute(Command.CLICK_ELEMENT)

File ~\Anaconda\lib\site-packages\selenium\webdriver\remote\webelement.py:396, in WebElement._execute(self, command, params)
    394     params = {}
    395 params['id'] = self._id
--> 396 return self._parent.execute(command, params)

File ~\Anaconda\lib\site-packages\selenium\webdriver\remote\webdriver.py:429, in WebDriver.execute(self, driver_command, params)
    427 response = self.command_executor.execute(driver_command, params)
    428 if response:
--> 429     self.error_handler.check_response(response)
    430     response['value'] = self._unwrap_value(
    431         response.get('value', None))
    432     return response

File ~\Anaconda\lib\site-packages\selenium\webdriver\remote\errorhandler.py:243, in ErrorHandler.check_response(self, response)
    241         alert_text = value['alert'].get('text')
    242     raise exception_class(message, screen, stacktrace, alert_text)  # type: ignore[call-arg]  # mypy is not smart enough here
--> 243 raise exception_class(message, screen, stacktrace)

ElementClickInterceptedException: Message: element click intercepted: Element <li role="option" tabindex="0" data-option-name="..." data-option-id="210" data-option-index="5" class="">2018/19</li> is not clickable at point (245, 437). Other element would receive the click: <li role="option" tabindex="0" data-option-name="..." data-option-id="274" data-option-index="4" class="">2019/20</li>
  (Session info: chrome=106.0.5249.119)
Stacktrace:
Backtrace:

Upvotes: 0

Views: 145

Answers (1)

Lucas
Lucas

Reputation: 23

I guess you can use exception treatment to avoid code stop working.

try:
    #Click to chose a season
    season_tags[i].click()
except Exception as e:
    print(e)
    pass

Hope it helps!

Upvotes: 1

Related Questions