CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
luka
luka

Reputation: 529

Can't scrape a class with BeautifulSoup

I want to scrape this website https://www.flashscore.com/football/uruguay/primera-division-2020/results/ but I can't scrape a class with score data

Code:

    table_ = soup.find('div', id='live-table')
    id_ = table_.find_all('div', title='Click for match detail!')


    currentYear = datetime.now().year
    for _ in id_:
        date_ = _.find('div', class_='event__time').text.replace('.', '/') + str(currentYear)
        home = _.find('div', class_=re.compile('event__participant--home')).text
        away = _.find('div', class_=re.compile('event__participant--away')).text
        try:
            hg = driver.find_element_by_class_name('event__part event__part--home event__part--regulation').text
            ag = driver.find_element_by_class_name('event__part event__part--away event__part--regulation').text
        except NoSuchElementException:
            hg = driver.find_element_by_class_name('event__score event__score--home').text
            ag = driver.find_element_by_class_name('event__score event__score--away').text

Error

 hg = driver.find_element_by_class_name('event__part event__part--home event__part--regulation').text
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".event__part event__part--home event__part--regulation"}

During handling of the above exception, another exception occurred:

    hg = driver.find_element_by_class_name('event__score event__score--home').text
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".event__score event__score--home"}

enter image description here

Upvotes: 0

Views: 138

Answers (1)

Mateusz Dorobek
Mateusz Dorobek

Reputation: 761

  1. If you are web scraping try to find elements by id or at least class name instead of titles or names.
  2. I would recommend checking if the element you are looking was found. assert table is not None, 'ERROR: table not found'
  3. It seems to be a dynamic page, so the HTML file that you get in response doesn't include the desired table. You need to use a library that will allow you to dynamically load the full content of the webpage and scrape it. I would recommend Selenium.

Upvotes: 2

Related Questions