CODEWITHSUNDEEP
pythonseleniumweb-scraping
Tom Callan
Tom Callan

Reputation: 65

Selenium only returns an empty list

I'm trying to scrape football team names from betfair.com and no matter what, it returs an empty list. This is what I've tried most recently.

from selenium import webdriver
import pandas as pd

driver = webdriver.Chrome(r'C:\Users\Tom\Desktop\chromedriver\chromedriver.exe')
driver.get('https://www.betfair.com/exchange/plus/football')

team = driver.find_elements_by_xpath('//*[@id="main-wrapper"]/div/div[2]/div/ui-view/div/div/div/div/div[1]/div/div[1]/bf-super-coupon/main/ng-include[3]/section[1]/div[2]/bf-coupon-table/div/table/tbody/tr[1]/td[1]/a/event-line/section/ul[1]/li[1]')

print(team)

Upvotes: 2

Views: 1978

Answers (1)

Moshe Slavin
Moshe Slavin

Reputation: 5204

You should use WebDriverWait. Also, you should use a relative xPath, not absolute xPath. One more thing you are using find_elements for a single element.

Here I'm printing all the teams 

from pprint import pprint
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

driver = webdriver.Chrome(r"C:\Users\Tom\Desktop\chromedriver\chromedriver.exe")
driver.get('https://www.betfair.com/exchange/plus/football')
teams = WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, '//*[@id="main-wrapper"]//ul[@class="runners"]/li')))
print([i.text for i in teams])

Output:

['Everton',
 'West Ham',
 'Tottenham',
 'Watford',
 'Chelsea',
 'Newcastle',
 'Wolves',
 'Southampton',
 'Leicester',
 'Burnley',
 'Aston Villa',
 'Brighton',
 'Bournemouth',
 'Norwich',
 'Crystal Palace',
 'Man City',
 'Man Utd',
 'Liverpool',
 'Sheff Utd',
 'Arsenal',
 'Eintracht Frankfurt',
 'Leverkusen',
 'Werder Bremen',
 'Hertha Berlin',
 'Augsburg',
 'Bayern Munich',
 'Fortuna Dusseldorf',
 'Mainz',
 'RB Leipzig',
 'Wolfsburg',
 'Union Berlin',
 'Freiburg',
 'Dortmund',
 'Mgladbach',
 'FC Koln',
 'Paderborn',
 'Hoffenheim',
 'Schalke 04',
 'St Etienne',
 'Lyon',
 'Nice',
 'Paris St-G',
 'Lyon',
 'Dijon',
 'Reims',
 'Montpellier',
 'Nimes',
 'Amiens',
 'Toulouse',
 'Lille',
 'Metz',
 'Nantes',
 'Angers',
 'Brest',
 'Bordeaux',
 'St Etienne',
 'Monaco',
 'Rennes',
 'Houston Dynamo',
 'LA Galaxy',
 'Philadelphia',
 'New York City',
 'Atlanta Utd',
 'New England',
 'Seattle Sounders',
 'Minnesota Utd',
 'DC Utd',
 'FC Cincinnati',
 'Orlando City',
 'Chicago Fire',
 'Montreal Impact',
 'New York Red Bulls',
 'Toronto FC',
 'Columbus',
 'Los Angeles FC',
 'Colorado',
 'FC Dallas',
 'Kansas City',
 'Shakhtar',
 'Dinamo Zagreb',
 'Atletico Madrid',
 'Leverkusen',
 'Club Brugge',
 'Paris St-G',
 'Tottenham',
 'Crvena Zvezda',
 'Olympiakos',
 'Bayern Munich',
 'Man City',
 'Atalanta',
 'Galatasaray',
 'Real Madrid',
 'Juventus',
 'Lokomotiv',
 'Ajax',
 'Chelsea',
 'RB Leipzig',
 'Zenit St Petersburg']

Upvotes: 2

Related Questions